Hattie, Homework and an Insightful Analytical Blog Post (and Cautionary Tale)

Our school is struggling with the issue of homework.

Recently, we have moved away from it.  Some parents equate more homework with academic rigor and quality.  Of course, the data says otherwise–but also supports that community perception of the school goes up when the homework load goes up.  Now, the rallying call is for a research based policy on homework.

I often start with John Hattie and his file reviews and meta-analyses because the guy covers everything.  From his file reviews you can also find great original research worth pursuing.

Rather than rehash, let me guide folks to an insightful analysis of Hatties’ conclusions in “Homework: What does the Hattie research actually say?” on Tom Sherrington’s headguruteacher blog.  Sherrington does a great job looking beyond the overall effective rating of d = 029, which implies that homework is not an effective (Hattie argues that a practice is “effective” when above d = 0.40, and even that statement is a simplification of his work).  Note: Sherrington’s analysis begins a bit technical, a reflection of Hattie’s technical meta-analysis that he’s analyzing.  Wade through it–it’s worth it.

In short, Sherrington notes that Hattie reports that homework is ineffective at the primary grade level (d = 0.15), but quite effective at the secondary level (d = 0.64).  Of course, these results have caveats about type of work and the like.  Read the article, as Sherrington goes in-depth with the details and their implications.

But Sherrington’s analysis is also instructive when we look at bias in analysis and reporting.  Up front, he makes his pro-homework views clear and shares a link to another piece of his, ‘Homework Matters: Great Teachers set Great Homework’.  The analysis that follows is, as I’ve said, pretty solid.  Until the end.

All of this makes sense to me and none of it challenges my predisposition to be a massive advocate for homework.  The key is to think about the micro- level issues, not to lose all of that in a ridiculous averaging process.  Even at primary level, students are not all the same.  Older, more able students in Year 5/6 may well benefit from homework where kids in Year 2 may not.  Let’s not lose the trees for the wood!  Also, what Hattie shows is that educational inputs, processes and outcomes are all highly subjective human interactions.  Expecting these things to be reduced sensibly into scientifically absolute measured truths is absurd.  Ultimately, education is about values and attitudes and we need to see all research in that context.

Let’s ignore the statement “All of this makes sense to me and none of it challenges my predisposition…” which is the definition of denial (or perhaps cognitive dissidence).  You would think, at least, Sherrington would concede that at the primary level homework offers little benefit (remember: d = 0.15 vs. d = 0.40 entering the effective range).  No.  Instead, he qualifies, “Older, more able students in Year 5/6 may well benefit…”  That word: May.  Ugh.

Through such phrases a bus runs through.  After presenting, breaking down and analyzing the data he a) ignores the data that refutes his viewpoint, b) presents an alternative based on no data.  You cannot do that!  It’s not good science!  If Sherrington wants to use his theory as a basis for more research, great.  Instead, he simply argues that the data says one things, he believes another and so he’s going with his gut.  At least he’s transparent about it.

But I have been in countless meetings and conversations like this: Someone concedes that a practice is not effective and then justifies its continuation.  Parents and educators will often come with opinion pieces that seem like common sense, but with little data.  With a little research these views are often refuted, and sometimes quite harmful practices.  Yet, the idea will persist.  As professionals and organizations we refuse to trust data–or even consider it.

The posts to Sherrington’s piece are a typical reaction to any presentation of data that challenges orthodoxy.  Some accept it, but many qualify their views and use that data Hattie presents in an interpretive way.  Again, a good study in identifying bias.

My suggestion in such situations is to turn it around.  First of all, if some primary kids “may” benefit, might it also be said that some secondary kids “may not” benefit?  Do they get penalized?  What’s the plan for them?  Remember–when there is a majority, there is also a minority.  Why do we make policy when the majority agrees with us, but ignore the majority when they disagree?

Second, all choices have consequences: When you choose one thing, you are giving up another.  To have homework, students and teachers are giving up something.  That might be something simple like time–a student with thirty minutes of homework loses thirty minutes of play time, for example.  Is the loss worth the gain?  Hattie’s analysis seems to indicate that little is gained at the primary level, so any loss might not be worth it.  Educators should focus on the trade-off.  Is it worth it?

For Sherrington, he seems as interested in getting in the content and practice in the face of limited class time.  He is trading off the student’s time for academic gain.  That might be a fair trade-off, especially if the students are part of the decision making process.  But there might be other inefficiencies in Sherrington’s lesson planning that can be exploited (I have no idea) that a student might want addressed before they give up their after-school time.

Is it worth it?  Really, that’s the essence of all of this data analysis.  Sherrington needs to respect that data a bit more instead of dismissing it.

Advertisements

2 thoughts on “Hattie, Homework and an Insightful Analytical Blog Post (and Cautionary Tale)

  1. Hi. It’s interesting for my post about Hattie to be discussed in this way. Far from dismissing the data, I think I’ve shown it the respect it deserves – and no more. Hattie himself suggests that he doesn’t think his data is argument against homework; it’s just a measure of how effective homework has been on average in different studies. My assertion that ‘older, more able students in year 5/6 may benefit’ is an extrapolation from Hattie’s analysis. In Visible Learning he goes to some lengths to explore the variation between effect sizes from different studies; older, more able students do in fact show bigger gains from homework. There’s every reason to suggest that this applies in primary as well as at secondary. ‘May’ is perfectly valid in any data-led discussion because, within the limits of significance and at the ends of a range from which an average is taken, there are all kinds of possibilities that are worthy of consideration. Your school has decided to move away from homework; it reads to me as though you’re looking to justify that decision. In truth, the data only tells you that setting homework in the ways it has been done in some studies, may not have been worth it compared to doing some other things instead – on average. I’d have thought the most sensible thing is to set the kind of homework that has a good chance of making a difference for the students who are likely to benefit. That’s what Hattie is saying; he’s not saying it’s a good thing or a bad thing. The data tells that story pretty clearly and, to reiterate, I haven’t dismissed it; I’m just looking beyond the averages to the underlying detail.

    Like

    • You seem to be looking for the exception to make the rule. Your arguments are exploiting gaps in the data. You may be right, and experience may bear it out, but because the data does not speak against all does not mean it argues for some. Hattie’s statements are…. guarded. He doesn’t close the door, but….

      One question I have is what a “benefit” for a mature primary students would look like? At present, the United States is embracing the standards movement. Those students, I imagine, are already at the top of the scale. While any students who is motivated will always extend their learning, homework or not, I’m wondering about measurement. From what to where? Enrichment is something every student can benefit from, and certainly embrace such a programs, but the measure of “benefit” here is certainly a different experiment than simple growth of the average student in very specific subject measures. It seems you are speaking more of enrichment than traditional homework.

      As for seeking to justify my school’s decision not to assign homework, I’m on the fence (so are our faculty and community). I’ve read Hattie’s comments about effective assignments. We do assign specific types of assignments, such as nightly reading and some practice math problems. Of our population, our successful students have many extracurricular activities and don’t need the stress or work, while our struggling students who might benefit won’t do the work without help and supervision. So, we do it differently.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s