Big Picture, Little Picture Data

My admins have begun to ask good questions that require data to form the answer, and as a result have given me access to vast amounts of data to explore with.  After answering their initial questions, one admin began to talk about using data in a Response to Intervention (RtI) type model.  A big believer in RtI, I find I have little interest in playing with data around it because it seems pretty straightforward and not particularly interesting from an analysis standpoint–no real toys to play with; you just see something, respond and remeasure.

It did make me wonder about micro vs. macro uses of data.  In this EdX course I am taking (Data, Analytics and Learning) they touch on Learning Analytics vs. Academic Analytics; that is, looking at a set of students and the teacher in a classroom, as opposed to the administrative and larger organization level.  Often, because conversations about data are so rudimentary, we blur the two.  At our school, we slap the NCLB numbers on the whole school in one bubble and then look at individuals in another bubble (if we look at data at all).  If we could, many would use a single number and call it analysis.

I find there are three types of people who look at data:

  • Micro data users: These are RtI folks, but also anyone who looks at individual students and assignments and uses that data to help that student, the class or general, personal instruction.  Many elementary teachers fall into this.  In fact, they are very good at this–pioneers!
  • Macro data users: We are interested in systems, starting with the classroom, but then scaling up results to schools and beyond.  If you can change the system for the better, there exists a higher, more solid foundation for the micro folks to build on.  Few teachers are here (unfortunately, few admins are here).
  • Non data users: These are the folks who know what they know (or think they do). From their point of view, when the data does not confirm, it’s flawed and useless.  When it does confirm, it’s a redundant waste of time.  I’m throwing half of all teachers and admins, and many secondary teachers, into this lot.

Of course, I have no actual data to support these conclusions (ha, ha).  It did, though, make me wonder which focused efforts–macro or micro–show best results.

In looking at the data that was given to me, I was able to track how cohorts did compared to other schools in our district.  I find it helpful to think in terms of sports: If a cohort beats the district average we “won” and if we did not, we “lost”.  Looking at six cohorts over three years, two cohorts clearly had something going on–they had “winning” records.

When I looked at teachers, only two seemed to have “winning” records.  One was me.  The other had those winning cohorts two of their three years, so it is hard to discern if they are a good teacher or just got good kids–the results of that third year indicates the former.

Beyond tooting my own horn, it has made me think about my program.  A big picture person, I tend to avoid the trivialities of my subject–spelling, grammar, historical dates and the like–and focus on large trends, like being coherent, doing analysis and having the reader understand the content.  It has been a struggle.  Besides a raft of coordinators demanding spelling programs, I want to do right by my students.  Cultural currency is important–our ideas will not be judged if the reader dismisses your work due to spelling errors.  I find, though, that the resources allocated to such things is better spent elsewhere–the kids who can’t spell can’t do plenty, while the others don’t need the spelling program.

Instead, I look at long timelines.  Until last year, I taught multi-age.  I had a two year timeline; students often came back from the summer seemingly smarter, and tested great before leaving me for good.  Success!  Success?

At this point, I am struggling with the balance.  Before you respond “both” to the question of micro or macro, know that choices have consequences–what you choose is also you not choosing something.  Imagine a spectrum where, on one end, we did 100% spelling.  Now, slide away form that so you can include grammar, then writing and then perhaps a little reading.  As you add, you approach zero spelling.

Question: Is there a point where you do so little spelling that it really isn’t worth it to do any?  That’s where I am (I don’t do any).

Question: What, then, is the point of diminishing returns where input and output are maximized?

Question: In plotting out the returns of all areas needed to be covered, which has the “most bang for the buck”?

In economics, this is called the “opportunity cost” of choices–what do you lose when you make a choice, and is that loss less than the choice you went with?

All a good struggle to have.