Counterfactual Questioning of Data and Needs

9780812973815I have been reading Nassim Nicholas Taleb’s The Black Swan: The Impact of the Highly Improbable after reading a Wikipedia article about Stephen Bannon and his influences.  This was one, and it made the news clearer in many ways.

It also showed me a different way of using the data we are now getting almost too much of.  Here are a few interesting take-aways as how I might look at data new–and use it moving forward.

1. Look at the outliers.  When we focus on what works, or decide that it does not work, we often don’t really look at who falls short.  We look at the bulk–the middle kids–and evaluate the program based on them.  Then, we look at how we can “fix” the others to fit into the system which (we think) works.  Or blame them for their shortcomings.

It becomes about fitting a square peg into a round hole.

For some, we intervene with Tier II or even go EST (Educational Support Team), if they don’t already have IEPs or a 504.  For others, we explain it away (often with history–they’ve always done that–or other–family issues, anxiety, a bad day….) which is another version of blame-the-kid.  But if 20% of the group is not succeeding, how can we call a program a success?  (FYI, I’ve seen programs at 40% success declared working, because you have to understand…)  And if the response is that those 20% get something different, a) does that work, b) is that the best use of resources, and c) is that best for all kids?

One concern about any program outside of the normal program is measuring effectiveness of that intervention.  Often, if the intervention works–great work–but if it fails the child we find excuses with the child.  Honestly, once a kid falls to Tier II or Tier III they stay there.  (There is little incentive to change that, but that’s addressed later).  So the first step needs to be coupled with….

2. Tailor the group to the outliers.  While a program should not revolve around a single student, are there changes that would help other students as well?  For example, in the talks about anxiety our school did the presenters sold the idea that such interventions help other students.  True.  Compared to our childhood, we no longer time tests while we do allow movement breaks, reteach, and incorporate a dozen other techniques that began with outlier kids and are now mainstream.  But what is required is….

3. Thinking of needs outside the standard toolbox.  My students, for example, started the year talking about needing “hands on” learning (whatever that means).  Then, it was movement breaks.  Then, social time.  All of these were valid, but they did not really have an idea of what they needed.  Instead, our team read between the lines and gave them what they actually needed.  But even then, our responses were pretty standard (more projects, less seat work).  And, we are having a difficult time with a different outlier group (we are working on working independently, working with peers, time management and other, new issues).

Recently, we have looked at other needs.  We have a few kids questioning their identity (sexual orientation, interests, even their given name).  We have several kids with family issues–really worried about their family, illnesses, moving, relationships and the like.  We can see with after school groups that a desire for identity and connection exists.  A social survey with gave students showed a desire for help with peers and impulse control.  What to do with that?  We are currently looking beyond the wisely constructed groupings to regroup with needs, and soft-sell the focus bringing them together now.

But we aren’t counselors and this is not a therapy session.  Our goal is academics.  Plus, there are other needs and other solutions.  For example, in 2014 I found that what moved scores was participation in a competitive activity (AAU, Mini Metro, Far Post, ballet…).  There are a number of needs, many you don’t see because our students are faces in a specific location and a data point (go out to recess or the common spaces and see them differently).  So, we find that need by….

4. Ask counterfactual questions.  In a counterfactual assessment you prove what is wrong, because it is easier than proving what is right.  For example, I can’t prove that coming to school daily works, but I can easily show that students who are tardy and miss a lot do poorly.

So, what do we know does not work?  What stops learning.  Time is a general issue.  Interruptions are more pressing.  Homework does not really help those we want it to help.  But what the homework issue shows us is that time is needed–to read, for example.  When we do get kids to read at home, they get better at reading.  Now, we don’t know if that is due to attitude or practice or what.  But by putting SSR (Sustained Silent Reading) in the school we can focus on that.  For those who is does not work, why?  Seating (move them), book selection (look at level and keep trying new titles), fluency (Google Text to Speech)….  And as we take away what does not work we are left with reading.  But you need to ask….

5. Instead of asking what they need, ask what they don’t need and shed it.  Does a student who reads all of the time need SSR?  They might like it, or it may serve a calming function, but it does not add much to the academic purpose of SSR for them.  Found time.  They might, though, need that calming function–can it be given more effectively (offer meditation or counseling)?  Or, when their non-needs are whittled away, what’s left?  You can ask this of the student who gets instrument lessons outside of school (why do they still get school lessons?), the kid who plays Mini-Metro during the basketball unit (is relearning how to dribble worth their time?), and so on.  And we know who is lacking–we can tick it off the top of our head.  We have a list ready of what is not working.

A simple example is leadership.  We say we need it.  In our school, we have a group of intelligent athletes who think leadership is getting their way, instead of making their peers look better and rise up together.  They pick the team, have the ball, take the shot, and are ready with the excuse of why X didn’t get the ball (“They aren’t very good.”)  They are teachers waiting to be shown how.  If they teach X, that kid X will get the ball.  A simple pivot around the data sets that up–Kid A is measured on basketball skills and Kid B is measured on his teaching others.

6. And before you talk about time, think about how much time we spend accommodating, chasing down a few kids, and adapting work.  Differentiation is hard and time consuming and still we have failure–because we differentiate the wrong thing.  Use data to find the right thing, but cutting back everything that does not work.

Of course, this is still turning around in my head.  But it might cause you to think about things differently as you approach configuration, schedule and PBL in the next month in anticipate of the next school year.


One thought on “Counterfactual Questioning of Data and Needs

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s