Counterfactual Reasoning, Black Swans, and Common Planning Conundrums

There are many nuts-and-bolts topics discussed (which is probably why you are here) in the name of counterfactual thinking and black swans.  All interesting, but you may by here for only one, so scroll down to what interest you.  In order, the topics are: Introduction, Configuration, Schedule, Placement, Last Thoughts.  Enjoy.

Counterfactual Reasoning and Black Swan Planning

A few weeks ago I wrote about counterfactual reasoning–that it is easier to prove something doesn’t work than it is that is does work.  For example, it is hard to prove that coming to school leads to learning, but we know truants fall behind.  This is important to remember as we plan–we can’t confirm much about the future, but we can plan with that blind spot in mind.

The idea becomes important when looking at the surprises we seem to encounter after we do a lot of planning for the next year (placement, schedule, resource allocation, etc.).  The theory of this is the “black swan“; that unexpected events have a major effect on plans.  Because of this, the theory goes, we should plan with the unexpected in mind.  The other half of the theory is that it is difficult to know what to plan for.  In short, just be ready for something.

Note: Much of this is inspired by  Nassim Nicholas Taleb’s Black Swan: The Impact of the Highly Improbable.  Much of this essay looks at the unexpected events that cause those disruptions every educator can relate to–nothing Earth shattering, but annoying nevertheless.

As educators, we tend to lock everything down–protocols, lessons, spaces, schedules, class lists–because we like to think we can predict everything.  For example, our school locked students into the three homerooms last June, but we did not take into account several unexpected developments.

The largest was we had asked students their World Language preference (French or Spanish).  Our students move through the school by classroom–Art, PE, Music and World Language.  During the summer we realized we could not honor most preferences because placement was based on a different criteria.  We punted.  But we also did not expect several students from one classroom to move away or that so many new families would move into our district.  We placed with insufficient information.  Then student preferences for WL created more turmoil, as parents demanded their kid get this or that language (they, of course, waiting for three weeks into September before doing so).  It also took a few weeks to figure out the WL/Tier II needs of new and leaving students.  Who knew?

As the old Yiddish proverb goes, “Man plans and God laughs.”

One of the mistakes people make is in trying to anticipate the black swan.  If we only anticipate more contingencies we think we can control it all.  No.  We certainly can straighten out many issues.  And we still have to make plans–schedules, room assignments, placement, etc.  How, though, can we plan while respecting the unknown?

Here are some examples of common issues facing schools.

Configuration

A few weeks ago I shared a configuration idea for our middle school.  For forever they have been multi-age, but they are contemplating going straight-grade “just for next year” because class sizes are getting too large and students are falling through the cracks.  There are several challenges facing the team, but the main one will be the influx or exodus of students.  I argue maintaining multi-age classrooms is the best safeguard against student numbers changing.

They are adding a fifth teacher to a four person team.  The Core is responsible for Language Arts, Math, Science and Social Science.  Currently, two teachers are licensed to teach STEM and two Humanities.  To alleviate class sizes they plan to hire a Science teacher.  In theory, this lowers the class average from 28 students to 23.  Of course, because the new teacher can only teach Science it does nothing to help the Humanities classes .  In theory, one of the STEM teachers will shift some of her load towards Math, but Math being taught straight-grade and other elements conspire against such neat divisions.

Using the our middle school’s straight-grade plan as an example, three teachers for the 7th and two for the 8th creates an automatic imbalance.  Of course, these can be mitigated (supports, aides, schedule).  But, should five students leave the 7th, or five join the 8th (or both), there is suddenly a difficult situation with no easy fix.  We can’t know if that scenario will play out, or be worse.  The plan, though, is crafted with little wiggle room should any black swan appear.

Using counterfactual thinking, instead create a multi-age made up of two teams.  Each team has a Science and Humanities teacher.  A single Math teacher instructs all students–a big load, but she is relieved of other duties and the two teams can focus on social-emotional middle school programming.

Imagine, now, five students leave the seventh–the release would be divided between two teams, offering a slight relief of two or three on each team.  In the case of the five joining the 8th, a much more devastating event, that burden would be shared and only result in two or three each.  In fact, if both scenarios play out, much of the effect is a wash.  Multi-age benefits can mitigate the other effects of population bubbles.  For example, each year teachers move (with their classrooms) and they need to relearn new curriculum and start new team protocols, etc.  Multi-age allows multi-dimensional flex.

But the question to ask is why that plan will not work?  I have shown the problems of straight grade (there are more, unwritten here), but that does not mean multi-age is better.  The black swan, counterfactual advice is to take each option and poke at it–with some of the more outlandish variables–until it reveals its flaws.  Then, put down other ideas, too, and poke them.  One area to look at is which “losses” are most tolerable and which would be devastating.  The plan with the most resiliency is probably your best bet.

Schedule

The difficulties of crafting a schedule are long and well known (we even know that we don’t know enough).  This is especially true in schools with many grades and diverse needs (we are a pre-K through 8).

I actually learned a lot recently on why crafting a schedule is so hard through an article on chaos theory (an article for later).  In short, such endeavors are dynamic and resetting course causes exponential problems down the path.   What we do know is that schedule craft works up to 80% of the elements are placed, but then conflicts trouble the last 20% of needs (that’s the dynamic nature of the process).  To fix it we begin to make compromises (which creates more conflict, exponentially).  Each year, someone (or many) is (are) unhappy.

Then, each year, we have a black swan surprise.  Last year, it was WL and everything that followed it.  Other grades had other emergencies.  Each fix seemed to bring more problems.  We live with it–we have no choice–but that 20% consumed a lot of resources and continues to create ripples (e.g., it is unclear if student preference for WL will be honored in 7th grade now that they have a year of one language under their belt.).

My solution is blocking out time instead of scheduling in time.  For example, one grade level figures four uninterrupted hours of seven school hours are needed for Core–90 minutes of Literacy, 60 minutes for Math, 60 for Science/Social Science and 30 for TA.  We demonstrate what’s important to an organization by the resources we put towards it (time, money) and four hours uninterrupted seems a reasonable commitment.  That leaves three hours for lunch, recess, Unified Arts (UA), band, mentor, counseling and any other non-Core activity.

How is a block resilient?  Let’s look at how our current system is not.  Predicting a seven hour day is hard; there are many variables.  It is also easy to be fooled by small numbers–three kids pulled out here, four there.  Currently, music lessons, mentor, speech and physical therapy, and other student needs are pulled from Core.  There are many reasons for this–part time employees and mentors have locked-in schedules, no one wants to deny a child their one Art class a week, and a few kids here or there are seen as non-disruptive.  But in my class of 20, one absence and three music lessons are 20% of the class.  Plus, recovery now becomes a negotiation not with two other Core teachers, but with three times that–pull from Art? PE? (And, because of the dynamics of the interwoven schedule, also lose 20% of their students and need recovery for them).  Again, here is the problem I am poking a hole in with our current schedule and how it was created.

By dividing the day into two blocks (4 hour, 3 hour) that do not overlap, students and teachers only have to negotiate within those blocks, with only those adults.  It is more likely I can flex with my Core teammate’s schedule because a) we have an aligned schedule, b) fewer pulls (classes) means fewer and smaller disruptions, c) fewer agents means less communication needed.  On Core, we can debate the virtues of Math vs. Science and time required to do it right, but with blocks we do not have to debate the death by a thousand cuts we currently face.  Four hours uninterrupted shows we value Literacy, Math, Science and Hive.  Three hours should be enough for others to negotiate what is best for students, among actors who have more in common (UA).  Or, to demonstrate that we want more for our students than the resources we have (or are willing) to give.  Again, this may be flawed.  Poke it.

Student Placement

Has there not been an issue with placement?  Do kids every come as advertised, stay friends (or enemies) with the kids from the previous year, or continue to need the same services?  Too often, we’ve even had to change the teacher assigned to the class (illness, pregnancy, number shifts).  For all the benefits we argue placement provides, those are the holes I poke in our locking kids in as we do.

Three years ago, the middle school wisely created two teams with a blank slate for placement.  First, Special Ed (SE) placed students on the two teams with a variety of rationales–service providers, friend/tormentor locations and the like.  This was open to debate/poking by the group, but we deferred to the experts.  Second, a small number of kids who do not receive services were separated or paired because of behavior or emotional needs.  Some kids need friendly peers, while others are a bit too friendly during class time.  All told, about 20% of placement was locked in like this.  That is an example of necessary planning with what is  known (at that time).  What’s important to note is what they did not do–lock in the remaining 80%.

Then, the sending teachers made two “balanced” teams.  More poking.  But the most important part was that no one was locked in until mid-September.  Each week the middle school would observe interactions, listen to students and families, and shift kids around.  Then, they locked in all kids, creating permanent teams and focused on creating identities for those teams and bonds within.

They waited because kids change–they move, they mature and their allegiances shift.  Because services did not have much flexibility (the nature of Spec Ed and part-timers), and some kids need reassurances (i.e., not being placed with a bully or knowing an adult face before the first day of school), those 20% never shifted.  But, as things shook out, a flexibility was built in for the other 80% so that movement was possible.  By the third week, the two teams were as solid as one is going to get.  It was a successful year.

Compare that flexibility with what happened when we locked in the grade last year that suffered at the hands of WL.  For what was gained (a faint sense of knowing their homeroom) much was lost.  While we should be aware of the anxiety we create with some unknowns, it can be fleeting compared to the year-long stress of a bad fit created by the black swan.  Again, the above might not survive a good poke, but our current spring lock-in is troublesome in the long term.

A Note About Counterfactual Poking

It is easy to spot flaws.  The question is a) if that flaw is true in any viable model, b) if that flaw has a major effect.

For example, one argument I have heard against postponing placement until a few weeks into the year is anxiety and parental concern.  Yet, when we locked in the placement of kids, one or two parents would inevitably call with concerns (or be more forceful about it).  Sometimes, that brought a dam break of other requests as parents micromanaged.  In short, parents and kids worry regardless.  The former is easily dealt with because the concerning elements are often soothed with more information, and solutions can be shared (including locking in needs).  The latter upends a carefully planned order, with results (according to chaos theory) that can prove exponential.

So, caution is necessary when weighing options.  People need to be objective that all plans have downsides–sometimes the same downsides–but some recover better.

Conclusion

The idea is that the we don’t know what we don’t know, but we still have to plan.  We want to poke holes in things early, and be honest about defects, before we choose the model that is best for students.

Why Data Matters

My primary blog is Middle School Poetry 180, which you can find here.  I live in the world of the Humanities even while I love all things data.  But, that’s why the bent is towards English majors.

Unfortunately, too many English and liberal arts majors throw their hands up and say, “I can’t do math.”  Idiots.  First, you can do math.  Second, that just gives license to all of the STEM kids to dismiss your teaching of poetry in a “why do we need to know this” way.  Don’t go there.

Let me be brief.

First, data should drive instruction.  If we believe in a growth mindset, data should be used to measure a baseline and then measure progress (if you don’t believe in a growth mindset, please get out of education).  What you do in the classroom should have an impact on student learning.  That’s what we are paid for.

When our school board planned to change how literacy intervention was delivered, I was asked to help craft a slide presentation defending how it had been done.  When I asked for data–some measure that the program worked–I was told there was none.  Instead, I was asked “How can you measure a child’s love of reading?” by one of the instructors.  Well, you can measure how many books they read independently, how much time they spend reading each week, or even how they rate the books they read.  None of those are perfect, but they begin to ask the question.

If you look, you will find numerous programs around your school that have little to no data showing their efficacy.  On a committee to create an alternative program (I began my teaching career in behavioral programs, so I get tapped for such initiatives), I was shown a model program at a local 5-8 school.  When I asked how many students “graduated” back to the mainstream, they had no answer.  In fact, none had.  The program was a warehouse for problem kids.  When I researched data or papers on such programs, I found (at the time) little existed.  I did find one interesting study that explained the phenomena of no data: When a kid succeeds in an alternative program the program is praised, and when they don’t the kids are blamed (“What can you do with those kids?  You tried.”).  Data tends to reveal just how wasteful such programs are–in dollars, time and in wasted youth.

In your own classroom, you might be teaching skills that kids already know.  You might also think they “got it” when they don’t.  Our school (and state) have embraced Proficiency Based Learning (PBL).  If you are not using data, look it up.  Formative and summatives.  No zero.  Redo work until it’s right.  It is transformative.

Second, data drives everything you do regardless of your awareness.  Take the schedule.  One of the issues in our middle school is that students pick up new subjects (Band, World Language) and don’t lose old ones.  Where does the time come from?  When they diced that out, the kids had a lot of transitions–fourteen over a seven hour day.  Adding it up, kids spent more time in the hallways over the course of a single day than they did learning Art in the week.  Data–time–revealed the issue.

Look at your own class.  With the push towards Project Based Learning (PBL, not to be confused with the other PBL) teachers really need to question the value of time spent.  For example, we had students create maps of Europe.  Some spent a full hour coloring them with colored pencils.  Did that add to their understanding of geography?  No, but it took an hour away from their working on their reading.  A basic question to ask is if the time put into something will get the value.

Time is a resource.  It is valuable.  Forget dollars and computers and supplies, time is the resource that helps students most.  Just having time to read can do wonders.  And we measure that in minutes.  In days.  Twenty minutes of Sustained Silent Reading (SSR) over one hundred and eighty days equals sixty hours of reading.  For a kid who refused to read, that’s gold.

Don’t put on a sad math face.  Just look out at your class; can you tell who “gets it” or who is progressing?  Is the project you are doing worth the time, or having effect at all?

Schedule Choices: Class Size vs. Number of Classes

For years educators have been advocating for smaller class sizes.  It’s a noble long term goal, but it is also an example of how a myopic focus one issue can blind people to the big picture, especially in the short term.  And it reinforces the notion that choices means losing something in order to gain.

When they sat down in May to create a schedule for the following year, Grace Have School, a K-8, had smaller classes sizes at the top of their list.  Unfortunately, the primary class sizes were locked.  There were a set number of students (s), and, when the budget was crafted, the number of classroom teachers was determined (t).  The formula was simple:

s/t = c

Because each primary class gets a teacher assigned to them, that class is one section.  In Grace Haven’s case, they had 500 students and 25 full time faculty for core instruction.  That resulted in classes of 20, on average.

Of course, averages are a funny thing as, in the real world of education, they often do not give an accurate picture of the situation.  The administration had decided to have smaller classes in the K-2 and much larger classes in the middle school.  But, the formula holds: If you know two of the variables, the third is a given.  So, if you want a different outcome (i.e., even smaller classes sizes) you need to change the other two variables by shrinking your student population or hiring more teachers.

The Specials, though, had more flexibility.  Because PE, Art, Music, Health and such could be untethered from the core classroom, they had an opportunity to control their class sizes because they could change the number of sections they offered.  By changing the number of sections, they were, essentially, adding teachers.

To understand this, let’s juggle the formula a bit:

s/c = t

Students (s) remains the same: 500.  That leaves the World Language teacher with a range: She could teach one section (c) of 500 (s), or 500 sections with a single student in it.  Both are ridiculous, but you can hopefully see the sliding range.

At present, each Specials teacher had 25 sections with 20 kids in each section.  Over five days, that meant 5 sections a day.

But the desire was to having smaller class sizes for a number of reasons; one of them being that fewer students meant more hands-on chances for each one.  It was solid pedagogy.  So, they aimed for 15 in a class.  Punch that into the formula:

500/15 = 33.3.

Let’s assume that’s only 33 needed, with a couple of classes having 16 students.  Still, in order to have the desired class size each Special would have to offer 33 sections.  Instead of 5 sections a day, they were now looking at 6.6 sections a day.  Yikes!

That, anyway, was the reaction of the faculty.  The gain of 1.6 new sections a day did not only mean they would be teaching more, or that each of those sections needed to be prepped for.  No, the gain of 1.6 classes also meant the LOSS of 1.6 prep periods.  So, more sections to prep for and less time to do so.

The Specials were caught unaware.  Unfortunately, by the time people realized the trade-off the schedule had gone further down the road.  Attempts to undo the damage were met by others on the committee with annoyance, and their pointing out that they were losing preps garnered little sympathy from primary teachers who had few to begin with.  In the end, they wound up with a difficult schedule.

Lesson: With limited resources (time, bodies), choices are trade-offs.  Know what you are giving up for what is gained.  Is your gain greater than your loss?  Go in eyes-open.