I caught up with Bob Somerby today and it turns out he's been lamenting a spate of recent articles about the "Mississippi Miracle." I don't blame him. Reports of educational breakthroughs are often overblown and should be taken with several grains of salt. Still, let's take a look at this one.
Mississippi passed a package of education reforms a decade ago, including a commitment to the "science of reading"—which mostly means focusing on phonics. So how have they done? We can look at their NAEP scores, but raw scores won't do us much good. There are at least two things we have to do first:
- Compare to national scores. If Mississippi's scores are just matching gains in the rest of the country it's not much of a miracle.
- Disaggregate Black and white scores. Black kids score consistently lower than white kids, and that will skew the results if the ratio of Black and white kids in schools has changed. You have to look at them separately.
With that said, here are scores in reading for 4th graders:
That looks fairly miraculous. White kids have gained 14 points on everyone else since 2013 and Black kids have gained 13 points. That's a lot! But do they hold onto those gains in 8th grade?
Still not too bad. White kids have gained 7 points and Black kids have gained 6 points. That's a little less than miraculous, but still fairly good performance. Now let's see how those 8th graders did in math:
Hmmm. That's exactly the same as the gains in reading. Why would a focus on phonics produce gains in math? Something isn't quite right here.
Bob is skeptical of these scores for another reason: Mississippi's reforms included something called the "third-grade gate," which means holding back kids who can't pass a reading test at the end of the year. This is obviously going to improve scores for 4th graders, but it's a bit of a statistical mirage.
On the other hand, only 9% of Mississippi's third-graders are held back, so the effect is probably modest. I'm more worried about this from a recent Nick Kristof column:
The third-grade gate lit a fire under Mississippi....As third grade progresses in Mississippi, there is an all-consuming focus on ensuring that every child can read well enough to make it through the third-grade gate. School walls fill with posters offering encouragement from teachers, parents and students alike. “Blow this test out of the water,” wrote Torranecia, a fifth grader, in a typical comment.
....Those who did not pass would get a second chance at the end of the school year. Children who fail this second try are urged to enroll in summer school as a last desperate effort to raise reading levels.
This is the kind of thing that leads to rampant teaching to the test or, more seriously, outright cheating. There's no evidence this is happening, but it's something to keep an eye on.
Overall, it looks to me like Mississippi is indeed doing well, but perhaps not as well as its boosters have it. Scores wash out somewhat by 8th grade and reading improvements are no better than math improvements, which calls into question how good the new reading program is. Grains of salt, people.