I never expected to get dragged down the rabbit hole yet again of Mississippi's "reading miracle," but I have no choice. It looks like I might have been wrong again. Sigh.
Let's recap: In 2013 Mississippi passed a new law that focused on teaching phonics in elementary school. The results were impressive. After the law passed, Mississippi went from well below the national average in reading to well above it in 2022. In all, their kids have gained about 1-1½ grade levels over the past decade.
But Mississippi also did something else: they put in place tough retention policies for third-graders. Those who don't pass a reading test at the end of the year are held back. About 10% of the class is retained each year.
The problem is that this has a mechanical effect on 4th grade reading scores. If you take a 4th grade class and eliminate the bottom 10% (by holding them back), the remaining average will be higher than it should be. When you account for this, it turns out Mississippi made no gains at all. Just the opposite. There's no Mississippi Miracle after all.
We're now up to date with our story. But I've gotten some plausible pushback from researchers who say that Mississippi has always held back lots of kids. In practice, the 2013 law didn't change anything.
This is where things get very subtle, so pay close attention. It turns out 4th-grade classes aren't just missing the 10% of weak students who are held back. What they're getting is 90% normal students plus 10% who were held back the previous year and are now finally being advanced to fourth grade.
But is this new and has to be accounted for, or has it been happening all along and nothing really changed in 2013?
The answer lies in a subtle analysis of age, courtesy of the boffins at the Urban Institute. Students who are held back and then advanced are a year older than the normal fourth graders. Overall, this means the average age of the class will be 0.1 years older than normal. So do we see this jump in age after the 2013 reforms? We do not:
In 2017, the average age of a fourth grade class is a minuscule 0.01 higher than the 1998-2013 average. That's no difference at all. This proxy is strong evidence that Mississippi's retention policies never changed in practice, which means it's entirely kosher to just compare their scores normally before and after reform.
And by that measure we're back to where we started: Something really did happen in Mississippi. After the switch to phonics, their kids could read a lot better than before.