I never expected to get dragged down the rabbit hole yet again of Mississippi's "reading miracle," but I have no choice. It looks like I might have been wrong again. Sigh.
Let's recap: In 2013 Mississippi passed a new law that focused on teaching phonics in elementary school. The results were impressive. After the law passed, Mississippi went from well below the national average in reading to well above it in 2022. In all, their kids have gained about 1-1½ grade levels over the past decade.
But Mississippi also did something else: they put in place tough retention policies for third-graders. Those who don't pass a reading test at the end of the year are held back. About 10% of the class is retained each year.
The problem is that this has a mechanical effect on 4th grade reading scores. If you take a 4th grade class and eliminate the bottom 10% (by holding them back), the remaining average will be higher than it should be. When you account for this, it turns out Mississippi made no gains at all. Just the opposite. There's no Mississippi Miracle after all.
We're now up to date with our story. But I've gotten some plausible pushback from researchers who say that Mississippi has always held back lots of kids. In practice, the 2013 law didn't change anything.
This is where things get very subtle, so pay close attention. It turns out 4th-grade classes aren't just missing the 10% of weak students who are held back. What they're getting is 90% normal students plus 10% who were held back the previous year and are now finally being advanced to fourth grade.
But is this new and has to be accounted for, or has it been happening all along and nothing really changed in 2013?
The answer lies in a subtle analysis of age, courtesy of the boffins at the Urban Institute. Students who are held back and then advanced are a year older than the normal fourth graders. Overall, this means the average age of the class will be 0.1 years older than normal. So do we see this jump in age after the 2013 reforms? We do not:
In 2017, the average age of a fourth grade class is a minuscule 0.01 higher than the 1998-2013 average. That's no difference at all. This proxy is strong evidence that Mississippi's retention policies never changed in practice, which means it's entirely kosher to just compare their scores normally before and after reform.
And by that measure we're back to where we started: Something really did happen in Mississippi. After the switch to phonics, their kids could read a lot better than before.
Pingback: UPDATE: Mississippi reading isn’t so miraculous after all – Kevin Drum
Without knowing the retention rate in the past one cannot reach the conclusion you have. The age aspect was brought by another commentator before as a key aspect to understanding the scores but even he did nor know the retention level in the past. Another commentator did say the state had a retention rate of 1% twenty years ago and that number had crept up. I think the retention rate doubled in between in the last four years and that would definitely have an impact on scores. The age situation can be explained by various factors including the shifting of the school starting age of the child. In my county, the starting age has moved forward one month. All the changes may be good in practice but do not allow for a firm comparison.
Fantastic work, Mike. Since I currently make more than $36,000 each month from just one simple web business, I commend your efforts. Despite the fact that these are the most basic internet operations occupations, you nd-05 may start making a reliable online income with as little as $29,000.
.
.
Modify your connection—————————>>> https://iplogger.com/1rkjJ9
Phonics… I think I was taught that way way back in 1970. Did they abandon it later? Now it’s back? If it ain’t broke, let’s break it. That seems to be the way of the world these days.
Yes they did abandon it
Just to avoid any possible confusion, I will point out that the "regular" NAEP assessments are done by grade (4th, 8th, and 12th), whereas the long-term trend (NAEP-LTT) assessments are done by age (9, 13, and 17), so the impact of holding students back on the results will obviously be different.
So Kevin’s Mississippi Astonishment continues to evolve. “We do read in the South you know” - https://youtu.be/d7ZOEE63TOw
Those who failed the 4th grade test evidently got new instruction including summer school. More special attention would presumably have helped, whatever method was used. This also does not address why there was little progress by 8th grade. There are many other questions that could be asked - for example what method was being used in Mississippi before - was it pure "whole language"? Was Mississippi the first and only state to make the switch - are all other states still using whole language? As Somerby points out, there is a long history of supposed breakthroughs or "miracles" that turned out to be unsupported, if not the result of deliberate fraud or cheating. The question of whether there is a miraculous method of teaching reading is interesting, but it is not going to be answered by a few hours analysis by anybody - there are too many ways that the Mississippi 4th grade results are not definitive.
Somerby has also pointed out that there has been a long steady increase in NAEP scores overall, in reading but especially math, for both black and white students. Given this genuine progress, it is inappropriate to automatically assume that any progress must be the result of fraud or cheating. Somerby appears to have blown a fuse over the casual mention of the word "miracle" to describe real progress in MS, when there are solid reasons for the noted increases in reading scores in that state. Somerby has noted the reasons why cheating and fraud have not affected the NAEP test, while it has affected other high stakes tests where district funding and teach salaries are based on testing outcomes. NAEP is not used for that purpose and its method of administration makes it very unlikely there is cheating on NAEP -- Somerby himself says that. At some point Somerby and Drum need to bite the bullet and admit that there is real improvement happening in MS.
Use of phonics could account for a lot of the improvement. Although some children will learn to read with the "look-say" method, phonics instruction works for most children. "the look-say method, was based solely on building the child's stock of known words (sight words) recognized instantly. The second approach, the phonics method, focused on helping the child to identify sounds of single letters and letter clusters and blending sounds into words."
Another thing to consider about the average age of the student, and I’m sure there are other factors as well, and if my memory serves me correctly, I recall Bob Somerby explaining how once the passing test rate was at 100%, Mississippi authorities would not factor the excess into the results (meaning that those students who had been retained and then passed the test the following year would not be tabulated because that would lead to a proficiency of over 100%, ostensibly because one cannot have more than a 100% passing rate but one can imagine what this action could possibly do to the age average and it certainly presents an excellent opportunity to cook the tests).
Makes no sense. Retaining children who fail cannot result in more than a 100% pass rate. That works only happen if you retained children who passed and if you stupidly divided number of passed tests by number of children rather than by number of tests taken.
I’m not exactly sure how it worked, but I believe the numerator was all the children including the children who has been held back and the denominator was the children who had not been held back, but once the number reached100%, Miss. authorities froze the number. I believe the daily howler discussed this but I have not seen a statistical analysis how the tabulation could have affected the age of the takers and the scores. Until we get those (and those numbers exist in the state) the comparison of the test results the last four years is quite suspect.
Its just incredible to me that instead of pointing out the obvious -- i.e., the answer to a kid being a grade level behind is to give them MORE INSTRUCTION, or, I don't know, maybe ANOTHER GRADE LEVEL - proponents of phonics or Mississippi adjust for age.
Just jaw droppingly nuts. I do not think that anyone wants to face the policy reality, that if it takes some kids 10 years to learn what others learn in 8 years the friggen answer is to give them TEN YEARS.
This, of course, costs money. Which is why its not done.
Its not done for another reason, which is that in the US there is this faith based belief that everyone could be a good student, when the reality is everyone can't be a good student if you define "good student" by whatever the top 20% of students achieve. The top 20% is the top 20% no matter how all are taught.
Not facing this reality (which leads to a discussion of class, which we cannot have in this country apparently) to my mind dooms discussion to a mix of technique and race. Ugh.
Retaining children gives them an extra year. to learn.
Something nobody ever seems to discuss is what these schools in Mississippi were using to teach reading before phonics. I find it very hard to believe they were using a whole language approach in that state. My guess is they were using the flashcard approach which is from the 19th century - holding up flash cards with pictures on them and having students repeat the word. Everybody know this is a terrible way to teach reading. So they went from the 19th century to the 20th century in their approach? And everybody is applauding this. But honestly I don't know because for some reason absolutely nobody mentions it.
The second thing nobody mentions is exactly what was the intervention? Was there a change in college and graduate school education? Do they directly teach classes on the science of reading? Even if successful that cohort of teachers would not even have entered the stream until about 2017 and considering it takes about five years to get your bearings as a teacher would not have had an impact until about 2020. Or was it based in professional development of in service teachers? Did Mississippi allocating multiple days for this PD (a great idea not matter what). That does not sound like Mississippi to me though. Or are they taking up the existing 3-5 PD days just with the science of reading? What happens to all the other subjects. And anyway, anybody who has been a teacher knows PD are primarily a time for teachers to decompress.
So exactly what were they doing before? And what are they doing now? Teaching phonics is not an answer. It is more complicated than that. To me, something smells really fishy. Anybody remember that Teach for America woman in Washington D.C. who became superintendent and then raised score 1.5 years. Lot off amazing press. Washington miracle. She did it by being really mean to students. How did that work out?
I totally agree. What were they doing? And was this the first time for many Mississippi teachers that they were provided where they could focus on reading? And I also wonder what happened with those 8th grade results? are they still low?
link below
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3688838/
Given the info in this paper, it appears that in 2002 to 2009 MS had a high failure rate in 1st grade (6-9%) and a lower failure rate in 3rd grade (1%).
Through other reporting it appears that after 2013 this changed so that MS has a lower fail rate in grades 1 and 2 but a higher fail rate in grade 3.
Other research appears to show that grade retention impacts a students scores/grades dramatically in the first year but fades quickly in later years.
Put these data points together and perhaps:
- While the average age of 4th grade test takers appears little changed, it seems lilely that there was a big shift in the proportion of kids who repeated 1st grade vs 3rd grade over the last decade.
- Repeating 3rd grade instead of 1st grade would leave average age in 4th grade the same.
- Repeating 3rd grade instead of 1st grade should result in a jump in 4th grade scores with little impact on scores in later years.
It still looks highly likely that the sudden jump after 2013 might be attributable to the retention policies put in place that year.
Here are some additional factors:
MS was the most segregated state in the USA and it still have 32 districts still under court desegregation orders. Until desegregation, there were districts where there were no schools for black children as recently as 1971.
Implementation of the legislatively mandated reading instruction did not happen uniformly but was rolled out as deficiencies were identified. Teacher training was improved when it was discovered that teachers had only a few weeks of training in reading instruction techniques. Reading specialists were added to schools only after it was discovered that some school had no reading specialists at all. College curriculum was beefed up when it was found that there were insufficient courses in reading for students studying to become teachers.
In some majority-minority schools, 100% of black students were retained. In others, up to 30% of those retained were black. It is unclear whether this was because black students were being neglected prior to 2013 or because the focus of reading improvement efforts fell mostly on black students statewide. But the recent intensive change did cause black NAEP scores to improve, not just white ones. A focus on change that includes both black and white students may be something new in MS.
Those taking the 8th grade NAEP may not have benefitted from changes that were not yet implemented when they were at the 3rd grade level. They almost certainly did not experience the new phonics-based reading instruction or the better-trained reading specialists assigned to help struggling individual students. The repeated references to lags at the 8th grade level seems to assume that the 8th graders had the same experiences as the 4th graders, and that is not the case given the way the legislation was implemented unevenly across districts and as needed, reactive to identified deficiencies instead of consistently from 2014 onward.
It is only reasonable to conclude that there is a fading of benefits by waiting 4 years to see whether the current improvement is maintained when today's 4th graders are in 8th grade. That is a fair comparison. Comparing today's 4th graders with the current 8th grade scores is comparing apples and oranges, since the 8th graders learned to read under different conditions.
Or the kids have learned how to take the test - regression to the middle.
I'm not the least bit surprised. I'm pretty solidly liberal on most issues, but the frenzy to uproot phonics in favor of a touch-feely educational theory of "whole language" instruction back in the '80s always struck me as bogus. I felt like it arose because angst over poor reading scores among non-white students led liberal educators to conclude that poor performance MUST be the result of institutional racism embedded in all the old ways of doing things. It took 40 years to admit that this was a mistake.