I caught up with Bob Somerby today and it turns out he's been lamenting a spate of recent articles about the "Mississippi Miracle." I don't blame him. Reports of educational breakthroughs are often overblown and should be taken with several grains of salt. Still, let's take a look at this one.
Mississippi passed a package of education reforms a decade ago, including a commitment to the "science of reading"—which mostly means focusing on phonics. So how have they done? We can look at their NAEP scores, but raw scores won't do us much good. There are at least two things we have to do first:
- Compare to national scores. If Mississippi's scores are just matching gains in the rest of the country it's not much of a miracle.
- Disaggregate Black and white scores. Black kids score consistently lower than white kids, and that will skew the results if the ratio of Black and white kids in schools has changed. You have to look at them separately.
With that said, here are scores in reading for 4th graders:
That looks fairly miraculous. White kids have gained 14 points on everyone else since 2013 and Black kids have gained 13 points. That's a lot! But do they hold onto those gains in 8th grade?
Still not too bad. White kids have gained 7 points and Black kids have gained 6 points. That's a little less than miraculous, but still fairly good performance. Now let's see how those 8th graders did in math:
Hmmm. That's exactly the same as the gains in reading. Why would a focus on phonics produce gains in math? Something isn't quite right here.
Bob is skeptical of these scores for another reason: Mississippi's reforms included something called the "third-grade gate," which means holding back kids who can't pass a reading test at the end of the year. This is obviously going to improve scores for 4th graders, but it's a bit of a statistical mirage.
On the other hand, only 9% of Mississippi's third-graders are held back, so the effect is probably modest. I'm more worried about this from a recent Nick Kristof column:
The third-grade gate lit a fire under Mississippi....As third grade progresses in Mississippi, there is an all-consuming focus on ensuring that every child can read well enough to make it through the third-grade gate. School walls fill with posters offering encouragement from teachers, parents and students alike. “Blow this test out of the water,” wrote Torranecia, a fifth grader, in a typical comment.
....Those who did not pass would get a second chance at the end of the school year. Children who fail this second try are urged to enroll in summer school as a last desperate effort to raise reading levels.
This is the kind of thing that leads to rampant teaching to the test or, more seriously, outright cheating. There's no evidence this is happening, but it's something to keep an eye on.
Overall, it looks to me like Mississippi is indeed doing well, but perhaps not as well as its boosters have it. Scores wash out somewhat by 8th grade and reading improvements are no better than math improvements, which calls into question how good the new reading program is. Grains of salt, people.
Just a thought: By 8th grade math can include geometry, trig, and algebra. It would be pretty hard to understand those if you have trouble reading the textbook.
Does the math test cover those subjects?
Drum's concerns and many of those expressed by commenters here were addressed in the comments at Bob Somerby's site. Somerby apparently never reads his comments, nor does he respond to them. Drum didn't read them either. The effect of retention was studied with citation of sources and found not to have affected the MS NAEP scores. There is a documented, well-studied connection between reading and math. Sources were cited for that too. But the larger question is why neither blogger is willing to read his comments and interact with readers on topics like this. They just go on repeating the same mistaken views.
hmm. teaching to the test is fine if the test is good. and extra focus on getting skills up in 3rd grade is also good. the worrisome part is that the gains may not persist. and of course, high stakes tests sometimes encourage cheating
Why did 8th grade NAEP reading scores decrease from 1998 to 2013? Most of that drop has not been made up. Did Mississippi convert to whole-language reading in 1998 (or before) and then suddenly convert to phonics in 2013? Since the 1998-2013 drop is bigger than the 2013-2022 gain it would seem to be more informative. These scores don't seem to agree with the big drop in performance that has come from other tests (including another NAEP test?) - what's going on there?
This subject is more complicated than most reporters and general pundits (like Kristof) can handle. Somerby is worth reading if you don't mind wading through all the elliptical verbiage.
My mom was a teacher. She told me of changing scores back when everything was pencil and paper. She used to scrub kids records of comments by the school shrink. Problem kids desks were right next to her desk.
My Mom was a reading teacher in NYC during the 60's, 70's and 80's, and has nothing good to say about the so called new methods of teaching reading, as opposed to good old phonics. I guess any improvement in education, especially in Mississippi, is a good thing and hopefully these results are real. How does Mississippi's SAT and ACT scores look compared to the national average (notwithstanding the cultural bias in those tests)? How do they look with respect to college acceptance rates?
I am curious as to how you would teach to a reading test other than, you know, teaching the kids to read.
"I am curious as to how you would teach to a reading test other than, you know, teaching the kids to read."
If you have a copy of the test it should be clear that you can teach to the test in various ways up to and including having the kids memorize the answers.
"Why would a focus on phonics produce gains in math? Something isn't quite right here."
Seriously, Kevin? Most math problems contain words/instructions. How can someone solve a math problem if they can't understand the instructions?
Kevin wrote, "This is the kind of thing that leads to rampant teaching to the test" -
Which is not a problem at all, as long as the test really measures what it claims to measure.
(Former HS teacher here. It bugs the crap out of me when people talk about teaching to the test like it's automatically bad. It's only bad if the test is crappy.)
”Why can’t Johnny read word problems?” …
Put me in the camp of.those.that would expect math scores to impruve given improved reading skills (assuming the exam is made up of word problems, which going to the NAEP to.see.some sample.questions seems.to confirm).
I suspect a lot of this is the 3rd grade retention thing. Tennessee started that this year -based on one single. not so well-regarded test. Parents and teachers are screaming.
BS will have more analysis this coming week. I think it is best to wait before reaching any conclusions, but yes holding back 9% of a class will obviously have an impact.
Several commenters suggest that the math scores improved because kids could read the instructions better. Then why didn't the 8th grade math scores decrease from 1998 to 2013 like the reading scores did?
Again, this is not something that anyone will understand without a great deal of information, a lot of which is not given in media stories, or even by specialists who omit things which are not consistent with their biases.
Reading is harder to teach and harder to improve than math. Racial gaps are easier to close and not as wide for math as they are for reading.
Reading relies on different cognitive skills and is more complex than math. Reading learning depends on a wide knowledge and experiences in order to understand what is being read, whereas math learning does not. Reading relies on passive vocabulary and spoken language whereas math does not. Math relies on manipulation of concrete quantities and spatial thinking. They are very different skills. Reading recruits more interconnected areas of the brain than math does. Reading aids math learning but not vice versa.
IQ tests have separated verbal and quantitative processing since the beginning because of observed differences and separateness of the two. Reading correlates more strongly with overall IQ than math does.
Readingbis harder to teach than math? You've obviously never had to teach math. Especially to kids who hear there parents say "I can't do math; it's too hard." Which is most of them.
And who -- given your position, you know I'm not joking -- take actual fucking _pride_ in that statement.
Considering how much of the Miss. improvement was with lower income kids, the 3rd grade retention program certainly seems like the most obvious explanation.
If Miss is showing improvement in the grade level testing but not the age-level testing, this could be confirmed. I dont see a way to find the 9 or 13 year old test data for Miss, the only data i see by age is at the national level.
"On the other hand, only 9% of Mississippi's third-graders are held back, so the effect is probably modest."
Not if all of that 9% is cut from the lowest scores. That should move the overall average significantly.
You are (almost surely) correct, sir! As I detailed on June 5, the lowest-scoring ten percent of kids on Grade 4 Naep tests produce extremely low scores.
Third grade retention gives the lowest performing kids an additional year of maturation and development before they have to take the Grade 4 Naep tests. You can't make a sensible comparison between states which promote almost all 3rd graders, including the lowest performers, and states which hold its lowest-performing third graders back for an additional year of instruction.
It's just a blindingly obvious point. Whatever you think of Mississippi's third grade retention policy, state-to-state comparisons are substantially skewed by such policy differences.
This was posted by mh in comments at Somerby's blog back on June 5:
"“As the Grade 3 retention policy went into effect, the lowest-performing nine percent of Mississippi's third graders didn't move on to fourth grade, as they otherwise would have. In this way, they were eliminated from participation in the next year's Grade 4 reading test.”
The grade 3 retention policy went into effect in 2013. The next naep test was in 2015. But in fact:
Two points:
This article, in which the score gains were attributed to the third grade retention policy, was published in 2019:
“Mississippi rising? A partial explanation for its NAEP improvement is that it holds students back”
https://fordhaminstitute.org/national/commentary/mississippi-rising-partial-explanation-its-naep-improvement-it-holds-students
However, the author updated his article almost 2 years later.
‘Author’s Update, August 5, 2022: Analysis of NAEP demographic data shows that retaining students was in fact not a major contributor to Mississippi’s improved fourth grade NAEP results in the last few years—at least not the way this article suggested. The average age of Mississippi’s fourth grade test-takers was almost identical in 2002 and 2017; increased retention should have raised the age. That suggests that Mississippi has long had a higher retention rate than most states, perhaps making its reading retention policy less controversial than in other places. For further thoughts on how the retention policy has impacted Mississippi’s NAEP results, see my related Fordham Institute post from January 2022: “Student retention and third-grade reading: It’s about the adults.”’
The author asks you to review his newer article:
“Student retention and third-grade reading: It’s about the adults”
https://fordhaminstitute.org/national/commentary/student-retention-and-third-grade-reading-its-about-adults
The second piece of information comes from the naep itself. You can add student age in as a factor. Here’s what happens when you do that:
Grade 4 reading, 2022:
National public, below modal age, 232
National public, at modal age, 216
National public above modal age, 216
Mississippi, below modal age, XXX (no data: Reporting standards not met.)
Mississippi, at modal age, 223
Mississippi, above modal age, 212
The older students, presumably the ones held back, (above modal age), performed less well than those at the modal age."
Both Kevin Drum and Bob Somerby need to read their comments. There are people who have thought about the questions they raise, people with actual information and data. It is wrong to speculate in unfair ways about the MS accomplishments, ignoring analyses like this one raised by mh. The teachers and schools in MS have valid improvement that is the result of real change and hard work. Dismissing their accomplishments as Somerby has done is unfair to those working with children in MS and also unfair to readers here.
The second piece of information (scores at or above modal age) doesn't appear to support the argument that Miss.'s retention policy isn't playing a large role.
Other facts
- Above Modal Age makes up 40% of students nationally. 54%!!! In Miss. This is clearly not only kids who have been held back a year. The large percentage Above Age kids in Miss. seems....odd?
- 2013 began a large jump in the number of Above Age kids in Miss.
- This large jump coincided with significant increases in the score improvements for the bottom quartile of 'At Age Kids'.
- Since 2015, the scores for Above Age kids has stagnated while the scores for 'At Age Kids' has shown steady progress.
Hard to know what to draw from this given that 54% of kids in Miss are Above Age.
"The average age of Mississippi’s fourth grade test-takers was almost identical in 2002 and 2017; increased retention should have raised the age."
If, as the report says, there was not increased retention because retention was already a policy in MS, then that cannot be the explanation for the increased reading scores across that same decade.
The difference appears to be that more children who were held back were passing the 3rd grade reading test the second time around. Previously, if children were held back, they still didn't pass the test a year later. The difference is the early identification of kids with reading problems and extra attention provided by reading specialists (now at each school), so help them do better, instead of just experiencing the same minimally trained teacher doing the same inadequate things the second-time around.
I find it odd that Somerby and Kevin Drum are both so resistant to the idea that a state making a major effort to improve reading instruction would not produce positive results. Why do they both think that kids cannot do better with better instruction?
"Why do they both think that kids cannot do better with better instruction?"
In theory, kids can do better with better instruction. It happens all the time.
In this case, the question is this:
To what extent did that happen in Mississippi, and to what extent may the score gains have resulted from something else?
I want to ask,
how meaningful are student reading scores?
Where are the adult, non-student reading scores?
I’m glad some can read. Now maybe they will learn to speak english too!
As someone who has taught school for many years (albeit not at the K-12 levels) I've got to ask: what's so bad about holding students back a year if they haven't mastered the requisite skills to advance a grade? I won't ask my usual question about giving teachers more authority, but that one ran into a brick wall a long time ago.
I would think it would indicate that you havent figured out a better way to teach.
It would be important to know if improvements are coming from new teaching methods or if they are coming from doubling the amount of instruction time.
Graduating from HS at age 30 with great test scores might not be a model we can emulate broadly.
Could you explain your reasoning? I'm astonished that you make such a blanket statement.
And no, past a certain age they can damn well go for their GED. Thirty is far, far past that cutoff.
On Edit: We've been teaching humans of literally tens of millenia. Teaching specific skills has got to be one of the oldest technologies there is. Is there room for improvement? Sure. But it if's even as much as a factor of two I'll eat my hat.
It may not be so puzzling that an emphasis on phonics could help raise math scores. To me, phonics organizes the process of reading almost in a math-like way, where words are broken down into pronunciation elements. This allows the student to organize words according to elements which correspond to a list of pronunciation keys. I could see that training a kid's brain to think about language in this way could reinforce their ability to learn math as well.
There were a lot of improvements made besides simply implementing a phonics program. First, the state legislature allocated more money for teacher training. It also examined the amount of time devoted to training teachers in reading instruction in certification programs. It was a matter of 3-5 weeks. They increased that to a requirement for 2+ courses. They found that very few existing teachers had training in reading instruction so they provided remedial training. They found that there were few reading specialists and many schools had none, so they instituted a requirement for a reading specialist at every school. They focused attention on early intervention for kids who were not responding to standard reading instruction, addressing their problems earlier.
This is a lot of change and it didn't happen over night, but Somerby's derisive response to the word "miracle" ignores the huge amount of effort that was invested in chaning reading instruction statewide in MS. It should show results and it does. I do not understand the apparent attitude that kids cannot learn no matter what you do, which is inconsistent with the results observed empirically in MS.
Perry doesn't understand the rejection of the term "miracle." Try this on for size:
According to the Naep reading data, Mississippi's lower-income white fourth graders are more than two years ahead of Mississippi's lower-income black kids. The gap is more than two years, and the kids are just in the 4th grade!
Perry, is that your idea of a miracle, a revolution? I've praised the people in Mississippi for putting in these extensive efforts. But if you can't see the problem with the (very familiar) upper-end claim of an educational miracle, something is very wrong inside your true-believing brain.
Kristof calls that "race gap" a revolution! I call it part of a giant, unsolved historical disaster. So too with your instinctive true belief with respect to all tribal verities.
This sort of thing has gone on forever--for fifty years, in my own experience. If you cared more about this topic (and about those kids), you'd be skeptical too.
Pitiful. There's really no other word...
Marjorie Taylor Greene says her tv turns on by itself at night to spy on her,
https://www.rawstory.com/marjorie-taylor-greene-smart-tvs/
Because not even tv can get enough of Marjorie Taylor Greene.
Where everyone else is watching it, it is watching her.
That's where it gets it's ideas.
Pingback: Here’s a little more about miraculous Mississippi – Kevin Drum
Pingback: UPDATE: Mississippi reading isn’t so miraculous after all – Kevin Drum