Diane Ravitch points us to a new study from Paul Peterson and Danish Shakeel that takes a look at student achievement over the past 50 years:
Our data consist of more than 7 million student test scores on 160 intertemporally linked math and reading tests administered to nationally representative samples of U.S. student cohorts born between 1954 and 2007.
....We estimate trends separately by testing program, subject, and grade level and report the median rather than average result to avoid giving undue importance to outliers.
We report changes in student achievement over time in standard deviation units....We interpret a difference of 25 percent of a standard deviation as equivalent to one year of learning.
That's all fine, but the news is not quite as good as their study suggests. As usual I'll focus on reading since that's more representative of general learning:
Here's the key thing to look at: gains in elementary school don't matter unless they're sustained throughout the entire 12 years of primary schooling. In the end, the only thing that matters is how well kids are doing by the time they graduate and enter the workplace.
So take a look at high school. Over the course of 50 years, white kids have improved by 0.6 grade levels. That's barely noticeable.¹
The results for Black kids are more impressive at first glance: they've improved by 2.2 grade levels. However, the bulk of that improvement came in the first two decades of our 50-year period. Peterson and Shakeel don't present their data over time, but of the five tests they use in their analysis only one goes back 50 years: the long-term NAEP. Here's what that test shows:
Black students made big gains in the '80s, but since 1990 the Black-white gap (shown by the green bars) has been nearly flat.²
And in case you're wondering, ending the chart at 2012 is not a typo. For some reason the NAEP folks haven't conducted the long-term test for 17-year-olds since then. I don't know why. However, in 2020 we got results for 13-year-olds and the Black-white gap had widened compared to 2012. There's every reason to think the same thing happened among 17-year-olds.
¹Though it's certainly better than a decline, which is what an awful lot of people believe has happened in primary education ever since the hippies took over.
²Since 1990, the average score for white kids has gone down two points and the average score for Black kids has gone up two points. That's basically nothing.
LTT (and NAEP in general) was adversely affected by Covid. LTT-13 was originally scheduled for fall 2019, LTT-9 for winter 2020, and LTT-17 for spring 2020. LTT-13 was mostly completed during the Before Times, but LTT-9 and LTT-17 had to be scrapped.
LTT-9 was completed this spring, and LTT-13 will be redone this fall. LTT-17 may be on the schedule to be done at some point, but I don't know the specifics. (I no longer work on NAEP, due to retirement.)
Can we just stop with the idea that test scores actually mean anything in terms of how students are doing? Seriously. This is a canard that needs to die. I get that people want objective measures, but there is so much that goes into whether people are doing well or being successful that simply cannot be measured on an academic test. Particularly over the last 20-30 years when our educational system has been almost entirely turned over to teaching to the test, which means that kids are no longer taught critical thinking skills or anything other than whether they can fill in the right bubble.
Kevin and others claim that lead from gasoline affected kid's IQ's, but there is no evidence of this in historical test results such as the NAEP. The kids born in the late 60's who had the highest lead levels reached age 17 at around 1985, but there is certainly no hint of a negative peak in test results then, or really at any time. I don't know if there are sufficient actual IQ tests for meaningful comparisons (you would need pretty detailed age data), but no one ever talks about dips in IQ, just an overall increase over time.
This is just one of the many things which are not consistent with the claimed effects of lead, and which Kevin never mentions.
I am not sure if it is possible to make comparisons of test scores then and now. I grew up in the 50s and 60s. The only standardized tests we took were in grade school — the Iowa basic skills test were given from grade 4 through grade 6, but not in the earliest grades. In high school only top students who were college bound took any standardized tests — the SATs and/or ACTs. Lots of kids went to work straight out of high school. The college entrance test scores dropped as more and more kids, not just the best students, started applying to colleges which pulled down the test scores which freaked the media out.
Now there are a lot more children in my grandkids’ classes whose first language isn’t English that there were when my kids went to the same schools in the 80s and 90s . Those kids almost always score lower on standardized tests.
All of these factors make comparisons unreliable.
They may be doing better in math and reading, but they are totally botching Civics, as proven one recent January 6 (and earlier).
Trump’s base is largely baby boomers who went to school back in the day when civics was supposedly a thing all kids learned.
In order to understand whether results for high school levels have improved, you must also report the drop out rate. If that has decreased (which would be good), then increasing numbers of lower performers will be included in the high school group, which would offset the impact of gains in learning over time. Reduction of the drop out rate would represent achievement that is not being considered in the graph Drum shows.
Drum shows reading scores but not math scores. Reading scores are more difficult to improve because they rely not just on decoding skills but also on life experience to interpret meanings of words and paragraphs as part of reading comprehension. Schools can improve decoding skills in the classroom but how do they address differences in life experiences which occur outside the classroom.
One factor that may affect NAEP scores is the attitude of families and students about test taking. This attitude affects their motivation to do well on a test that does not report results back to individual students and has no impact on their grades. When there are attacks on the concept of testing, as there have been since NCLB, then it may filter down to students as a lack of desire to put full effort into the tests (or perhaps even parents exempting their children from the test altogether) which may work against improvement of scores.
Kevin does not report whether changes in the inclusion of language learnings and children with learning disabilities, which have affected the most recent scores on the regular NAEP, were also made on the long-term NAEP. Inclusion of these additional, previously excluded students, would tend to decrease overall scores on the latest tests.
Simply presenting graphs of this data is a far cry from interpreting it. What do other educational analysts say about whether education is improving over time?
From what I have seen in my school district there are a whole lot more kids whose first language isn’t English in my kids’ schools than there were when their parents attended the schools here. Today my city’s schools have an amazingly diverse student population which is a great thing but which keeps test scores down.
This is not longitudinal testing. It is cohort testing a different points in time. Drum says the gains made in elementary school don't hold up because there are no similar gains at the high school level. You cannot claim that without testing longitudinally and comparing each child's gains with their own scores later in their school careers. NAEP doesn't do that.
Because they are not comparing the same groups of children later on, we don't know whether the gains are holding up, or whether other impacts are causing high school kids in general to test more poorly than younger children. The impact of drop outs on the high school scores is only one example of a possible impact. The inclusion of more language learners in the older age group (due to fluctuations in the number of older kids entering the US via immigration or the tendency to incorporate language learners into older cohorts as they learn more English) may be another impact on the older age groups.
Drum doesn't know enough about teaching to be able to say what is happening. Ravitch does. I would be inclined to accept her interpretations of this data, not Drum's because aside from presenting the data, he doesn't seem to know what it represents.
In the spirit of "we've been attacking a problem for decades with little to show for it", I do think we should take a hiatus from the "black-white achievement gap", and focus exclusively on the "black-asian achievement gap" (since asians are in fact always the ones with the highest test scores, not whites).
This frees us from constantly trying to root out that last bit of hidden racism that surely must be at the root of the differences, and allows us to consider other possibilities that may finally give us real insight into what's going on.