A few months ago I took a look at remote learning during the COVID pandemic and concluded that it hadn't made much difference. States that kept schools open did about as well as states that closed schools. The results internationally were similar.
But that was based on partial data and a specific measure of school openness. Today the New York Times points to a study done about a year ago which concludes that school closures did have an effect. Here's the key chart from the study:
There is a difference, but it's surprisingly small: about a tenth of a grade level between the highest and lowest quintiles of school openness.¹ What's more, the variance is huge, which suggests that the results are pretty sensitive to the precise metrics and controls.
Even more interesting is this:
Unfortunately, the authors presented this only for math, but if I'm parsing their tables correctly the effect is even stronger for reading. What it shows is that among white students, remote learning made virtually no difference: a few percentage points with a lot of noise. Nearly all of the effect of remote learning comes from Black and Hispanic students.
Why would remote learning only affect minority students? The same effect is evident when you look at poor students, so the obvious answer is that it's neither race nor remote learning per se that causes problems, but remote learning in places where kids have less access to computers and parental supervision. Here's a chart that shows this:
Remote learning has an impact, but % FRPL (free lunch students) has the biggest effect of all. There's also a large independent effect from "COVID-19 Disruption," which measures the general social disruption from the pandemic.
Taken together, the evidence suggests that remote learning probably had a negative impact, but it was pretty negligible among middle-class students. They did about the same regardless. However, among poorer students the impact was more noticeable.
These results are, of course, politically charged. Conservatives have been blasting liberals for a long time over school closures, so they're eager for any evidence that they were right. And generally speaking, I think they were. However, if you're going to cite the evidence, you need to cite all of it:
- Among middle-class students, remote learning had very little negative effect. Multiple lines of evidence point this way.
- Among poor students the impact was significant.
- Regardless, it doesn't appear that there would have been any harm in keeping schools open since infection rates didn't change much between places that kept schools open and those that closed them. Keeping schools open would have helped some kids and certainly would have vastly reduced the stress on working parents who had no ready way to take care of house-bound children.
That appears to be the current state of our knowledge about this.
¹The difference in math was larger: about a fifth of a grade level between the top and bottom quintiles.