My homework assignment today was to figure out if COVID-19 lockdown restrictions and other countermeasures have been effective in reducing COVID-19 cases. I spent some time googling around, and what I discovered is that (a) conservative publications all have charts showing there's no correlation, while (b) academic papers all show a positive correlation. Hmmm.
The problem is that there are a million ways to measure this, and it's pretty easy to keep slogging away until you find one that fits your ideology. So for better or worse, I decided to try the simplest possible scatterplot and publish it regardless of how it turned out. On the horizontal axis I've used WalletHub's scores for how restrictive each state is. The vertical axis shows the cumulative percentage of each state's residents that have contracted COVID-19 as of June 1. Here's the result:
There are some interesting things here. For starters, the correlation is indeed positive: more restrictions generally lead to a lower case rate. But there are some high-profile exceptions. Texas and Florida both had very few restrictions, but their case rate is only slightly higher than California and New York, which both had lots of restrictions.
My conclusion is twofold. First, restrictions matter, but maybe not all that much. Second, you'd need to do some serious analysis with proper controls to really be confident in these results. In the meantime, this bloggy horseback estimate will have to do.
Want more? Here's a WalletHub analysis showing that blue states have been safer than red states. Here's a study in the Lancet showing that increased use of face masks leads to lower COVID-19 infection rates. Here's a study in PLOS One that divides states into buckets and finds that the bucket with the highest mask-wearing does better than the bucket with the lowest mask wearing. And here's a study from the Baker Institute showing that the correlation between openness and death rates has increased over time:
For what it's worth, I don't trust any of these studies, including mine. Still, the bulk of the studies do indeed seem to show a modest positive correlation between COVID-19 countermeasures and case rates. That's certainly the way to bet at the moment.
Personally, I find the absence of a flu season to be the most persuasive evidence of the efficacy of the virus restrictions. There's a long history of flu seasons affecting millions of people, and this came to a dead halt during the last season. It would take a truly heroic effort to attribute that to anything other than the restrictions.
Flu disappeared this year all around the globe, even in places where masking wasn't used (Sweden) and restrictions were limited (the US South). There's a phenomenon known as "viral interference," in which viruses interfere with one another. Here's a STAT article on the phenomenon (https://www.statnews.com/2021/01/31/a-viral-mystery-can-one-infection-prevent-another/).
It may be that viral interference (not just NPIs) factor into the disappearance of flu and other respiratory infections during the COVID-19 pandemic.
Flu has to start or come from somewhere, if it would ordinarily begin in or spread through more masked areas before it gets to the South or Sweden that may have stopped it.
It's an interesting idea. I doubt the math works out given the low prevalence of COVID in the population, but of course one would have to run through the numbers to be sure.
even in places where masking wasn't used (Sweden)
Masking was "used" in Sweden, like in most countries. It's true the government of Sweden was (at best) tentative in terms of its masking policy in the early-going, and this undoubtedly lead to lower rates of mask-wearing in Sweden than would otherwise have been the case. But that doesn't mean Swedes eschewed masks en masse.
Social distancing was also employed in Sweden, both by people on their own (when the Swedish government was engaging in its infamous experiment) and with the backing of the government, when this policy was abandoned.
But plenty of Swedes wore masks, and continue to do so.
A Yougov survey showed that only 5% of Swedes wore masks in spring 2020 - about the same as the other Scandinavian countries. This rose to 15% in the fall and 30% this past winter
Do people in Florida and Texas spend more of their social time out of doors than other places?
Doubtful! The heat is pretty unbearable in FL and TX a lot of the year, and they spend most of their time indoors with air conditioning during the summer months. The climate is not nearly as nice as that in Southern California, to say the least.
Might as well be Sweden.
Much "social time" in Texas and Florida is spent by tourists, who obligingly take their infections to other states, resulting in lower COVID numbers for residents in Texas and Florida. The entire issue is really a good reason why I think people who use the phrase "herd immunity" are either pandering to their audience and/or have little idea about what they are saying.
I think it's pretty widely acknowledged in (or at least suspected by) the epidemiological community that mild/warm climates may corelate negatively with the spread of covid. It's only one factor of many, and lack of cold weather itself is not going to be sufficient to keep the pandemic at bay. But, all things equal, a mild/warm climate probably helps at the margins both by inhibiting indoor crowding and by encouraging outdoor air ventilation (it's speculated that the latter may actually enable or generate low-viral load infections, which could conceivably result in protective immunity). The more abundant UV radiation of sunnier climates is also probably helpful.
I've personally long thought it telling that the initial, severe outbreaks during phase 1 of this pandemic were largely concentrated in areas experiencing comparatively similar climate conditions. South-Central China in midwinter isn't that much different from large swaths of Western Europe in late winter or the Northeastern United States in early spring: basically chilly and damp, but not bitterly cold.
I have similarly wondered if humidity and chilliness may create an environment in the lungs that's more conducive to infection, and similarly high humidity in Brazil and India, without the chill.
I can't remember if I've seen a report about a study about this or not.
Flu disappeared this year all around the globe, even in places where masking wasn't used (Sweden) and restrictions were limited (the US South). There's a phenomenon known as "viral interference," in which viruses obstruct one another, though the precise mechanisms aren't well understood. Here's a STAT article on the phenomenon (https://www.statnews.com/2021/01/31/a-viral-mystery-can-one-infection-prevent-another/).
It may be that viral interference (not just NPIs) factor into the disappearance of flu and other respiratory infections during the COVID-19 pandemic.
Masks aren't the only defense that was used. Schools were closed. Social distancing. Restrictions on gatherings. Restaurants closed. Work-from-home increased. Etc. R0 for flu is 1 to 2. R0 for Covid-19 is 2 to 3. If you don't wear masks but do some other minor things that cut Covid-19 in half so that it is still spreading rapidly but slower than it might spread, you've stopped the spread of the flu.
+1
As Newsom even admitted, strict lockdowns don't work. Hopefully woke neoliberals learned their lesson.
Closing parks was stupid and recognized scientifically as stupid pretty early on.
"Strict lockdowns" appeared to work in Australia, New Zealand and mainland China.
What you can say is that restrictions don't work in countries where "freedom" is more important than not killing your friends, family or perfect strangers by giving them a deadly contagious disease. In places that value friends and family over "freedom", they seem to work pretty well.
"Still, the bulk of the studies do indeed seem to show a modest positive correlation between COVID-19 countermeasures and case rates. That's certainly the way to bet at the moment."
Thank heaven for small mercies! Consider the implications if the 'studies' didn't show 'a small modest correlation.' We would have to discount everything that we know about the virus!
Blogging is hard:
https://www.vox.com/recode/2021/6/2/22465127/trump-desk-blog-gone
You have to like to write.
I think the biggest issue is that like him, his base is lazy. That is why twitter was perfect for him. The message was short and automatically delivered to everyone who followed him.
That is why his blog tanked. It required his fans to actively check regularly if their Dear Leader had written something, and when he did, they had to read a lot more than 280 characters. Both of those things require some effort on their part.
The fake billionaire was too cheap to hire a ghost writer.
Maybe ghost writers he tried to “hire” realized they wouldn’t get paid. Trump is crime, and crime doesn’t pay (joke stolen from Richard Pryor).
You have to have a purposeful interest to look at someone's blog, where a tweet just appears and you can easily retweet it to look like you're on the ball about something and not think about it twice, but you have to really have a purpose with a blog entry, you have to think about it to share it, and if you think about it, Private Citizen Princess Tiny Paws is an idiot.
The main problem is that, especially early on, the lockdowns were reactionary. You have an outbreak--now it's time to try a lock down to starve it. You didn't get hit yet, you don't have a lockdown. Pick the right time frames, and you can say anything you'd like.
+1
This sounds correct to me.
No question. In the US, for instance (and yes, in Western Europe, too) January/February/early March were largely wasted. By the time strong measures were taken, it was too late to prevent coronavirus endemicity. At that point truly squelching the pandemic probably required a hyper-strict, 90-day lockdowns, universal, strictly-enforced masking, mandatory isolation for infected persons, mass-testing, etc. (In other words, measures liberal democracies generally weren't equipped to undertake). That doesn't mean we haven't seen a range of results based on effectiveness and thoroughness of policies since then, of course, hence relatively stronger performers like Canada, and relatively weaken ones like the USA. But only a half dozen or so countries globally appear to have gotten back to normal without vaccines.
It is far more than six countries. Vaccination came too late to make much of a dent in live saved over all. In the U.S., Canada, U.K, Western Europe, Japan and South Korea Covid-19 cases plummeted in January, February into early March with very little vaccination apart from Britain in February.
It is far more than six countries.
Far more than six countries have gotten back to normal without vaccines? Really? I suppose I could be overlooking a few principalities or tiny island states. Maybe we could stretch to definition of "normal" to get it up to ten or twelve. But that doesn't seem like a very accurate use of "far." (And I'm using a relatively liberal, non-strict definition of "normal" as in, "Have mostly squelched the virus and returned to normal, pre-pandemic life, but regular outbreaks are still being dealt with.")
(mainland China, New Zealand, Australia, Singapore, Iceland, Greenland....Taiwan might still belong on the list if they can get back on track in the next few weeks. Ditto Vietnam. Who else am I missing? That makes eight).
Portugal, South Korea and Japan's cases have been plummeting with few vaccinated.
I think there are a number of countries that have returned to something akin to normal without masks but with indoors activities. The issue for them (as it could have been with us) is one of prevalence.
If you have a hundred people in a restaurant and none of them are contagious with Covid-19, everyone is perfectly safe. If even one person is contagious, everyone is at risk. And if one of those people is a super spreader, lots of people will likely be infected.
But with the right public health infrastructure and approach, you are able to stamp out the occasional flare up. If the west had used the months of advance warning productively we could have enjoyed the same results.
My present concern about the lackadaisical vaccination programs in some of those countries is that, to borrow a metaphor, they are like a fiddler on the roof.
Portugal, South Korea and Japan's cases have been plummeting with few vaccinated.</i.
Well, then we are indeed talking about different things. I'm talking about getting life fairly close to pre-pandemic norms -- stopping community spread (or at least reducing it to only occasional, sporadic outbreaks).
Portugal is diagnosing hundreds of new cases a day -- proportionately more than the US. "Back to normal" in my book means going days and days (at least for a small country like Portugal) with zero community spread. Portugal is in a very different situation from, say, New Zealand, or mainland China. Japan is still suffering nearly 100 daily deaths per day much less ending community spread. (Better than the US, but far from "back to normal" by any reasonable definition).
South Korea has overall been a success story, agreed, although I think they fall well short of the top tier (successful covid squelchers like New Zealand, China, Australia). They've been averaging about 700 new cases daily for a while now, and more days than not S. Korea suffers covid deaths.
Maybe you need to also control for the weather. No lockdown or Mask restrictions in cold States like NY would have been an epic disaster. A tad easier in Fla and Texas. Then control for summer in those spots when people are forced in for A/C.
This analysis runs into the very problem that drove people to start counting Covid deaths instead of Covid cases... "cases" is a very unreliable measure, since you have to assume that the states who were more serious about lockdowns were also far more serious about testing.
As Trump and his governor lackies loved to say, "Derp! Hey, if we just don't test, there aren't any cases!"
Never mind who know what kind of chicanery was pulled in Florida, etc, to sugar coat their Covid numbers.
+1
+1
There is a timing problem with your data. New York and California had big covid-19 hits early before the epidemic was understood.
If you take out the April 2020 numbers for all states and look at Texas and Florida, they have large increases in covid-19 cases correlated with their relaxation of controls.
In the recent past few months Florida continues this trend.
Puzzlingly Texas does not follow this trend but instead cases continue to decline. There is a behavior for Texas in the last few months which does not match the more general pattern.
+1 Agreed. I think a time axis is needed if you are trying to discern trends and/or correlations. Also, I seem to recall vaccine uptakes hitting the numbers at different times for different states.
Yes. I think all of the "let's look at the correlation between X policy and covid cases/deaths" should start counting in May or June of 2020. I'd also prefer to use excess deaths, where possible, rather than COVID counts, because different states have different quality COVID datasets (e.g., https://jamanetwork.com/journals/jama/fullarticle/2778361).
Looking at cases is pretty much guaranteed to introduce noise. Look at excess deaths instead.
The number one thing you need here is independent confirmation that Florida and Texas are reporting correct numbers. Since Florida literally tried to jail one of their data researchers for publishing data the state did not want published, I find it hard to trust their numbers.
You are assuming of course, that the data from Texas and Florida is correct, i.e., that the way covid deaths are counted is the same everywhere. Not so sure that's absolutely true. But also not sure how much covid deaths could be miscounted and show up in the data? The case of Texas not so much, but Florida, despite it's very aged population, really seems to have avoided the virus. Count me skeptical.
The problem with doing this by state is that it ignores the differences that exist within each state. Take California for instance. Politically, a massive blue state. But geographically, it’s got a whole lot of red in it. In response to a similar question last week, I started to look at various places somewhat randomly.
San Francisco, a blue city if there ever was one, initiated lockdowns and other harsh measures sooner than most other places. They also kept those measures in place more consistently and for a longer period of time. Yet Montana, a red state with approx the same # of residents had three times the cases. Or ... take Merced, a red county in California. It had a case rate significantly higher than SF’s. Every red state or city I looked at had a much higher case rate than SF.
But here’s the other issue that isn’t addressed ... enforcement. It’s one thing to have a “lockdown.” It’s a whole other thing for that to actually be enforced. My sense in blue Sacramento is that it wasn’t enforced. And I think that’s accurate for a lot of other places.
Until these analyses get more granular and take enforcement and/or compliance into account, they are all rather meaningless.
Well said, sir.
Yes, a breakdown by county (these records seem to exist) would be more meaningful, although there are still many other confounding factors.
The more localized stats definitely are out there.
Beat me to it. I was going to suggest that if the chart could be made by county, not by state, the correlation might be more clear. Here in San Mateo County (near San Francisco) we have just passed 81% vaccination rate. We had fairly good masking here. I almost never saw anyone without a mask. I've read that some northern counties in CA are experiencing expanding case rates, they strong Republican counties which likely did not observe mask mandates.
Exactly.
Good points, especially about compliance. I suspect that political affiliation was fairly strongly determinative of individual decisions to wear masks, avoid crowds, distance, and generally follow CDC and public-health recommendations, regardless of what state or local governments were doing.
This is true. It would also be the case with a lot of red states. In Texas, a lot of (responsible/conscientious) individuals and businesses still observed distancing and mask-wearing, even as state leaders were running around encouraging people to lick doorknobs and sing extra loud in jam-packed churches to pwn the libs. In CA, I think demographics and employment patterns had a lot to do with differences between, e.g. the Bay Area and the Central Valley moreso than politics. Workers in SF were by and large able to work remotely and isolate effectively. Farm laborers and food processing workers in Merced or Modesto? Not so much. I'm sure a good number of cases in the Central Valley were rightwing yahoos flouting masking and distancing rules, but the vast majority of cases were among poor farm workers who lived and worked in cramped quarters with little or no PPE.
I understand some of us really like studies but there’s studies and then there’s studies. Not all created equal and aggregating them doesn’t point the way to a more reliable answer. GIGO.
Texas
https://www.texasobserver.org/how-flawed-death-investigations-can-leave-texas-covid-19-deaths-uncounted/
https://www.houstonchronicle.com/news/investigations/article/The-Chronicle-investigated-Texas-lag-in-15618691.php
https://www.thedailybeast.com/texas-erases-covid-cases-and-fans-conspiracy-theory-flames
Florida
https://www.msn.com/en-us/news/opinion/florida-s-covid-19-data-is-unreliable-confusing-and-hazardous-to-our-health/ar-BB16C7qT
https://www.sun-sentinel.com/coronavirus/fl-ne-ss-prem-covid-deaths-florida-election-20201216-f4kgezjf4rf75ppumt4omxfsxy-story.html
Re: Texas, a potential confounding factor is that a large portion of the population live in Dallas, Houston, San Antonio, and Austin. These places have had significant county and city restrictions. Now the governor eliminated the legal basis for the restrictions but the residents mostly followed them all the same. In fact if I go into a Home Depot in Austin 80% of people are still wearing masks despite every government official and the company policy aligning to CDC guidance (we are over 70% vaccinated in this county).
So point being, the state policies may not be representative of what has actually happened at the local level.
Yeah, that was my thinking too.
State level Covid restrictions are only one piece of the puzzle. Counties and cities will often have their own restrictions. And of course, businesses will have their own.
So, using only State level restrictions won't give a complete picture of how those restrictions affected Covid rates.
And then there are all other factors such as density, weather, etc. that affected the Covid rates.
We'll never have a really good answer.
And here's a study from the Baker Institute showing that the correlation between openness and death rates has increased over time:
This seems very intuitive to me. My strong sense is the Northeast was being very heavily seeded before lockdowns and widespread mask-wearing commenced in the US, and by the time these measures did take effect, it was like trying to put the proverbial genie back in the bottle.
Yes. Also, treatment protocols have improved over time. North Dakota, with less than one-quarter the population of Connecticut, has had half again more cases, but a considerably lower case-fatality rate, because cases there peaked much later than in the Northeast.
Set aside the academic studies. What to health insurers think happened?
What lockdown? I worked from home in the March - June 2020 timeframe then went back to work where everyone wore masks and meetings were held via Webex. I ate lunch in the cafeteria which looked like any pre-pandemic restaurant except for the plastic barriers at tables. When restaurants opened I went out a few times, but not as much as before. The service was terrible and remains so. Otherwise, my life went on as normal.
There was no lockdown, really. The winter wave washed over the whole country without any obvious cause. Nothing stopped it either. The data tells us nothing at this stage. It would have been nice if more were done to prevent spread in nursing homes and elderly assisted living. That's what, I think, required a real coherent policy response.
You are describing a pretty significant change in your own behavior. You may not call it a "lockdown" but it is certainly a lot of counter measures.
Sorry, but your conclusions are wrong. All academic publications are peer reviewed whereas the others are not; consequently, one group is scrutinized by experts, all of whom have specific expertise in the analyses of epidemiological data whereas who knows about all the others. If anyone out there thinks this is a trivial fact, then read the publications and determine whether you could replicate or improve upon their analyses. For God sakes, these folks have spent 2-3 yrs in an MSc, 5-6 yrs in a PhD, and anywhere from 2-6+ years in post docs BEFORE they actually got a job. Then another 4 or 5 yrs to get tenure. Most of the folks I hired in my 16 yrs as a Dept Head of a Biology Dept. were somewhere between 32 and 40 yrs old before they got their first job. Comparing that background to people with no background whatsoever is utter nonsense. This denialism of expertise is what drives the bloody Republican Party and is driving the US to a state of blissful ignorance.
I would use a measure of excess deaths instead of infection rate, as there are stories of COVID not being diagnosed in states like Florida when it could be pneumonia. I know that this is also an imperfect measure, but it is easy to misdiagnose, but hard to not count a death.
The idea that somehow no one in FL or TX self-isolated, wore masks, or social distanced just because both states have a$$hole governors is just pig ignorant. In FL many many local governments did have mask mandates, as did the big national chain stores where people shopped. Schools were closed or had more limited in-person classes. People did not go out like they used to. Things were very different here in FL I can assure you. Studying COVID countermeasures by only looking at statewide regulations is just not a very intelligent way to go about this.
Strongly agree. Unfortunately, we have woefully underinvested in public-health services. We haven’t even been able to contact-trace to the extent we should. But we should have had people in the field taking randomly-distributed observations of cautious and incautious behaviors in various public places, and obtaining quantitative data on numbers and durations of interactions. Self-report data should be relied on as little as possible. That would give us far better calibration of the efficacy of behavioral measures.
There's an enormous amount of data on both cases and restrictions, so I'm guessing over the next few years we'll see some interesting studies. Maybe some will even help us draw some conclusions.
The factors that I suspect had an impact are (in no particular order):
1) Various restrictions (each of which probably had different levels of efficacy)
2) Level of compliance with restrictions
3) Weather (temperature and humidity) and how much time the population was indoors
4) Population density
5) Timing (how far the spread was into the population before countermeasures took place)
6) Social effects - Everything from super-spreaders to the way segments of communities interact and congregate. I think this sort of community "flow" is poorly understood but made the spread harder to predict.
It's a lot, and there may be other factors as well. Isolating each of them will be a real challenge, but like I said, there is a lot of data, so maybe we'll find something.
A few points in defense of restrictions:
1) Europe did much better than the US in the summer of 2020, and I have to suspect that a lot of that was due to the US (particularly the south) wanting to abandon restrictions as soon as possible.
2) The worst part of the pandemic was November-December, when even the more aggressive states were getting lax about countermeasures. (Certainly Michigan looked a lot different in April/May 2020 than in November/December, and we paid a price for that.)
3) Eradication of the flu (although that may have been viral interference, as other comments have pointed out.)
There was a natural (understandable) tendency early on to try to point to country X or Y (Singapore or Germany or Canada or...) and say they did it right, but I don't think countermeasure policies alone explain all those differences. Restrictions probably helped, but other factors may have been more important.
A lot of political actors during the pandemic injected a lot of stupidity for nakedly selfish purposes, Trump probably chief among them, and I think that made this pandemic -- which already had enough uncertainty -- harder to manage. But we had a great variety of national, state, and local government responses to the pandemic across Europe and the Americas, and everybody struggled to some extent with it. If masking and distancing alone could handle it, we would have seen greater divergences in outcomes.
There were orders-of-magnitude differences between many nations of the Asia-Pacific region and the rest of the world. What they had in common was prior experience with SARS-CoV-1, a much deadlier coronavirus. They had plans and preparations for dealing with novel infectious diseases. From the start, at least some that I read about treated all returning travelers as potential carriers, isolating and testing until they were sure the arrivals were not infectious. (Someone is going to say, they’re islands. Vietnam isn’t. And nobody walked from China or Europe to North America.) They therefore had far fewer cases at the beginning, well within their capacity to contact-trace, test and isolate.
"Restrictions probably helped, but other factors may have been more important."
Yes, you are correct. One of the key factors I think is the attitude of the national government towards the problem. Not just in terms of the actual policies or rules adopted (which in most places were a combination of national and local mandates), but simply how serious the leaders of the nation thought the problem was in their attitudes, speeches, and personal actions.
For example, in Canada the majority of the restrictions were set at the provincial level, however from the very beginning Trudeau treated the pandemic very seriously (probably because his wife was one of the first high level government related people who got sick), even going as far as isolating himself in the PM residence for a long stretch while still giving PSA type of speeches daily to encourage people to follow the guidelines set by their respective local governments, and to explain the national ones. That attitude helped unify the provincial governments to take matters seriously, even among provinces governed by the opposition. Things were not perfect, but overall there was a big sense that either everyone cooperated or the nation as a whole would suffer, and as a result the death toll has been one of the lowest among the western nations.
The opposite is also easily seen. The USA, the UK, Russia, Brazil, Hungary, Mexico, and India, either for the entire time or at least early on, they all tried to dismiss the pandemic as a hoax, or not a big deal, and not coincidentally they are all (or were at the time) led by authoritarian leaders who think simply wishing problems away or ignoring them is enough to make them disappear, who regularly dismiss expert advice, and who went as far as encourage people to flaunt local rules when they went against their preferred response, and as a result they all ended with some of the highest body counts in the world.
Whether a leader is good or bad, people tend to follow the example set by the leader.
The worst part of the pandemic was November-December, when even the more aggressive states were getting lax about countermeasures.
Deaths in the US peaked in January, during Trump's last full week in office.
KD: “the bulk of the studies do indeed seem to show a modest positive correlation between COVID-19 countermeasures and case rates. That's certainly the way to bet at the moment.”
The mathematical treatment of probability originated among gamblers, in a context where gain and loss were well-defined. One of the problems with null-hypothesis significance tests (NHSTs) as used in the social sciences is that the ‘payoffs’ are ill-defined, or not defined at all. I spent my career in applied work (as did William Gosset, ‘Student’) where the costs and benefits were typically much better understood. In the case of Covid-19, you are potentially betting your life. You go with whatever the odds favor, no matter how slight the apparent benefit.
I think some of the data is misleading. In the top chart it shows Florida as having the fewest restrictions. That may be true at the state level but Gov. DeSantis left imposing restrictions up to cities and counties. Virtually all of them enacted some form of restrictions. I live in Tampa and, prior to the CDC loosening its recommendations a month or so ago, it was well over a year since I'd seen anyone inside a public place NOT wearing a mask.