When ‘good’ news might not be so good

See updates for April 2, 2020, April 4, 2020, and April 5, 2020 at end of post

So my mom sent me an article from the Washington Post which appears to show that California and Washington, having acted to contain the novel coronavirus sooner, may be having greater success in “flattening the curve.”^[1]

There are two major problems with this appearance which merit caution, rather than optimism, and what they both mean is that we don’t know how many people have been exposed, how many people really have fallen ill with COVID-19 (the disease caused by the coronavirus), and how many have recovered or really even died.^[2]

The first problem is that we haven’t tested anything like a representative sample of the population. We fell behind to begin with, testing has focused on people exhibiting symptoms, and demand for the tests has far exceeded supply. In a great many cases, symptoms are absent or mistaken for something else.^[3] That means we don’t really know how many people have come down with COVID-19.

The second problem is that the test may well be crap. Experience in the field (not confirmed through controlled studies) suggests that there may be something like a thirty percent false-negative rate. This means that the test fails to detect COVID-19 in something like thirty percent of the people who actually have the disease.^[4] Which means we know even less about how many people have come down with COVID-19 than we thought.

It appears that in severe cases, COVID-19 can be diagnosed by other means.^[5] But that entirely leaves out folks who are asymptomatic or who experienced mild symptoms. The picture here is so badly incomplete that any claim to a complete picture can only be regarded as ludicrous.

Which means that a lot of the data going into the models and graphs the Washington Post article presents, some apparently from a White House briefing, is crap. As we at least used to say in the computer business, garbage in, garbage out (GIGO).

That means considerable caution is called for in interpreting this data. It’s good that California and Washington got a jump on containment and now appear to be confirming fewer cases. But it’s much too soon for optimism.

Update, April 2, 2020: Indeed, it turns out there are lots of questions about the White House models and projections^[6] that Philip Bump really didn’t examine critically.^[7]

Update, April 4, 2020: Yet more criticism has emerged of the models used by the White House and others to project a “flattening of the curve.” The problem here is essentially that, short of a vaccinated or otherwise immune population, the moment you lift the lockdown, the virus is free to spread again.^[8] This is consistent with what Joe Pinsker wrote in the Atlantic earlier: Things can return to normal “when enough of the population—possibly 60 or 80 percent of people—is resistant to COVID-19 to stifle the disease’s spread from person to person.”^[9] There are a couple ways this can happen: One is that a vaccine is developed—this is unlikely before next spring.^[10] Another is that enough folks catch the disease, some asymptomatically, and either recover, hopefully but not certainly gaining immunity, or die. Other folks might—this is unknown—have a natural immunity. And because testing has been so haphazard, we flatly do not know how many people fall into the asymptomatic or immune categories.^[11] Either way, it basically amounts to the disease stopping when it runs out of people to infect.

Update, April 5, 2020: The Washington Post confirms that even some who die from COVID-19 aren’t being counted as having died from COVID-19 because they weren’t confirmed to have suffered COVID-19.^[12] I have inserted the citation.

1. ^[1]Philip Bump, “Decoding the graphs that may have saved millions of American lives,” Washington Post, April 1, 2020, https://www.washingtonpost.com/politics/2020/04/01/decoding-graphs-that-may-have-saved-millions-american-lives/↩
2. ^[2]Emma Brown, Beth Reinhard, and Aaron C. Davis, “Coronavirus death toll: Americans are almost certainly dying of covid-19 but being left out of the official count,” Washington Post, April 5, 2020, https://www.washingtonpost.com/investigations/coronavirus-death-toll-americans-are-almost-certainly-dying-of-covid-19-but-being-left-out-of-the-official-count/2020/04/05/71d67982-747e-11ea-87da-77a8136c1a6d_story.html↩
3. ^[3]Carolyn Y. Johnson, “Testing coronavirus survivors’ blood could help reopen U.S.,” Washington Post, March 31, 2020, https://www.washingtonpost.com/health/2020/03/31/coronavirus-serology-blood-tests/; Derek Thompson, “All the Coronavirus Statistics Are Flawed,” Atlantic, March 26, 2020, https://www.theatlantic.com/ideas/archive/2020/03/fog-pandemic/608764/↩
4. ^[4]Christopher Weaver, “Questions About Accuracy of Coronavirus Tests Sow Worry,” Wall Street Journal, April 2, 2020, https://www.wsj.com/articles/questions-about-accuracy-of-coronavirus-tests-sow-worry-11585836001↩
5. ^[5]Christopher Weaver, “Questions About Accuracy of Coronavirus Tests Sow Worry,” Wall Street Journal, April 2, 2020, https://www.wsj.com/articles/questions-about-accuracy-of-coronavirus-tests-sow-worry-11585836001↩
6. ^[6]William Wan et al., “Experts and Trump’s advisers doubt White House’s 240,000 coronavirus deaths estimate,” Washington Post, April 2, 2020, https://www.washingtonpost.com/health/2020/04/02/experts-trumps-advisers-doubt-white-houses-240000-coronavirus-deaths-estimate/↩
7. ^[7]Philip Bump, “Decoding the graphs that may have saved millions of American lives,” Washington Post, April 1, 2020, https://www.washingtonpost.com/politics/2020/04/01/decoding-graphs-that-may-have-saved-millions-american-lives/↩
8. ^[8]Teghan Simonton, “Pittsburgh professors see flaws in coronavirus modeling, predict more grim outlook,” Pittsburgh Tribune-Review, April 3, 2020, https://triblive.com/local/pittsburgh-allegheny/pittsburgh-professors-see-flaws-in-coronavirus-modeling-predict-more-grim-outlook/↩
9. ^[9]Joe Pinsker, “The Four Possible Timelines for Life Returning to Normal,” Atlantic, March 30, 2020, https://www.theatlantic.com/family/archive/2020/03/coronavirus-social-distancing-over-back-to-normal/608752/↩
10. ^[10]Joe Pinsker, “The Four Possible Timelines for Life Returning to Normal,” Atlantic, March 30, 2020, https://www.theatlantic.com/family/archive/2020/03/coronavirus-social-distancing-over-back-to-normal/608752/↩
11. ^[11]David Benfell, “When ‘good’ news might not be so good,” Not Housebroken, April 4, 2020, https://disunitedstates.org/2020/04/02/when-good-news-might-not-be-so-good/; Joe Pinsker, “The Four Possible Timelines for Life Returning to Normal,” Atlantic, March 30, 2020, https://www.theatlantic.com/family/archive/2020/03/coronavirus-social-distancing-over-back-to-normal/608752/; Teghan Simonton, “Pittsburgh professors see flaws in coronavirus modeling, predict more grim outlook,” Pittsburgh Tribune-Review, April 3, 2020, https://triblive.com/local/pittsburgh-allegheny/pittsburgh-professors-see-flaws-in-coronavirus-modeling-predict-more-grim-outlook/↩
12. ^[12]Emma Brown, Beth Reinhard, and Aaron C. Davis, “Coronavirus death toll: Americans are almost certainly dying of covid-19 but being left out of the official count,” Washington Post, April 5, 2020, https://www.washingtonpost.com/investigations/coronavirus-death-toll-americans-are-almost-certainly-dying-of-covid-19-but-being-left-out-of-the-official-count/2020/04/05/71d67982-747e-11ea-87da-77a8136c1a6d_story.html↩