Today, we present a guest post written by Jeffrey Frankel, Harpel Professor at Harvard’s Kennedy School of Government, and formerly a member of the White House Council of Economic Advisers. A shorter version appeared at Project Syndicate.
Mark Twain said, “There are three kinds of lies: lies, damned lies, and statistics.” Too often, the pandemic has unnecessarily allowed scope for the sort of popular suspicions reflected in Twain’s bon mot.
Statistics are in fact a critical component of the fight against Covid-19. Their use ranges from judging the efficacy of different vaccines to judging the performance of different governments.
But throughout the pandemic, comparisons across countries have focused too much on the wrong statistics. The problem is worse than impeding the evaluation of governments’ performance. The focus on the wrong metrics gives has given some political leaders a strong incentive to under-react to the pandemic, to suppress testing for example, and thus has arguably contributed to the loss of millions of lives.
Per capita cases
The official global death toll from Covid-19 stands at 3.3 million. One sees tendentious media reports comparing fatalities across countries or states.
For professionals analyzing the data, the more information the better. But for the purposes of the general conversation, the choice of which sort of statistics to emphasize is consequential. Everyone has limited bandwidth and in this age of mistrust of experts and mainstream media, it is important that prominence be given to statistics that are as informative and reliable as possible.
Counts of countries’ officially recorded infections and deaths have been the most prominently featured. The first thing one needs to do is pretty obvious: divide cases and deaths by population, to express them in per capita form. It is not very enlightening to hear that the United States has suffered more cases, in absolute terms, than the Netherlands. Or India than Peru.
But even reported on a per capita basis, official counts of infections and deaths are tremendously inaccurate. Specifically, they are usually gross under-estimates of the true numbers. Carriers of the virus are not counted as infected unless they are tested or even hospitalized. Victims are not counted as Covid-19 deaths unless a doctor puts that on their death certificates.
Estimates of the extent of the undercount of infections vary. The same with deaths. The World Health Organization this week reported an official mortality under-count of 50 % in the European Region and 60% in the Americas. The Economist magazine’s model of excess mortality calculates that the true number of deaths attributable to Covid-19 globally is currently in the range of 7 – 13 million, that is, about three times the standardly reported official account.
The estimated undercount varies widely across countries. It is often greater under autocracies — a factor of 5 in Russia, for example.
The undercount also tends to be more extreme in lower-income countries. Egypt, for example, has 13 times as many excess deaths as the number officially attributed to Covid-19. The lowest-income countries may not even be able to count total deaths. In that case, estimates of excess deaths can only be inferred. The Economist’s model shows an undercount factor of 14 in Sub-Saharan Africa.
For richer countries, the overall undercount is smaller, estimated at 17 % for OECD members. In the US, the estimated undercount was running at only 7 % in March-April. The American undercount is much smaller in 2021 than it was last year. In the early days of the pandemic, President Trump said there were only 15 American casualties and congratulated himself for doing such a good job (February 26, 2020). As late as July 2020, the professional estimates of true infections ran 2-to-6 times higher than the official US count.
Of course, even accurate measures of infections or deaths would be just the starting point for evaluating the effectiveness of public measures to combat the virus. One would need to control for other determinants, such as population density and age distribution, before passing judgment on the quality of government crisis management.
That official reports of cases and attributed deaths understate the true count is no secret. But the focus on official reports, which only count infections according to positive coronavirus tests or deaths according to what is on the death certificate, does more damage than merely understating the gravity of the problem. Worse, it gives politicians an incentive to reduce testing and suppress bad news, e.g., by encouraging doctors to list co-morbidities (diabetes) or complications (pneumonia) as the cause of death rather than Covid-19. Since testing, social distancing, and vaccination — most of it voluntary — have been so necessary for fighting the coronavirus, whatever undermines them causes deaths.
Some statistics that are more informative
It would be better if the media and popular discussion paid more attention to some other measures that are more informative and also less vulnerable to political influence.
The positive coronavirus test rate as a percentage of all tests administered is much more instructive, compared to positive results as a percentage of the entire population. India’s current positivity rate is a very alarming one in four. (It is as high as one in two in West Bengal. In contrast, the US positivity rate has recently fallen below 3 %.) If President Modi had focused on the positivity rate a few months ago, perhaps he would have known not to prematurely declare victory against the virus. As it was, he facilitated super-spreading mass political rallies and religious celebrations.
If the positivity rate had received more attention than the reported infection rate, it would have constructively given governments an incentive to increase testing. Instead, the focus on verified infections as a percentage of the population gave governments a dangerous incentive to reduce the extent of testing. When Donald Trump was president, he was acutely sensitive to this incentive: “When you test, you find something is wrong with people. If we didn’t do any testing, we would have very few cases” (May 14, 2020). He claimed that the US was doing more testing than most others, but the claim was completely false.
Trump was not alone among leaders. Brazilian President Jair Bolsonaro limited testing for the sake of appearances and actually cut back when Covid-19 cases accelerated sharply. Others who have sought to downplay the seriousness of the virus include Mexican President Andrés Manuel López Obrador, Russian President Vladimir Putin, U.K. Prime Minister Boris Johnson, and now-deceased Tanzanian President John Mugufuli.
In the current phase, the vaccination rate can be a revealing measure of government progress. Israel has now vaccinated 57 % of its population, Chile 41 %, the US 39 % and the U.K. 34 %, whereas most EU countries have only vaccinated around 15 % of their populations, Brazil and Mexico 9 %, and India 3 %. Within the US, the vaccination rate ranges from above 60 % in New England states to a low of 33 % in Mississippi.
Positivity rates, vaccination records, and excess mortality calculations have their own limitations as criteria of how bad things are. But they are generally more illuminating than the statistics that are most widely cited.
Epidemiological models often focus on the parameter R, the rate of exponential spread of the virus (or, after turning the corner, the rate of exponential decline). Unfortunately, data availability does not allow reliable estimation of R.
Historical comparison of death rates
To recognize the undercounts is to realize that the pandemic is even worse than one thought. But the news is already too depressing as it is. An historical perspective allows us to end on a more encouraging note — provided we handle the statistics properly.
A recent newspaper headline proclaimed that the 2020 jump in the US death rate (essentially what we have been calling excess deaths) not only was the worst in the last century, but supposedly surpassed even the global influenza of 1918: “Record Jump in the US Death Rate Last Year” (NYT, 4/25/2021). “The U.S. death rate in 2020 was the highest above normal since the early 1900s — even surpassing the calamity of the 1918 flu pandemic.”
It would seem surprising if, with all the advantages of modern medical science and greater economic resources, Americans did worse in 2020 than in 1918. The truth is, they didn’t. It is true that the federal and state governments could have done far better. But the outcome wasn’t worse than the so-called Spanish flu. The New York Times article was misleading in its description of the comparison of excess deaths between 1918 and 2020. (The mortality rate in 2020 “increased 16 percent from 2019, even more than the 12 percent jump during the 1918 flu pandemic.”)
According to the CDC, “from 2019 to 2020, the estimated age-adjusted death rate increased from 715.2 to 828.7 deaths per 100,000 population.” (Unadjusted for age, 3.4 million Americans died last year, out of a population of 330 million.) That was an increase in mortality of 0.114 percentage points [from 0.715% to 0.829%]. To be sure, the rise in 2020 was tragically large, reflecting the pandemic. But why describe the result as an increase in the death rate of 16% [calculated as .114%/.715%]?
If one compares the 2020 crisis to the rise in the age-adjusted death rate from 2,278 per 100,000 in 1917 to 2,542 per 100,000 in 1918, an increase in mortality of 0.264 percentage points [from 2.278% to 2.542%], one learns that the earlier episode was more than twice as deadly. It is misleading to describe the 1918 jump in mortality as 12 % [calculated as 2.542%/2.278 %] and therefore less severe than the recent experience.
The point is that the 2020 hit only looks worse than the 1918 hit when compared to other causes of death, which have declined by much more over the last century.
What is most striking in a graph of the death rate in the US over time – even more striking than the exceptional leaps of 1918 and 2020 – is the otherwise gradual but strong downward trend from 1910 to the present. Even the awful reversal of 2020, from 0.7% to 0.8%, still leaves the death rate below what it was as recently as in 2000.
Steven Pinker’s book Enlightenment Now is a good antidote to the impression we all get that the world is generally going to hell. Favorable long-term trends show up in statistics on life span, health, literacy, safety from violent death, escape from poverty, and more other criteria than one would have thought possible. But such historical progress is not automatic and we should not take it for granted, as the pandemic illustrates. Faith in “science, reason, and humanism,” have made it possible. So has holding leaders accountable. Good statistics help.
This post written by Jeffrey Frankel.