Corona virus disease 2019 (Covid-19) appears to have first infected humans in the Wuhan province of China sometime in late 2019.¹ In many of the earliest identified cases, pneumonia, severe acute respiratory infection symptoms, rapidly developing acute respiratory distress syndrome (ARDS), acute respiratory failure, and death have occurred. The virus has since been identified in, and is presumed to have spread to, many regions of the world. Fears of a possible pandemic have led to some level of panic in the US and around the world, with stocks of surgical grade masks disappearing from pharmacy shelves and online sellers faster than manufacturers have been able to keep up. Global stock markets have also reacted with some level of panic.

During a 25 February 2020 Senate hearing regarding 2021 budget request issues,² the Acting Secretary of the Department of Homeland Security, Chad Wolf, was asked by Senator John Kennedy of Louisiana a question about the novel corona virus now known as Covid-19: “What’s the mortality rate so far?”

Senator Kennedy clarified that he was asking about the mortality rate worldwide. Acting Secretary Wolf responded, “Worldwide I believe it’s under two percent.”

Senator Kennedy pressed, “how much under two percent?” Secretary Wolf responded, “I’ll get you an exact figure. I’ll check with CDC on … they’re monitoring the world-wide mortality rate…I will…I can get that for you.” Pressed further by the senator, Secretary Wolf eventually agreed that the rate is now estimated to be between one-and-a-half and two percent.

Senator Kennedy next asked for context. “What’s the mortality rate for influenza over, say, the last ten years in America?” Mr Wolf replied, “It’s also right around that percentage as well, I don’t have that offhand, but it’s right around two percent as well.” Asked by the senator whether he was sure of that, Mr Wolf replied simply, “Yes, sir.”

(We will get into all of this a bit more deeply in a moment, but we should note at the outset that if two percent of cases of Covid-19 have really ended in death, this would be far higher than any comparable measure of the mortality related to influenza, and would be great cause for concern relative to Covid-19. It is not at all clear that this is the case.)

Mr Wolf (who is not an epidemiologist) was in a difficult position. The public, and senators, are worried about a new and possibly very dangerous virus infecting humans around the world, and they are demanding answers. People would like answers to many important questions, including: (a) how, and how easily, is the virus spread from person to person; (b) how deadly (fatal) is this virus; and (c) how does its deadliness or fatality compare to other viruses with which we have common experience? Put simply, people want to know: “how worried should we be?”

Mr Wolf cannot possibly have definitive answers for these questions, nor can the CDC, World Health Organization (WHO), or anyone at this point. While some data that contribute to answering those questions have been collected, far more will be needed before any solid conclusions can be drawn.

The question Senator Kennedy asked about the mortality rate of Covid-19 brings us back to the idea of losing sleep over denominators (and in this instance, numerators), recalling the quip from Dr Tom Frieden, former Director of the Centers for Disease Control and Prevention (CDC), of which we had written previously (again paraphrasing):

An epidemiologist is someone who loses sleep worrying about denominators.³

The precise epidemiologic concept likely of greatest interest to Senator Kennedy is the case-fatality rate (or case-fatality proportion or ratio, as it is sometimes called). The CDC defines this as follows:⁴

Case-fatality rate

The case-fatality rate is the proportion of persons with a particular condition (cases) who die from that condition. It is a measure of the severity of the condition. The formula is:

(Number of cause-specific deaths among the incident cases ÷ Total number of incident cases) × 10ⁿ

Note that the 10ⁿ tacked on to the end of this is just for convenience. Without it, the result will be a fraction smaller than one. With an appropriate value of n, this can be turned into a percent (if n = 2), or per thousand (n=3), or per 100,000 (n=5), etc.

The CDC course materials from which this definition is taken explain further:

The case-fatality rate is a proportion, so the numerator is restricted to deaths among people included in the denominator. The time periods for the numerator and the denominator do not need to be the same; the denominator could be cases of HIV/AIDS diagnosed during the calendar year 1990, and the numerator, deaths among those diagnosed with HIV in 1990, could be from 1990 to the present.

And:

The case-fatality rate is a proportion, not a true rate. As a result, some epidemiologists prefer the term case-fatality ratio.

(We imagine some non-statisticians and non-epidemiologists might find this confusing: First, assert that CFR is not a rate, but a proportion; then call it a ratio. The point is, however, that a rate, which involves an element of time in the denominator, is different from either a proportion or a ratio, each of which need not involve time explicitly.)

According to the World Health Organization (WHO) on 25 February 2020,⁵ there were 80,239 confirmed cases of Covid-19 worldwide, and 2,700 deaths among infected persons. These numbers were increased from the day before by 908 cases and 82 deaths. The WHO is not clear in their reporting whether all of these deaths are “cause-specific” to Covid-19 (see the CDC definition of case-fatality rate), but assuming they are, the case-fatality rate would be:

2700 ÷ 80239 x 10ⁿ = 0.03365 (if n=0),

or 3.365% (if n=2).

However, this measure uses as numerator and denominator numbers of confirmed deaths (presumably deaths having something to do with Covid-19 infection) and confirmed cases. Neither of these numbers is likely comparable to any corresponding estimates used in calculating the CFR for influenza. Influenza has been with us for decades, perhaps centuries, and estimates of cases and deaths related to influenza have been continuously refined. A recent study by authors from the CDC explained the method for estimating total number of cases of influenza to be used in such calculations.⁶ The methodology is complex, and provides different estimates of the possible numbers of influenza infections in a given year depending on how severe the cases are (symptomatic, resulting in a medical visit, resulting in a hospitalization). Part of the method involves extrapolating from the number of cases resulting in hospitalization and laboratory-confirmed diagnoses of influenza to the total possible number of infections that might have occurred (resulting in any level of symptomatology) in the broader community. As the study explained:

Estimates of the case-to-hospitalization ratio, obtained from studies during the 2009 pandemic in the United States, were used to calculate the number of illness episodes that occurred in the community from the number of hospitalizations with laboratory-confirmed influenza. Estimates of the proportion of ill persons who sought medical care, obtained from a nationwide behavior survey conducted in the United States during the 2009 pandemic, were used to estimate the number of outpatient medical visits relative to the estimated number of influenza illnesses in the community.⁶

For the 2018-2019 flu season, these methods resulted in the following estimates (and 95% uncertainty intervals):⁷

Symptomatic illnesses: 35,520,883 (31,323,881 – 44,995,691)

Medical visits: 16,520,350 (14,322,767 – 21,203,231

Hospitalizations: 490,561 (387,283 – 766,472)

Deaths: 34,157 (26,339 – 52,664)

These numbers give a point estimate for the case-fatality rate of influenza of:

34,157 ÷ 35,520,883 = 0.0009616, or slightly less than one tenth of one percent.

If asymptomatic cases were included, the estimate could be still lower. But even this estimate, on first pass, seems far lower than the estimate we arrived at above for the case-fatality rate of Covid-19. In fact, the estimate we obtained for Covid-19, 3.365%, is about 35 times as high as that for influenza in 2018-2019. One might imagine this gap could only grow wider, given the “season” (if there is one) for Covid-19 may not be over yet. But whether the season is over or not is not really the issue – we are calculating proportions, so as the season wears on, assuming a relatively stable viral infection process, we should expect a stable case-fatality ratio (if we are calculating it correctly). No, the more important point here is that we do not yet know with any certainty what the true case-fatality ratio is. Even with influenza, the numbers (of which we can be more certain) are the result of a process of estimation that has evolved over many years. In the case of Covid-19, the estimation process is in its infancy.

The numerator for Covid-19 (2700 deaths as of 25 February 2020) is probably less problematic than the denominator. It is true that this count may be an underestimate for many reasons. In some cases, people may die after an infection with Covid-19 in the setting of pre-existing serious medical conditions (e.g., chronic congestive heart failure, end-stage renal disease) and no diagnosis of Covid-19 may ever be made, much less confirmed. Still, during an epidemic such as that in the Wuhan province of China, some effort to confirm any suspected case may be undertaken (conspiracy theories aside). However, even if the true number of deaths attributable to Covid-19 were double what has been reported, this would have a much smaller impact on the overall CFR than the possible undercounting of actual cases of Covid-19 may have. The denominator is our main concern.

In the estimate of CFR for Covid-19 we have put forth above, we have used as denominator the number of confirmed cases worldwide as reported by the WHO, 80,239. Consider the three numbers of cases reported by the CDC for the 2018-2019 flu season in the US: Approximately 35.5 million symptomatic cases; approximately 16.5 million medical visit cases; and approximately 0.5 million hospitalizations. To which of these is the 80,239 number of confirmed cases of Covid-19 most comparable? Given that confirmation of Covid-19 likely involves at least a medical visit, and perhaps even a hospitalization, the 0.5 million to 16.5 million may provide more appropriate comparison numbers. Using these, the CFR for flu could come out as low as 0.2% or as high as 7.0%. Our argument here is not that these are more reasonable estimates of the true CFR for influenza – We think the CDC estimate of less than one tenth of one percent is reasonable. However, we do argue that the estimate of the CFR we derived above for Covid-19 is probably more comparable to these “alternative-denominator” influenza estimates of 0.2% to 7.0%. For this reason, we are not yet convinced that Covid-19 is obviously worse than influenza. We are not unconcerned, however: influenza generally leads to (or contributes to) death mainly among the very young, the very old, or those who have significant pre-existing medical issues. Early, anecdotal evidence suggests that Covid-19 may be contributing to the deaths of a higher proportion of otherwise healthy young and middle-aged adults than is typical for the flu. There is also anecdotal evidence that some who have become infected and ill with Covid-19 may have recovered only to succumb to it a second time during this same outbreak. That is highly unusual for the flu. If these anecdotes prove to be statistically significant as more data become available, this would be cause for concern even if the overall CFR for Covid-19 were no different than that of most flu strains, the Spanish Flu of 1918 being a possible exception.⁸

The true CFR for Covid-19 is unknown. The head of the Department of Homeland Security does not know. The CDC and the WHO do not know. Anyone who feels that they do know with any certainty what the true case-fatality ratio of Covid-19 is should read the references below regarding the historically evolving picture of the estimation of CFR for influenza.^6-8 All of this is not to suggest that the experts don’t know anything. The scientists at the CDC and at the WHO are undoubtedly very knowledgeable about the evolving picture of Covid-19 infection, illness, recovery, and death. However, the picture is rapidly evolving, and just how many cases (at any level of severity) really exist (or have existed) in the world is very difficult to know at this point. Until more data have been collected, and a far better understanding of the process of infection, illness, recovery, and death have been achieved, we will continue to lose sleep over this denominator.

References

1. Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, Qiu Y, Wang J, Liu Y, Wei Y, Xia J, Yu T, Zhang X, Zhang L. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020 Feb 15;395(10223):507-513.
2. CSPAN. Department of Homeland Security Fiscal Year 2021 Budget Request. 25 February 2020. Available here.
3. Day SM, Reynolds RJ. Losing sleep over denominators, Part I: An introduction to the problem of Plioplys 1998.
4. Centers for Disease Control and Prevention (CDC). Principles of epidemiology in public health practice, third edition. US Department of Health and Human Services: Atlanta, Georgia; 2011.
5. World Health Organization (WHO). Coronavirus disease 2019 (COVID-19) Situation Report – 36. [xx https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200225-sitrep-36-covid-19.pdf?sfvrsn=2791b4e0_2 xx]
6. Rolfes MA, Foppa IM, Garg S, Flannery B, Brammer L, Singleton JA, Burns E, Jernigan D, Olsen SJ, Bresee J, Reed C. Annual estimates of the burden of seasonal influenza in the United States: A tool for strengthening influenza surveillance and preparedness. Influenza Other Respir Viruses. 2018 Jan;12(1):132-137.
7. Centers for Disease Control and Prevention (CDC). Disease burden of influenza. US Department of Health and Human Services: Atlanta, Georgia; 2020.
8. Centers for Disease Control and Prevention (CDC). 1918 pandemic (H1N1 virus). US Department of Health and Human Services: Atlant, Georgia; 2019.