Coronavirus: Have we been right to base our response on data models?

COVID-19 is the greatest public health crisis in a century, but, paradoxically, public health considerations are not the primary drivers of policy response to the global spread of SARS-CoV-2, the virus that causes the disease. Instead, political leaders have largely ceded priority to mathematical models. These models use descriptive epidemiology, viral dynamics, and human behavioral factors as inputs, along with judgment about a multitude of unknown or uncertain parameters to navigate the huge uncertainties and data gaps that accompany any fast-moving health crisis like this one. Taken as part of a broader scientific and public health evidence base, models like these can provide useful input for decision-making. However, focusing models on a single indicator, such as COVID-19 deaths, could result in a lack of understanding of relevant trade-offs and the overall health effect of the pandemic on society.

The initial COVID-19 models, which drove much of the current approach, are of questionable accuracy. First, they do not consider the real-life conditions under which the virus is transmitted in a complex urban environment. For example, real-time hospitalization data from New York City reveals that the virus hit earliest and hardest in low-income communities with medically vulnerable households. The fact that some areas can differ from others in terms of these risk factors is not accounted for in the models. Second, the models assume most people infected with the novel coronavirus would either exhibit symptoms or be hospitalized. This assumption inflates projected death rates. Third, the models make assumptions about the effectiveness of proposed mitigation strategies (e.g., school closures, social distancing) to influence transmission dynamics. However, these strategies, now being applied across a range of socioeconomic and cultural contexts, have never been systematically evaluated nor is their policy end-point clearly defined.

Many past examples of infectious disease models have overshot reality in projections of cases, death, and disruption, including the 2009 H1N1 influenza pandemic and the 2014 Ebola outbreak. Overestimated risk can appear as a win for policymakers because whatever mitigation efforts are put forward will appear to have worked. However, even the most sophisticated of today’s epidemic models explicitly exclude the complex trade-offs that accrue when countries adopt a strategy against a single endpoint and do not consider other health implications, and economic or ethical considerations.

Moreover, there is no one-size-fits-all solution. Measures proposed to mitigate the epidemic will certainly differ in every country and will depend on each country’s age profiles, the capacity of its health systems, the relative risk of COVID-19 versus other diseases, and other assumptions, including about how many people are already infected and how quickly the virus spreads. Although models can contribute to the discussion about COVID-19, they are only one of the many inputs required for sound public health decision-making. In addition to a clear and open discussion by policymakers about model uncertainty and limitations, independent scientific experts should conduct an ongoing review of policy-influencing disease models. These must be complemented by a robust understanding of the evidence base behind proposed mitigation strategies and an evaluation of their real-time effect on COVID-19 and other health-related determinants or “influencers” (such as school interruptions, job loss, poverty, and changes to care-seeking behavior). This requires consideration of the potential effect of any proposed strategy on other health outcomes, such as mental health issues, domestic violence, and lack of access to regular preventive and curative care.

The objective of public health is maximizing health and well-being. In the case of COVID-19, instead of focusing solely on minimizing COVID-19 deaths alone, policymakers should consider the totality of the public health impact during and after the pandemic. This includes consideration of risks and alternatives, and transparency about choices and trade-offs that inform policymakers' decisions. No mathematical model of a virus alone can do that.

The authors would like to thank Babak Pourbohloul, Lisa White, Edmond Ng, Susanna Lehtimaki, Anna Matterson, Justin Silverman, Alex Washburn, and Katherine Holland for their contributions to this piece.