Search

Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021-22 and 2022-23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble is the 2 nd most accurate model measured by WIS in 2021-22 and the 5 th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change., (© 2024. This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.)

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making., Competing Interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: APP report grants from Metabiota Inc outside the submitted work. J.S. and Columbia University declare partial ownership of SK Analytics. No other authors have competing interests to declare., (Copyright: This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.)

When an influenza pandemic emerges, temporary school closures and antiviral treatment may slow virus spread, reduce the overall disease burden, and provide time for vaccine development, distribution, and administration while keeping a larger portion of the general population infection free. The impact of such measures will depend on the transmissibility and severity of the virus and the timing and extent of their implementation. To provide robust assessments of layered pandemic intervention strategies, the Centers for Disease Control and Prevention (CDC) funded a network of academic groups to build a framework for the development and comparison of multiple pandemic influenza models. Research teams from Columbia University, Imperial College London/Princeton University, Northeastern University, the University of Texas at Austin/Yale University, and the University of Virginia independently modeled three prescribed sets of pandemic influenza scenarios developed collaboratively by the CDC and network members. Results provided by the groups were aggregated into a mean-based ensemble. The ensemble and most component models agreed on the ranking of the most and least effective intervention strategies by impact but not on the magnitude of those impacts. In the scenarios evaluated, vaccination alone, due to the time needed for development, approval, and deployment, would not be expected to substantially reduce the numbers of illnesses, hospitalizations, and deaths that would occur. Only strategies that included early implementation of school closure were found to substantially mitigate early spread and allow time for vaccines to be developed and administered, especially under a highly transmissible pandemic scenario.

Background: The COVID-19 Scenario Modeling Hub convened nine modeling teams to project the impact of expanding SARS-CoV-2 vaccination to children aged 5-11 years on COVID-19 burden and resilience against variant strains., Methods: Teams contributed state- and national-level weekly projections of cases, hospitalizations, and deaths in the United States from September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of 1) vaccination (or not) of children aged 5-11 years (starting November 1, 2021), and 2) emergence (or not) of a variant more transmissible than the Delta variant (emerging November 15, 2021). Individual team projections were linearly pooled. The effect of childhood vaccination on overall and age-specific outcomes was estimated using meta-analyses., Findings: Assuming that a new variant would not emerge, all-age COVID-19 outcomes were projected to decrease nationally through mid-March 2022. In this setting, vaccination of children 5-11 years old was associated with reductions in projections for all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880-0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834-0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797-1.020) compared with scenarios without childhood vaccination. Vaccine benefits increased for scenarios including a hypothesized more transmissible variant, assuming similar vaccine effectiveness. Projected relative reductions in cumulative outcomes were larger for children than for the entire population. State-level variation was observed., Interpretation: Given the scenario assumptions (defined before the emergence of Omicron), expanding vaccination to children 5-11 years old would provide measurable direct benefits, as well as indirect benefits to the all-age U.S. population, including resilience to more transmissible variants., Funding: Various (see acknowledgments)., Competing Interests: JL has served as an expert witness on cases where the likely length of the pandemic was of issue. MCR reports stock ownership in Becton Dickinson & Co., which manufactures medical equipment used in COVID-19 testing, vaccination, and treatment. JS and Columbia University disclose partial ownership of SK Analytics. JS discloses consulting for BNI. There are no other competing interests to declare.

After a period of rapidly declining U.S. COVID-19 incidence during January-March 2021, increases occurred in several jurisdictions (1,2) despite the rapid rollout of a large-scale vaccination program. This increase coincided with the spread of more transmissible variants of SARS-CoV-2, the virus that causes COVID-19, including B.1.1.7 (1,3) and relaxation of COVID-19 prevention strategies such as those for businesses, large-scale gatherings, and educational activities. To provide long-term projections of potential trends in COVID-19 cases, hospitalizations, and deaths, COVID-19 Scenario Modeling Hub teams used a multiple-model approach comprising six models to assess the potential course of COVID-19 in the United States across four scenarios with different vaccination coverage rates and effectiveness estimates and strength and implementation of nonpharmaceutical interventions (NPIs) (public health policies, such as physical distancing and masking) over a 6-month period (April-September 2021) using data available through March 27, 2021 (4). Among the four scenarios, an accelerated decline in NPI adherence (which encapsulates NPI mandates and population behavior) was shown to undermine vaccination-related gains over the subsequent 2-3 months and, in combination with increased transmissibility of new variants, could lead to surges in cases, hospitalizations, and deaths. A sharp decline in cases was projected by July 2021, with a faster decline in the high-vaccination scenarios. High vaccination rates and compliance with public health prevention measures are essential to control the COVID-19 pandemic and to prevent surges in hospitalizations and deaths in the coming months., Competing Interests: All authors have completed and submitted the International Committee of Medical Journal Editors form for disclosure of potential conflicts of interest. Katriona Shea reports receipt of two National Science Foundation (NSF) COVID-19 RAPID awards, and a Huck Institutes of the Life Sciences Coronavirus Research Seed Grant. Rebecca Borchering reports funding from an NSF COVID-19 RAPID award. Katharine Tallaksen, Kaitlin Rainwater-Lovett, Laura Asher, Luke C. Mullany, Molly E. Gallagher, Matt Kinsey, Richard F. Obrecht, and Lauren Shin report funding from the U.S. Department of Health and Human Services (HHS), Office of the Assistant Secretary for Preparedness and Response to the Johns Hopkins Applied Physics Laboratory. Matteo Chinazzi reports grants from the National Institutes of Health (NIH), the Council of State and Territorial Epidemiologists (CSTE), and Metabiota to Northeastern University. Ana Pastore y Piontti reports funding from Metabiota, Inc. to Northeastern University and royalties from Springer Publishing. Joseph Lemaitre reports funding from the Swiss National Science Foundation, State of California, HHS, and the Department of Homeland Security (DHS). Kyra H. Grantz reports support from the California Department of Public Health, Johns Hopkins Bloomberg School of Public Health, NIH, and travel support from the World Health Organization (WHO). Elizabeth Lee and Claire Smith report support from the California Department of Public Health, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins Health System, HHS, and DHS, and computing resources from Amazon Web Services, Johns Hopkins University Modeling and Policy Hub, and the Office of the Dean at the Johns Hopkins Bloomberg School of Public Health. Justin Lessler reports support from DHHS, DHS, California Institute of Technology, NIH, honorarium from the American Association for Cancer Research, personal fees for expert testimony from Paul, Weiss, Rifkind, Wharton & Garrison, LLP. Lindsay Keegan reports support from the State of California, and NIH, a University of Utah Immunology, Inflammation, and Infectious Disease Seed Grant, and a scholarship from the University of Washington Summer Institute in Statistics and Modeling of Infectious Diseases. Lucie Contamin, John Levander, Jessica Salerno, and Willem Gijsbert van Panhuis report a National Institute of General Medical Sciences grant. Ajitesh Srivastava reports a grant from the National Science Foundation. Michael C. Runge reports stock ownership in Becton Dickinson & Co., which manufactures medical equipment used in COVID testing, vaccination, and treatment. Alessandro Vespignani reports grants from NIH, NSF, WHO, CSTE, Metabiota Inc., Templeton Foundation, Scientific Interchange Foundation, Bill & Melinda Gates Foundation; royalties from Cambridge University Press, World Scientific, Springer Publishing, and Il Saggiatore; consulting fees from Human Technopole Foundation, Institute for Scientific Interchange Foundation, honorarium for lecture module at University of Washington; Scientific Advisory Board member of the Institute for Scientific Interchange Foundation, Italy, Supervisory Board member of the Human Technopole Foundation, Italy; and gifts to Northeastern University from the McGovern Foundation, the Chleck Foundation, the Sternberg Family, J. Pallotta, and Google Cloud research credits for COVID-19 from Google. Akhil Sai Peddireddy, Pyrros A. Telionis, Anil Vullikanti, Jiangzhuo Chen, Benjamin Hurt, Brian D. Klahn, Bryan Lewis, James Schlitt, Joseph Outten, Lijing Wang, Madhav Marathe, Patrick Corbett, Przemyslaw Porebski, and Srinivasan Venkatramanan report institutional support from the National Science Foundation, Expeditions, NIH, the U.S. Department of Defense, Virginia Department of Health, Virginia Department of Emergency Management, University of Virginia (internal seed grants), and Accuweather. No other potential conflicts of interest were disclosed.

We report key epidemiologic parameter estimates for coronavirus disease identified in peer-reviewed publications, preprint articles, and online reports. Range estimates for incubation period were 1.8-6.9 days, serial interval 4.0-7.5 days, and doubling time 2.3-7.4 days. The effective reproductive number varied widely, with reductions attributable to interventions. Case burden and infection fatality ratios increased with patient age. Implementation of combined interventions could reduce cases and delay epidemic peak up to 1 month. These parameters for transmission, disease severity, and intervention effectiveness are critical for guiding policy decisions. Estimates will likely change as new information becomes available.