Even a good influenza forecasting model can benefit from internet-based nowcasts, but those benefits are limited

The ability to produce timely and accurate flu forecasts in the United States can significantly impact public health. Augmenting forecasts with internet data has shown promise for improving forecast accuracy and timeliness in controlled settings, but results in practice are less convincing, as models augmented with internet data have not consistently outperformed models without internet data. In this paper, we perform a controlled experiment, taking into account data backfill, to improve clarity on the benefits and limitations of augmenting an already good flu forecasting model with internet-based nowcasts. Our results show that a good flu forecasting model can benefit from the augmentation of internet-based nowcasts in practice for all considered public health-relevant forecasting targets. The degree of forecast improvement due to nowcasting, however, is uneven across forecasting targets, with short-term forecasting targets seeing the largest improvements and seasonal targets such as the peak timing and intensity seeing relatively marginal improvements. The uneven forecasting improvements across targets hold even when “perfect” nowcasts are used. These findings suggest that further improvements to flu forecasting, particularly seasonal targets, will need to derive from other, non-nowcasting approaches.
Author summary It has been demonstrated in retrospective settings that flu forecasting can be improved by augmenting forecasting models with internet-based nowcasts (i.e., one-week-ahead forecasts generated with internet data). The improvement of internet-assisted forecasting models, however, has not translated to overall improvement in flu forecasting of public health-relevant targets. This is partially due to the relatively simple forecasting models that nowcasts are augmenting and partially because nowcasts do not directly inform all public health-relevant targets, such as the start, the peak timing, and the peak intensity of the flu season (i.e., seasonal targets). This paper conducts an experiment to provide clarity on the benefits and limitations of augmenting flu forecasting models with internet-based nowcasts in practice. We show that 1) nowcasts can improve flu forecasting across all geographic regions, flu seasons, and public health-relevant targets, including seasonal targets, and 2) the benefits of nowcasting are uneven, as even in the perfect nowcasting setting, only marginal improvements to forecasts of seasonal targets are possible relative to short-term forecasts. This is because nowcasts provide direct information about short-term targets but only indirect information about seasonal targets. This paper supports previous findings that nowcasts can improve flu forecasting but clarifies the limitations of those improvements, both in terms of magnitude of possible improvement as well as which targets can be improved. The continued advancement of flu forecasting hinges on the development of model improvements separate from nowcasting.