We present evaluations for US weekly incident case forecasts on a national, state-by-state, and county-by-county level.
The methodology for computing errors for incident case is straightforward: we simply compute the incident weekly cases and compare them to the latest truth data.
We are only evaluating models with forecasts for at least 40 states or 1000 counties. For models with missing state or county projections, we substitute the missing error with the mean absolute error for that state/county (among all the models with available forecasts).
As usual, all the results here are reproducable. See the Usage page for more details.
python evaluate_models_cases.py 2020-08-03 2020-08-29 --out_dir evaluations
python ../summarize_evaluations.py --weeks_ahead 4 --evaluations_dir evaluations --out_dir summary --summarize_counties
All formats and columns in the evaluations directory follow the same pattern as the forecast evaluations for deaths.
The results are summarized in the summary directory.