generate-latest-aggregate.py: KeyError: '16056'

[jgehrcke]

In scheduled data updated, 27 minutes ago:

+ python tools/generate-latest-aggregate.py
211022-15:19:31.930 INFO: read ags.json at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../lib/../ags.json
211022-15:19:31.932 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../more-data/7di-rki-by-ags.csv
211022-15:19:31.996 INFO: latest timestamp in file: 2021-10-21 17:00:00+00:00
211022-15:19:31.996 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../more-data/7di-rl-by-ags.csv
211022-15:19:32.042 INFO: latest timestamp in file: 2021-10-22 15:00:00+00:00
211022-15:19:32.042 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../cases-rki-by-ags.csv
211022-15:19:32.087 INFO: latest timestamp in file: 2021-10-21 17:00:00+00:00
211022-15:19:32.087 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../cases-rl-crowdsource-by-ags.csv
211022-15:19:32.132 INFO: latest timestamp in file: 2021-10-22 15:00:00+00:00
211022-15:19:32.132 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../deaths-rki-by-ags.csv
211022-15:19:32.175 INFO: latest timestamp in file: 2021-10-21 17:00:00+00:00
211022-15:19:32.175 INFO: parse CSV file at /home/runner/work/covid-19-germany-gae/covid-19-germany-gae/tools/../deaths-rl-crowdsource-by-ags.csv
211022-15:19:32.217 INFO: latest timestamp in file: 2021-10-22 15:00:00+00:00
211022-15:19:32.305 INFO: expected: RL dataset does not contain AGS 11001
211022-15:19:32.306 INFO: expected: RL dataset does not contain AGS 11002
211022-15:19:32.306 INFO: expected: RL dataset does not contain AGS 11003
211022-15:19:32.306 INFO: expected: RL dataset does not contain AGS 11004
211022-15:19:32.307 INFO: expected: RL dataset does not contain AGS 11005
211022-15:19:32.307 INFO: expected: RL dataset does not contain AGS 11006
211022-15:19:32.307 INFO: expected: RL dataset does not contain AGS 11007
211022-15:19:32.308 INFO: expected: RL dataset does not contain AGS 11008
211022-15:19:32.308 INFO: expected: RL dataset does not contain AGS 11009
211022-15:19:32.308 INFO: expected: RL dataset does not contain AGS 11010
211022-15:19:32.309 INFO: expected: RL dataset does not contain AGS 11011
211022-15:19:32.309 INFO: expected: RL dataset does not contain AGS 11012
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3080, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 4554, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 4562, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: '16056'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "tools/generate-latest-aggregate.py", line 251, in <module>
    main()
  File "tools/generate-latest-aggregate.py", line 175, in main
    raise exc
  File "tools/generate-latest-aggregate.py", line 165, in main
    rl_deaths_total = df_rl_deaths[ags].iloc[-1]
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pandas/core/frame.py", line 3024, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3082, in get_loc
    raise KeyError(key) from err
KeyError: '16056'

Certainly relates to #1748

[jgehrcke]

The Risklayer sheets started to drop data for AGS 16056 entirely, context was explained in #1748.

I have repaired the data pipeline with #1842.