-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No RKI updates since 4 days ... (Landkreis 16056 disappeared from RKI data set) #1748
Comments
Thank you @mathiasflick for the report. I had a quick look into logs and found
Looks like once again the set of amtliche gemeindeschlüssel changed in the RKI data set -- in the past that has always been a human error somewhere in the pipeline. The code might be overly strict. I might be able to precisely understand and fix this tomorrow. Hopefully. |
Data for this Landkreis were missing, recently:
|
I may want to remove the |
On vacation. Didn't get to this yet. Sorry about that :/ |
In oct 2021 the AGS 16056 disappeared from the RKI data set. That is, lacking_wrt_ref was not set([11000, 3152]) anymore but set([11000, 3152, 16056]) which is when the program expectedly crashed, leading up to issue #1748. Now, deal with unexpected loss of a column. This is accomodated for in the epsilon merge tool which forward-fills a column when it's in the base but not in the extension.
I have addressed this in #1827. |
I have looked at the data more closely to better understand what happened. The fact that 16056 disappeared from the RKI data set made me 'hope' that reporting for this Landkreis was merged with another Landkreis. Indeed, there is a pretty suspicious case numer jump for Landkreis 16063 at the time when the case count for Landkreis 16056 did not change anymore: That jump is specifically from 8579 to 10572:
The last reported case count value for Landkreis 16056 was 1975. I think we can safely conclude that on September 12, reporting for Landkreise 16056 and 16063 was merged, and reported together under AGS 16063. |
With the solution from #1827 I have now retained Landkreis 16056 in the CSV files, simply forwarding the last known value (1975). That's incorrect, the value should drop to 0 so that the sum over the Landkreise evolves more correctly. Given the relatively small number though I think I will just leave this as-is. Feedback appreciated. |
I have just looked at the columns 16056 and 16063 the RL data set. They have seemingly be synced a while ago: they contain the same values, for the entire time range of interest. (that is, the sum is also wrong) |
The two landkreise in question:
(from https://www.bik-gmbh.de/download/Gebietsreform_Thueringen_zum_GS1906.pdf) |
So, I think it's fair to say that |
Some research regarding local reporting of corona-related indicators (e.g. for Eisenach and Wartburgkreis) clearly support your assumption - although I was not able to find any kind of official confirmation. Probably it is a politically motivated move in order to get "better" (i.e. lower) numbers by averaging the high one out ... But that is just my personal opinion! By the way, the zero for Luckenwalde/Parchim is caused by a hacking incident - they are not able to deliver ... Greetings from Cologne |
Thank you Mathias for the additional insight! Huh. :) |
RL did drop the data colums for landkreis 16056 and that required further patches -- done in #1842. Both the RL and RKI heatmaps now show 16056+16063 both using the data from 16063. |
Perfect! Thank you so much for your work! |
After a little bit of research I probably found the reason for the unexpected change: |
Hey Mathias. Ouch. Thank you for that reminder. I will have to double-check, but it's likely that 7di number have been a little off for 16063 because I didn't think this through before. Thank you! Keeping track of this topic here: https://github.com/opstrace/opstrace/issues/1472 |
There are no updates to the rki files since four days now (as of 2021-10-04, 20:45 local time).
Is there a problem with changes to the input data provided by RKI?
If yes, how can I help?
Greetings from Cologne
Mathias
The text was updated successfully, but these errors were encountered: