Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add weather data infographic [don't merge] #1551

Draft
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

Hedingber
Copy link

@Hedingber Hedingber commented Oct 20, 2020

This PR adds a weather infographic which includes the following list of the following data:

{
    'severity': severity code ([0-3], int),
    'severity_hebrew': severity hebrew name (str),
    'amount_of_accidents_caused_by_rain': ammount of accidents (of this severity) which their rain rate is above our threshold and therefore counted as "caused by rain" (int),
    'accidents_caused_by_rain_percentage': the percentage of the accidents (of this severity) caused by rain out of all of the accidents (of this severity) (int),
}

This allows to show infographic that shows amount of accidents caused by rain (and their percentage) by severity

Notes:

  • This PR marked as don't merge since we currently don't have a token that enable us to get data from the IMS (Israel meteorological service) API, for now there just random function there (get_weather_at_station function in weather_interpolator.py)
  • Accident is counted as caused by rain when its rain_rate is above the ACCIDENT_RAIN_RATE_THRESHOLD which is defined in infographics_utils.py
  • This PR added call to the ensure_accidents_weather_data function after every import of new accidents from the cbs.
    Basically backfill will happen automatically the next time we'll get update from the cbs (the function ensures all db records, not just new ones)
    But if we don't want to wait we can do it manually, it can be done by running:
    python main.py process weather-data
    Running it on the whole history is pretty heavy, so we can separate it to several batches by using the start_date filter, e.g.
    python main.py process weather-data --start_date 01-01-2019
    After the backfill everytime new accidents are inserted from new data from the cbs the ensure_accidents_weather_data function will be triggered which will fill weather data for the new accidents

@codecov
Copy link

codecov bot commented Oct 20, 2020

Codecov Report

Merging #1551 (2efdb94) into dev (ba91a6d) will decrease coverage by 0.54%.
The diff coverage is 86.41%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev    #1551      +/-   ##
==========================================
- Coverage   50.62%   50.07%   -0.55%     
==========================================
  Files          52       54       +2     
  Lines        5485     5665     +180     
==========================================
+ Hits         2777     2837      +60     
- Misses       2708     2828     +120     
Flag Coverage Δ
unittests 50.07% <86.41%> (-0.55%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
anyway/db_views.py 0.00% <ø> (ø)
anyway/parsers/cbs/executor.py 0.00% <0.00%> (ø)
main.py 0.00% <0.00%> (ø)
anyway/models.py 90.85% <76.19%> (-0.22%) ⬇️
anyway/infographics_utils.py 85.81% <77.41%> (-0.49%) ⬇️
anyway/parsers/cbs/weather_data.py 100.00% <100.00%> (ø)
anyway/parsers/cbs/weather_interpolator.py 100.00% <100.00%> (ø)
anyway/parsers/location_extraction.py 24.74% <0.00%> (-46.91%) ⬇️
anyway/parsers/news_flash_db_adapter.py 60.86% <0.00%> (-8.70%) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba91a6d...2efdb94. Read the comment docs.

@Hedingber Hedingber changed the title Add weather data infographic [WIP] Add weather data infographic [don't merge] Oct 20, 2020
@Hedingber Hedingber marked this pull request as ready for review October 20, 2020 21:50
@idosavion
Copy link

FYI: you can just mark it as draft

@Hedingber
Copy link
Author

Hedingber commented Oct 21, 2020

@idosavion
It was marked as draft in the beginning
But since 99% of the logic is in, I wanted to move on and start with the review <-> fix cycles, therefore marked it as ready for review (but left the [don't merge] in the title)

@idosavion
Copy link

@Hedingber can you please split this one to a few logically separated commits?
It's a bit hard to review a 2000 line change without missing anything :)

@Hedingber Hedingber force-pushed the add_weather_infographic branch 2 times, most recently from b80676a to bb49592 Compare November 18, 2020 19:39
@Hedingber Hedingber force-pushed the add_weather_infographic branch from bb49592 to 2efdb94 Compare November 18, 2020 19:55
@Hedingber
Copy link
Author

@idosavion Done

Copy link

@idosavion idosavion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 👏

@@ -1041,6 +1046,66 @@ def pedestrian_injured_in_junctions_mock_data(): # Temporary for Frontend
]


@WidgetCollection.register
class AccidentCausedByRainWidget(Widget):
# the rain rate threshold after which we count the accident as a cause of the rain

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe .. as caused by rain?

@atalyaalon atalyaalon marked this pull request as draft March 31, 2022 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants