Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

476 analysis comparison; COUNTRY=myanmar #880

Merged
merged 49 commits into from
Jul 24, 2023

Conversation

jbperidypathtech
Copy link
Collaborator

@jbperidypathtech jbperidypathtech commented Jun 19, 2023

This fixes issue 476.

How to test the feature:

  • Use Myanmar
  • Choose Hazard layer "Flood extend", you should be able to:
  • Select Exposure option in statistic
  • When selected, two new inputs appear to fill the operator and the associated value

Notice: I tried to improve the responsivity of the app when we change the statistic (to max to min for example) because values are available in the front. But there is a lot of thing to change and I am not sure of the impacts. So I think it should be great to create new ticket to globally improve the left menu UX to auto-recalculate statistic table and map when any configuration value in the left menu change.

Screenshot/video of feature:
Capture d’écran du 2023-06-23 10-28-07

@ericboucher ericboucher changed the title 476 analysis comparison 476 analysis comparison; COUNTRY=myanmar Jun 20, 2023
@github-actions
Copy link

github-actions bot commented Jun 20, 2023

Build succeeded and deployed at https://prism-880.surge.sh
(hash 1a72da3 deployed at 2023-07-22T08:16:18)

@@ -111,6 +111,7 @@ const operations = {
const ceil = sortedValues.length / 2;
return (sortedValues[floor] + sortedValues[ceil]) / 2;
},
intersect_percentage: (data: number[]) => mean(data),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about calculation required for intersect_precentage. Let me know if we need to change it.

Copy link
Collaborator

@ericboucher ericboucher Jun 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I think we need to error out in this case or at least raise a warning. With just the percentage we can't give an accurate number here from the frontend. But this shouldn't be triggered atm as the analysis is run by the backend, no?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I don't really understand what is the usage of this operator. I can see it is used by the loadFeaturesClientSide function that is called by the fetchImpactLayerData. I was thinking it was used to display the layer with the selected statistic but it seems not. If I change with "intersect_percentage: () => null" it continues working fine. I change this line because of typescript but I don't know the usage/impact and what to do with it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if I recall correctly this is used to make the analysis calculations locally, if you don't have access to a stats API.
It is less relevant now but at the beginning, some deployments wanted to be "independent" from the API.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok understood ! I change to raise an error if trying to calculate intersect_percentage from client side then

@wadhwamatic
Copy link
Member

I see this is WIP, but wanted to mention there's an error in the calculation where an entire area is listed as flooded (100%) which is clearly not the case.

Screenshot 2023-06-27 at 10 47 22

@jbperidypathtech
Copy link
Collaborator Author

Hi @wadhwamatic,
Thanks for the test. I just checked and the problem seems to come from the API.

When we query the stats with: intersect_comparison | ">=3"

We get in response for Thegon TS_PCODE: MMR008005

    {
        "TS_PCODE": "MMR008005",
        "stats_min": 3.0,
        "stats_max": 3.0,
        "stats_mean": 3.0,
        "stats_sum": 33.0,
        "stats_std": 0.0,
        "stats_median": 3.0,
        "stats_intersect_percentage": 1.0
    },

@ericboucher
Copy link
Collaborator

Yes this is an edge case with the API.

That particular grid is mostly empty and the API returns the percentage of pixels with the correct value. But it cannot count empty pixels. So what it means is that 100% of the grid pixels available in this area have a value of 3.

The API should work well with normal rasters already. I'll look into a way to count/add empty pixels as well.

@ericboucher
Copy link
Collaborator

The API edge case should be resolved by #893

@ericboucher
Copy link
Collaborator

ericboucher commented Jun 29, 2023

@jbperidypathtech copy-pasting from Slack to share context:

After discussing with Amit, we would like you to investigate the following behavior, if it’s not too tricky to handle on the frontend (I know most of the components are built with one metric in mind so it might be 🤷 )

  • use the intersect_percentage as the main value (in the legend when using admin boundaries)
  • display both intersect_percentage and intersect_area metrics where possible (mainly in the table and in the tooltip)
  • rename the metric selector to “Area exposed”

In addition, I noticed that the table ordering is not working correctly for percentages atm

@wadhwamatic
Copy link
Member

@jbperidypathtech - this is looking good. There's a couple of things I'll need to sync with @ericboucher on and will share more feedback afterwards. In the meantime, I tried running an analysis on flood extent, and then another one on rainfall. The tooltip label still referred to flood extent:

Screenshot 2023-07-18 at 11 14 28

@wadhwamatic
Copy link
Member

@jbperidypathtech - I just sync'd with @ericboucher on this and agreed that a couple of things should be added here:

  1. We should filter out areas where 0% was exposed. They shouldn't be colored in the map, and don't need to appear in the table. Eric suggested we can handle this in the backend
  2. When a user chooses 'area exposed' as the statistic in the analysis tab, it's not clear what the threshold parameter refers to. Let's add % to the right of the above input field when the statistic is 'area exposed'. See screenshot below
  3. The user input on the threshold value is not scaled. So, right now I have to enter 0.01 to set the threshold to 1%. We should handle the scaling to avoid this
Screenshot 2023-07-19 at 11 16 57

@jbperidypathtech
Copy link
Collaborator Author

Hi @wadhwamatic,
Last comment should be done :)
image

Comment on lines +410 to +411
# and filter out properties that have "no" intersection
# by setting a limit at 0.005 (0.5%).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wadhwamatic As it happened there were actually very few admin boundaries with zero value, but very small percentages. I set an arbitrary limit of 0.5%. Let me know what you think.

@jbperidypathtech jbperidypathtech merged commit 23df73f into master Jul 24, 2023
6 checks passed
@jbperidypathtech jbperidypathtech deleted the 476-analysis-comparison branch July 24, 2023 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expand analysis feature to allow comparison for different data types while also using operators
3 participants