Starting Backintegration of Tutorial Notebooks

[fsenf]

This starts backintegration of tutorial notebooks and is far from being complete. It aims to solve #282

This will be updated in the upcoming days/weeks (you know ...). Please treat it for now as a draft until all is complete.

Check list for tutorials:

• .github/pull_request_template.md
• .readthedocs.yml
• LICENSE
• Makefile
• README.md
• conf.py
• docs/RunningTutorials.md
• docs/Testing-Sphinx-based-Rendering.md
• index.rst
• requirements.txt
• tutorials/Basics/Idealized-Case-1_Tracking-of-a-Test-Blob-in-2D.ipynb
• tutorials/Basics/Idealized-Case-2_Two_crossing_Blobs.ipynb
• tutorials/Basics/Methods-and-Parameters-for-Feature-Detection_Part_1.ipynb
• tutorials/Basics/Methods-and-Parameters-for-Feature-Detection_Part_2.ipynb
• tutorials/Basics/Methods-and-Parameters-for-Linking.ipynb
• tutorials/Basics/Methods-and-Parameters-for-Segmentation.ipynb
• tutorials/Example_OLR_Tracking_model/Example_OLR_Tracking_model.ipynb
• tutorials/Example_OLR_Tracking_satellite/Example_OLR_Tracking_satellite.ipynb
• tutorials/Example_Precip_Tracking/Example_Precip_Tracking.ipynb
• tutorials/Example_Updraft_Tracking/Example_Updraft_Tracking.ipynb
• tutorials/Example_VIS_Tracking_Satellite/Example_VIS_Tracking_Satellite.ipynb
• tutorials/Example_vorticity_tracking_model/Example_vorticity_tracking_model.ipynb

Check list for tobac examples:

• ./Example_OLR_Tracking_model/Example_OLR_Tracking_model.ipynb
• ./Example_OLR_Tracking_satellite/Example_OLR_Tracking_satellite.ipynb
• ./Example_Precip_Tracking/Example_Precip_Tracking.ipynb
• ./Example_Updraft_Tracking/Example_Updraft_Tracking.ipynb
• ./Example_Track_on_Radar_Segment_on_Satellite/track_on_radar_segment_on_satellite.ipynb

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 57.00%. Comparing base (d1f181b) to head (d16b1a1).

Additional details and impacted files

@@              Coverage Diff              @@
##           RC_v1.5.x     #334      +/-   ##
=============================================
+ Coverage      56.88%   57.00%   +0.12%     
=============================================
  Files             20       20              
  Lines           3444     3454      +10     
=============================================
+ Hits            1959     1969      +10     
  Misses          1485     1485              

Flag	Coverage Δ
unittests	57.00% <100.00%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fsenf]

Hi all,

I am coming closer to finish backintegration of tutorials

@JuliaKukulies : I got your cyclone tracking example in again :-D

The two notebooks I have not touched so far are

1. Example_VIS_Tracking_Satellite/Example_VIS_Tracking_Satellite.ipynb
2. Example_Track_on_Radar_Segment_on_Satellite/track_on_radar_segment_on_satellite.ipynb

In the first, GOES data had been retrieved from AWS using boto3. This appears to be broken ... @w-k-jones : You are using s3fs in the 2nd notebook to do the same. Could confirm that this is the smarter way to go?

In the second, I would rather use proj for projection calculations and would also re-structure it a bit. @w-k-jones : Would you be okay with this?

@freemansw1 : This provides already a few templates how to work with the decorators (Basics + Cyclone Example). I hope this is not in conflict with the typehint development.

Cheers

[w-k-jones]

In the first, GOES data had been retrieved from AWS using boto3. This appears to be broken ... @w-k-jones : You are using s3fs in the 2nd notebook to do the same. Could confirm that this is the smarter way to go?

In the second, I would rather use proj for projection calculations and would also re-structure it a bit. @w-k-jones : Would you be okay with this?

I think @freemansw1 did most of the work on these notebooks, but using s3fs and proj would be best in both cases

[github-actions]

Linting results by Pylint:

Your code has been rated at 8.72/10 (previous run: 8.72/10, +0.00)
_{The linting score is an indicator that reflects how well your code version follows Pylint’s coding standards and quality metrics with respect to the RC_v1.5.x branch.
A decrease usually indicates your new code does not fully meet style guidelines or has potential errors.}

[w-k-jones]

Two discoveries from today:

rioxarray requirement breaks proj as it installs libtiff v4.6.0, we will need to specific libtiff<=4.5.1 in the requirements for in to work correctly

The "Example_VIS_Tracking_Satellite.ipynb" notebook uses a lot of memory (>14GB), we need to look into reducing this to make it better able to run on personal computers and CI

[w-k-jones]

@w-k-jones : Thank you for the update! I agree to put VIS sat tracking aside for a while until compute time and memory consumption are reasonable.

At revision, c69f3bc we restructured the notebook examples/Example_Track_on_Radar_Segment_on_Satellite/Example_Track_on_Radar_Segment_on_Satellite.ipynb a bit. With your update to c61bd78 , these changes have been overwritten again.

These are the last three steps:

git hist --follow -- examples/Example_Track_on_Radar_Segment_on_Satellite/Example_Track_on_Radar_Segment_on_Satellite.ipynb
* 8adf5c8 2024-02-17 | Reprocess all notebooks (HEAD -> p334, fork/backintegration_of_tutorials) [William Jones]
...
* c61bd78 2024-02-17 | Update vorticity tracking and radar/satellite segmenation notebooks [William Jones]
...       
| * | | | c69f3bc 2024-01-08 | fixed notebook formatting (fabians-tutorial-update) [fziegner]

Did you merge with -X theirs?

PS: Notebook diffing is really a pain ...

Yes, apologies the notebook was broken due to a previous merge conflict so I reverted to the previous version. Let me see if I can grab the updated version

[fsenf]

Are there any other updates to this notebook other than our restructuring?

[fsenf]

@w-k-jones : No worries! I saw that I need to update some descriptive parts of this notebook anyway ... I will do it in the next hour

[fsenf]

Just realized that the example is using GOES Band 10 7.3 um -> lower-level water vapor. I guess this is a mistake and window channel was intended?

[fsenf]

@w-k-jones:
OK; it took a while, but track on radar, segment on sat is finally updated.

Following changes:

• switched to window IR channel
• included some combined plotting of the two fields

[fsenf]

@w-k-jones & @fziegner : I selected you as reviewers for this PR. Should be not too much work ... All checks passed and several pairs of eyes have been already looking at the content.

@fziegner : Could you please carefully check if some minor correction you maybe implemented, did not disappear during merge with RC1.5.x? Thanks!

[w-k-jones]

The only difference I can see is the loading of files with xarray rather than iris, but I think we should stick to iris until v1.6

[freemansw1]

Quick question on these... maybe this is another PR, but do we want these visible (in some way, shape, or form) from our documentation page?

[fsenf]

Thanks, Sean. I added the issue #409 to keep us reminded. Cheers, Fabian.

[w-k-jones]

Happy for this to be merged

[fsenf]

@w-k-jones : Thanks!
@fziegner : What is your opinion?

[freemansw1]

Can I move the milestone here to 1.5.3?

[JuliaKukulies]

Can I move the milestone here to 1.5.3?

I think that is a good idea

[fsenf]

Can I move the milestone here to 1.5.3?

Good suggestion! I would love push this PR (as one of our tobathon targets)

[fziegner]

@fsenf I opened a PR on your fork for adding a save function to the Radar/Satellite notebook. Otherwise everything looks good to me.

[fziegner]

Great, should be good to go now.

[fsenf]

@w-k-jones & @fziegner : Thank you for your additions! I will now carry out the merge.