Add Crop Type Mapping tutorial

[burakekim]

Adding a new tutorial according to #2418

This tutorial demonstrates how to combine Sentinel-2 and CDL EuroCrops datasets using the Sentinel2CDLDataModule Sentinel2EuroCropsDataModule. It covers training a semantic segmentation model, along with evaluation and inference steps.

[nilsleh]

@burakekim Can you add the tutorial to the table of content in the rst file? Then one can view it in the CI docs to review as well :)

[burakekim]

The initial and somewhat end-to-end draft is now out:

• downloads a Sentinel-2 patch with rasterio's windowed reading
• prepares EuroCrops
• visualizes the Sentinel-2 patch and its corresponding EuroCrops mask (with matplotlib and on a dynamic map)
• trains and loads a dummy model for qualitative and quantitative evaluation

There are quite a few things I want to correct and improve:

• train on GPU for a reasonable number of epochs, with proper dataloader and Trainer hyperparameters
• maybe host the pretrained model + Sentinel-2 patch on HF?
• use a bigger Sentinel-2 patch for training and possibly download another patch for inference or use some sort of opportunistic sampling (can we do that with GridSampler?) for proper evaluation that tones down potential spatial autocorrelation
• EuroCrops has over 300 labels, but each country has its own distinct subset. The number of classes Slovakia has is still high. Shall we just turn this into a binary crop classification?
• addressing the question above comes down to what we want to do with the trained model, i.e., does it add value to form multi-class classification?
• there is a skimage dependency to visualize Sentinel-2 with percentile normalization; and folium, pyproj, shapely for plotting the Sentinel-2 and EuroCrops bounds on a dynamic map -- or is it fine to download 3rd party libraries for individual case studies?

P.S. In the next iteration, I am thinking of renaming the case study to Crop Type Classification. That would describe the task better

[adamjstewart]

Still need to actually look at the code, but here are responses to your TODOs:

train on GPU for a reasonable number of epochs, with proper dataloader and Trainer hyperparameters

Note that this needs to run in CI, preferably in seconds, not days. We can monkeypatch certain hyperparams to make this faster, but it shouldn't require a GPU.

maybe host the pretrained model + Sentinel-2 patch on HF?

Happy to do this if it makes the above faster while still getting good results.

use a bigger Sentinel-2 patch for training and possibly download another patch for inference or use some sort of opportunistic sampling (can we do that with GridSampler?) for proper evaluation that tones down potential spatial autocorrelation

Avoid big data, this needs to run in CI where we have very limited storage, don't want to wait 10 min to download data during a tutorial. Not sure what you mean by opportunistic sampling, but there are various GeoDataset splitting methods that you can use to chop a tile into east/west splits, grids, etc.

EuroCrops has over 300 labels, but each country has its own distinct subset. The number of classes Slovakia has is still high. Shall we just turn this into a binary crop classification? addressing the question above comes down to what we want to do with the trained model, i.e., does it add value to form multi-class classification?

I think both add value. Basically, we should have some kind of binary semantic segmentation application, and some kind of multiclass semantic segmentation application. They don't both have to be for agriculture though. For binary, something like building mapping may make more sense.

Also, tasks involving agriculture benefit greatly from time-series data. I'm planning on extending this tutorial for time series once we add support for it. So don't worry too much about the details right now, they will change in the future. This will also make the big data problem even worse, so keep the images small for now.

there is a skimage dependency to visualize Sentinel-2 with percentile normalization; and folium, pyproj, shapely for plotting the Sentinel-2 and EuroCrops bounds on a dynamic map -- or is it fine to download 3rd party libraries for individual case studies?

Would prefer to avoid any additional dependencies if we can. Any reason we can't plot a static map with matplotlib? eurocrops.plot(sample) and sentinel2.plot(sample) should get you pretty far. If we do need to add additional deps, they need to be installed in .github/workflows/tutorials.yaml and .github/workflows/release.yaml like we did with planetary_computer. But I'm trying to get rid of those too, since they aren't absolutely necessary and aren't tracked by dependabot like our formal deps.

P.S. In the next iteration, I am thinking of renaming the case study to Crop Type Classification. That would describe the task better

I agree with the rename. Both "Crop Classification" and "Crop Type Mapping" are common names. I think the latter may actually be even more common, and more technically correct. A computer vision person may argue that this is semantic segmentation, not classification. Of course, semantic segmentation is just pixelwise classification, so the distinction isn't too important.

[burakekim]

Re: Using a dynamic map of overlaid mask and Sentinel-2 AOIs: I think it just looks nice, which is not a legit reason to keep it. I will discard it later

Re: Naming: Changed to crop type mapping, although I do not think the mapping wording perfectly fits the task -- not in the technical sense, but in the practical sense. We do not often say, "In this tutorial, we map crop types[...]" but rather, "In this tutorial, we classify crop types[...]" Still, I am fine with the current naming. As for classification vs segmentation, I lean toward segmentation

Re: CPU-friendly pipeline: With the current setup, each epoch takes ~8 minutes. I was targeting a training+test of 30mins on my laptop, which gives me a budget of 3 epochs. However, this way, the prediction does not look pretty at all for 10-class classification. We can do one or both: increase the tutorial time budget or lower the number of classes from 10 to, say, 3 (three most occuring classes in our AOI). What are your priorities?

Re: Uploading the weights and Sentinel-2 patch on HF: This does not seem to offer a great deal of convenience unless we have a particular focus on allowing users to have the pretrained weights and Sentinel-2 patch and skip straight to inference -- which I do not think is the case because that bypasses the end-to-end showcase of TG abilities, which is the key feature of this tutorial

Re: Multi-class setting: I first listed all the crop type classes that fall under our AOI, got the 10 crop type classes with the highest occurrence, and set that as the classes argument

Thinking about it, we could consider a method for the users to choose the top-N crop types based on their occurrences for their AOIs. Otherwise, for larger AOIs (ours is fairly small and initially had 30 classes and for example, 239 for Estonia), the number of crop types might make the task challenging for those with specific experimental needs (we do not likely need 239-class classification task). This would be another PR though

[adamjstewart]

With the current setup, each epoch takes ~8 minutes. I was targeting a training+test of 30mins on my laptop, which gives me a budget of 3 epochs.

Change this from minutes to seconds and then this might get merged. Imagine presenting a tutorial in person and clicking "run" and telling people to just wait 30 min and it will be ready. It isn't hard to hack this to be fast in CI, but it should also be fast in person as well. This is where pre-trained models can help.

Exact number of crop types that is appropriate depends on the region, but 10 sounds okay to me.