Investigate NaN pixels in DEM

[kvantricht]

By accident stumbled upon a weird DEM result for a patch extraction in Russia, generated in job j-240830089c57467a8873ee79c0c5a6d2. COP-DEM is a non-temporal band but gets replicated along the t axis. However, some nodata patches appear, but only in the first temporal element. Visually in RGB the first 2 temporal bands look like:

And a timeseries in one of these red patches looks like:

We should find out why this first element is nodata.

[kvantricht]

Reproduce DEM + S2 for this case without GFMAP and see if the issue persists.

[jdries]

The gaps in the dem seem to coincide with the gaps due to clouds in sentinel-2.
The sentinel-1 bands for first date are also fully nodata, not sure if that is normal.

Joining the DEM cube to the temporal cube will only give results at timesteps where the temporal cube has data, hence the issue.
This is also another side effect of optimizing away tiles that are fully nodata.

[kvantricht]

No clue why for S1 this would be the case. For S2: we don't interpolate, because Presto handles missing data natively. However, for DEM this is not the case. By the way:

Joining the DEM cube to the temporal cube will only give results at timesteps where the temporal cube has data, hence the issue.

Is that really true? Because we don't interpolate optical, S2 has more no data timesteps, while DEM only has missing data in the first one. And the real question then is: how can we work around this?

[jdries]

I believe the missing S1 is the real issue, it's a monthly composite, so there should really be data, and also Copernicus Browser seems to confirm this. When that is solved, I expect the DEM to also appear.

The missing S1 is the problem, and it's also normal: the date range starts at 07-30, and first available sentinel-1 is: S1A_IW_GRDH_1SDV_20200802T031408_20200802T031433_033722_03E88C_66C6_COG.SAFE

[jdries]

Ok, I found what we can improve:
https://github.com/Open-EO/openeo-geotrellis-extensions/blob/a201d12defcf6d3558ac18816646d49164c77356/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/OpenEOProcesses.scala#L568

the apply_dimension on bands which is done on the aggregated Sentinel-1 data, will remove all tiles that are fully nodata.
It is actually cleaner to replace them with our internal 'empty tile', allowing subsequent operations to still fill in the nodata if needed.

This might have a small performance impact, but seems better for correctness and not giving unexpected results like this one.

[kvantricht]

I believe the missing S1 is the real issue, it's a monthly composite, so there should really be data, and also Copernicus Browser seems to confirm this. When that is solved, I expect the DEM to also appear.

The missing S1 is the problem, and it's also normal: the date range starts at 07-30, and first available sentinel-1 is: S1A_IW_GRDH_1SDV_20200802T031408_20200802T031433_033722_03E88C_66C6_COG.SAFE

this is now fixed on our client side already, where we enforce start date as start of the month.
however, with the recent absence of S1 in some regions, this problem will reoccur, so we can't trust the S1 cube to always have data.