Python embedding to read geotiff data (sentinel 3 products)

[carolineowens]

hi all - I am struggling to set up a python embedding script to read geotiff data. here are the steps I am following:

1. Use gdalwarp to convert the satellite image (UTM zone 17N) to latlon
   gdalwarp -t_srs 'EPSG:4326' sentinel-3.2024185.0703.1532_1610C.ab.L3.FL3.v960V20233_2_4.CIcyano_E.tif sentinel-3.2024185.latlon.tif
2. Use gdalinfo to find the metadata for the converted file and write the python embedding script my_gridded_pyembed.py:

import numpy as np
import rasterio

input_file = 'sentinel-3.2024185.latlon.tif'
met_data = rasterio.open(input_file).read()[0,:,:]

attrs = {
   'valid': '20240703_030000',
   'init':  '20240703_030000',
   'lead':  '000000',
   'accum': '000000',
   'name':      data_name,
   'long_name': data_name + '_word',
   'level':     'Surface',
   'units':     'None',

   'grid': {
      'type': 'latlon',
      'Ny': 2316,
      'Nx': 1704,
   # Lower Left  ( -84.6220924,  24.1618351) ( 84d37'19.53"W, 24d 9'42.61"N)
   # Upper Right ( -79.8332414,  30.6706256) ( 79d49'59.67"W, 30d40'14.25"N)
      'lat_ll': 24.1618351,
      'lon_ll': -84.6220924,
      'lat_ur': 30.6706256,
      'lon_ur': -79.8332414,
      'delta_lat':0.0028103585924006908, # (lat_ur - lat_ll)/Ny
      'delta_lon':0.0028103585680751136, # (lon_ur - lon_ll)/Nx
   }
}

print("Attributes:\t" + repr(attrs))

3. Run plot_data_plane to check the file
   ${MET_BUILD_BASE}/bin/plot_data_plane PYTHON_NUMPY my_gridded_pyembed_plot.ps 'name="my_gridded_pyembed.py ${METPLUS_DATA}/sentinel-3.2024185.latlon.tif CIcyano";'

This fails and gives the error ERROR : grid_from_python_dict() -> bad projection type: "latlon". Does anyone know what might be happening, or is anyone able to replicate this error (.tif file attached)?

Thanks very much!
sentinel-3.2024185.0703.1532_1610C.ab.L3.FL3.v960V20233_2_4.CIcyano_E.zip

[CPKalb]

Hi:

I think what may be happening here is that the grid type specification needs to have capitalization. So, for a lat/lon grid, it should look like this:
'grid': {
'type': 'LatLon',

Bot the the L's are capitalized here. When I make this change, the projection error goes away, but I get a new error:

ERROR :
ERROR : Python3_Dict::lookup_string(const char * key) -> value for key "name" not found
ERROR :

What's missing is the value of 'name' in the grid dictionary. This link provides the required input for a lat/lon grid in python embedding. The name can be whatever you want to name the grid. Scrolling down to the bullet point with LatLon in the previous lonk, the script is also missing Nlat and Nlon, and has two extra variables, lat_ur and lon_ur. I replaced Nx and Ny with Nlon and Nlat, and commented out lat_ur and lon_ur, and then plot_data_plane runs on my side.

I'm not certain exactly what the output should look like, so there may be some settings that need to be adjusted further to get the expected output. Do these changes work on your end and produce the expected output?

Christina

[carolineowens]

Thank you so much Christina! Yes, these changes worked :)

I had been looking at the grid specification strings in section 33.2 of the documentation. Are the grid strings different if not using python embedding, or is this section out of date?
thanks again!!!

Caroline

[DanielAdriaansen]

Hi @carolineowens, great to hear it's working!

Yes, I surmised that you had found the grid specification strings which are indeed different than the strings used for Python embedding. We've updated this issue: dtcenter/MET#2414 to allow case-insensitive strings to avoid this, since there's no reason the case should matter in this instance.

Apologies for the confusion!

[CPKalb]

Hi Caroline:

The grid specification strings in section 33.2 are more commonly used when regridding in MET. For python embedding there are multiple options for specifying the grid. One option is to use a string like the ones in section 33.2 directly as the grid variable, as seen in box 37.5. Another option is to specify the grid using a dictionary which is how the grid is specified in your python embedding script.

For the above two options, the grid type is unfortunately inconsistent, with the specification string requiring latlon and the python embedding version needing LatLon. As Dan said, we are hoping to address this inconsistency as part of [issue 2141 (https://github.com/dtcenter/MET/issues/2414) in MET.

It seems like it might also be useful for us to update the documentation so that all of this is more clear.

Christina

[CPKalb]

I'm going to go ahead and lock this conversation since it has been marked as answered. If you have additional questions, feel free to start a new discussion.

Christina