Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading LI L2 point data is not daskified #2814

Open
gerritholl opened this issue Jun 13, 2024 · 0 comments
Open

Reading LI L2 point data is not daskified #2814

gerritholl opened this issue Jun 13, 2024 · 0 comments

Comments

@gerritholl
Copy link
Collaborator

Describe the bug

Loading "radiance" from a LI L2 LGR file gives me the data directly, instead of a dask-aware array.

To Reproduce

# Your code here
from satpy import Scene
from satpy.utils import debug_on; debug_on()
from glob import glob
sc = Scene(filenames=glob("/media/nas/x21308/MTG_test_data/LI/20230712/*LGR*_0042_003*.nc"), reader="li_l2_nc")
sc.load(["radiance"])
print(sc["radiance"])

Expected behavior

I expect that the NetCDF dataset encapsulates a dask array.

Actual results

It encapsulates a regular array, with data values, as follows:

[DEBUG: 2024-06-13 12:47:02 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2_nc.yaml',)                                                                                                    [5/1066]
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.yaml_reader] Assigning to li_l2_nc: ['/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023504_L2PF_IV_20230713065623
_20230713065633_N__T_0042_0039.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023408_L2PF_IV_20230713065513_20230713065523_N__T_0042_0032.nc', '/media/n
as/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023432_L2PF_IV_20230713065543_20230713065553_N__T_0042_0035.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUME
TSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023424_L2PF_IV_20230713065533_20230713065543_N__T_0042_0034.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK
-BODY---NC4E_C_EUMT_20231221023448_L2PF_IV_20230713065603_20230713065613_N__T_0042_0037.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023456_L2PF_IV_20
230713065613_20230713065623_N__T_0042_0038.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023440_L2PF_IV_20230713065553_20230713065603_N__T_0042_0036.nc
', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023352_L2PF_IV_20230713065453_20230713065503_N__T_0042_0030.nc', '/media/nas/x21308/MTG_test_data/LI/202307
12/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023416_L2PF_IV_20230713065523_20230713065533_N__T_0042_0033.nc', '/media/nas/x21308/MTG_test_data/LI/20230712/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+LI-2-
LGR--FD--CHK-BODY---NC4E_C_EUMT_20231221023400_L2PF_IV_20230713065503_20230713065513_N__T_0042_0031.nc']
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Product type is: 2-LGR
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_base_nc] Adding 10 datasets for 2-LGR input product.
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.li_l2_nc] The current product 2-LGR is not an accumulated product so it will not be regridded.
[DEBUG: 2024-06-13 12:47:02 : satpy.composites.config_loader] Looking for composites config file li.yaml
[DEBUG: 2024-06-13 12:47:02 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2024-06-13 12:47:02 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/pyorbital/etc
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.yaml_reader] No coordinates found for DataID(name='longitude', modifiers=())
[DEBUG: 2024-06-13 12:47:02 : satpy.readers.yaml_reader] No coordinates found for DataID(name='latitude', modifiers=())
<xarray.DataArray 'radiance' (y: 7105)> Size: 28kB
array([ 6. ,  4. , 13. , ...,  8. ,  6.5, 22. ], dtype=float32)
Coordinates:
    crs      object 8B +proj=longlat +ellps=WGS84 +type=crs
Dimensions without coordinates: y
Attributes: (12/18)
    long_name:            Radiance of group
    units:                mW.m-2.sr-1
    use_rescaling:        True
    sensor:               li
    add_offset:           0.0
    name:                 radiance
    ...                   ...
    start_time:           2023-07-13 06:54:53
    end_time:             2023-07-13 06:56:33
    reader:               li_l2_nc
    area:                 Shape: (7105,)\nLons: <xarray.DataArray 'longitude'...
    _satpy_id:            DataID(name='radiance', modifiers=())
    ancillary_variables:  []

Environment Info:

  • OS: openSUSE Leap 15.3
  • Satpy Version: main branch v0.49.0-55-g367016e29

Additional context

I encapsulated the whole thing in with dask.config.set(scheduler=CustomScheduler(max_computes=0)), expecting it to fail, but it doesn't fail. So it looks like nothing is being computed, it just never is a dask array to begin with.

Reading gridded products, such as AFA, correctly results in a dask array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant