Alternate names within netcdf dimensions

[robdarvell]

When running plot_data_plane with a netcdf file which contains a dimension on pressure levels (called pressure) I can view the various levels in the above MET task by setting the string levels=(2,*,*) for example.

I have now been testing on some of the new LAEA functionality which has been added.
We now have some netcdf files which are storing different percentile values as a third dimension (rather than pressure), but I am unable to run the MET task with any value of level except for 0. The output I get from the job is:

ERROR  : 
ERROR  : MetNcCFDataFile::data_plane(VarInfo &, DataPlane &) -> the requested vertical offset 1 for "visibility_in_air" variable is out of range (between 0 and 0).
ERROR  : 

Would there be any merit in adding extra functionality in order to allow this type of variable to also work? I think the use of percentiles may be increasing (don't know how others use them). I have also have been given files which use the word 'threshold' as this extra dimension and they get the same sort of error output.

No doubt (albeit I haven't seen it yet), someone may come up with a file with percentiles for different pressure levels in it (but that might be for a different time).

I may be doing something wrong (ie my setup isn't correct), so happy to be given the correct syntax to use:

$HOME/MET11.1.0rc2/bin/plot_data_plane \
$SCRATCH/Improver/percentile_extract_20230729T1200Z-B20230725T0030Z-visibility_at_screen_level.nc \
improver.ps \
'name="visibility_in_air";level="(1,*,*)";'

[JohnHalleyGotway]

Hi @robdarvell, thanks for writing this up in GitHub discussions. Yes! I definitely agree that we'd like to make this work. It was my understanding that dtcenter/MET#1815 actually did provide the functionality needed to support NetCDF data structured in this way.

If we're able to figure this out here, great. If not, please post that sample percentile_extract_20230729T1200Z-B20230725T0030Z-visibility_at_screen_level.nc file to our anonymous ftp site following these instructions.

However, I do want to clarify one detail. The integers in your level string level="(1,*,*)"; should be 0-based. Please run:

ncdump -h $SCRATCH/Improver/percentile_extract_20230729T1200Z-B20230725T0030Z-visibility_at_screen_level.nc

And check the dimensions of the visibility_in_air variable. What is the size of the first dimension? If it has size 1, you should use (0,*,*) instead of (1,*,*). And also note that as of dtcenter/MET#1815, you should now be able to specify the dimension INDEX as an integer or the dimension VALUE by preceeding it with the @ symbol.

For example, for a variable defined as visibility_in_air(percentile,lat,lon), assume that there's another variable named percentile that defines the percentile values. Note this is called a NetCDF coordinate variable... where there is a dimension and variable with the same name. In this case, you could be able to use either:
level=(9,*,*) to get the 2D data slice for the 10th percentile index...
or level=(@75,*,*) to get the 2D data slice for the 75th percentile value.

Please let me know how it goes and if/when you post sample data.

[robdarvell]

Think my lack of knowledge of netcdf is showing up.

Have taken a look at the ncdump of this file. In the dimensions part there is a dimension of percentile which is set to 13.
When I look at the visibility_in_air variable its first dimension is set as percentile, and then further down in the variables there is a percentile variable.

I have taken a look at my version of the code base, and tried adding a few write statements. I can work out how to set an increase to the number of levels to include and then get more info out (albeit not in the @ format method), but haven't worked out when is going on.

I have put the file within the ftp server for you to take a look at. Hopefully you can read it OK. I see there is also a bnds dimension. Hopefully that isn't causing issues with everything.

[JohnHalleyGotway]

@robdarvell thanks for sending the sample data file. I tested it and observed the same unexepected behavior you describe. I wrote this up as a new GitHub issue and requested that @hsoh-u work on it:
dtcenter/MET#2638

I recommend that we mark this discussion as being answered, and then move any future discussion about it over to the comments for that new issue.

[jprestop]

The issue referenced above has been closed. I'm going to go ahead and close this Discussion. Please feel free to post a comment in dtcenter/MET#2638 if any else is needed.