Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ferret 7.6 won't handle 5d data #1988

Open
apcraig opened this issue Jul 19, 2021 · 2 comments
Open

ferret 7.6 won't handle 5d data #1988

apcraig opened this issue Jul 19, 2021 · 2 comments

Comments

@apcraig
Copy link

apcraig commented Jul 19, 2021

This seems to be related to recent changes #1962.

We have a large data file with

dimensions:
	d2 = 2 ;
	ni = 5148 ;
	nj = 4400 ;
	nc = 5 ;
	nkice = 7 ;
	nksnow = 1 ;
	nkbio = 9 ;
	nkaer = 11 ;
	time = UNLIMITED ; // (1 currently)
	nvertices = 4 ;
variables:
	double time(time) ;
		time:long_name = "model time" ;
		time:units = "days since 0000-01-01 00:00:00" ;
		time:calendar = "NoLeap" ;
		time:bounds = "time_bounds" ;
	double time_bounds(time, d2) ;
		time_bounds:long_name = "boundaries for time-averaging interval" ;
		time_bounds:units = "days since 0000-01-01 00:00:00" ;
	float TLON(nj, ni) ;
		TLON:long_name = "T grid center longitude" ;
		TLON:units = "degrees_east" ;
		TLON:missing_value = 1.e+30f ;
		TLON:_FillValue = 1.e+30f ;
	float TLAT(nj, ni) ;
		TLAT:long_name = "T grid center latitude" ;
		TLAT:units = "degrees_north" ;
		TLAT:missing_value = 1.e+30f ;
		TLAT:_FillValue = 1.e+30f ;
	float ULON(nj, ni) ;
		ULON:long_name = "U grid center longitude" ;
		ULON:units = "degrees_east" ;
		ULON:missing_value = 1.e+30f ;
		ULON:_FillValue = 1.e+30f ;
	float ULAT(nj, ni) ;
		ULAT:long_name = "U grid center latitude" ;
		ULAT:units = "degrees_north" ;
		ULAT:missing_value = 1.e+30f ;
		ULAT:_FillValue = 1.e+30f ;
		ULAT:comment = "Latitude of NE corner of T grid cell" ;
	float NCAT(nc) ;
		NCAT:long_name = "category maximum thickness" ;
		NCAT:units = "m" ;
	float VGRDi(nkice) ;
		VGRDi:long_name = "vertical ice levels" ;
		VGRDi:units = "1" ;
	float VGRDs(nksnow) ;
		VGRDs:long_name = "vertical snow levels" ;
		VGRDs:units = "1" ;
	float VGRDb(nkbio) ;
		VGRDb:long_name = "vertical ice-bio levels" ;
		VGRDb:units = "1" ;
	float VGRDa(nkaer) ;
		VGRDa:long_name = "vertical snow-ice-bio levels" ;
		VGRDa:units = "1" ;
	float tmask(nj, ni) ;
		tmask:long_name = "ocean grid mask" ;
		tmask:coordinates = "TLON TLAT" ;
		tmask:missing_value = 1.e+30f ;
		tmask:_FillValue = 1.e+30f ;
		tmask:comment = "0 = land, 1 = ocean" ;
	float blkmask(nj, ni) ;
		blkmask:long_name = "ice grid block mask" ;
		blkmask:coordinates = "TLON TLAT" ;
		blkmask:comment = "mytask + iblk/100" ;
		blkmask:missing_value = 1.e+30f ;
		blkmask:_FillValue = 1.e+30f ;
...
	float keffn_top_d(time, nc, nj, ni) ;
		keffn_top_d:units = "W/m^2/K" ;
		keffn_top_d:long_name = "effective thermal conductivity of the top ice layer, ca
" ;
		keffn_top_d:coordinates = "TLON TLAT NCAT  time" ;
		keffn_top_d:cell_measures = "area: tarea" ;
		keffn_top_d:missing_value = 1.e+30f ;
		keffn_top_d:_FillValue = 1.e+30f ;
		keffn_top_d:cell_methods = "time: mean" ;
		keffn_top_d:time_rep = "averaged" ;
	float Tinz_d(time, nc, nkice, nj, ni) ;
		Tinz_d:units = "C" ;
		Tinz_d:long_name = "ice internal temperatures on CICE grid" ;
		Tinz_d:coordinates = "TLON TLAT VGRDi NCAT" ;
		Tinz_d:cell_measures = "area: tarea" ;
		Tinz_d:missing_value = 1.e+30f ;
		Tinz_d:_FillValue = 1.e+30f ;
		Tinz_d:cell_methods = "time: mean" ;
		Tinz_d:time_rep = "averaged" ;
	float Sinz_d(time, nc, nkice, nj, ni) ;
		Sinz_d:units = "ppt" ;
		Sinz_d:long_name = "ice internal bulk salinity" ;
		Sinz_d:coordinates = "TLON TLAT VGRDi NCAT" ;
		Sinz_d:cell_measures = "area: tarea" ;
		Sinz_d:missing_value = 1.e+30f ;
		Sinz_d:_FillValue = 1.e+30f ;
		Sinz_d:cell_methods = "time: mean" ;
		Sinz_d:time_rep = "averaged" ;
	float Tsnz_d(time, nc, nksnow, nj, ni) ;
		Tsnz_d:units = "C" ;
		Tsnz_d:long_name = "snow internal temperatures" ;
		Tsnz_d:coordinates = "TLON TLAT VGRDs NCAT" ;
		Tsnz_d:cell_measures = "area: tarea" ;
		Tsnz_d:missing_value = 1.e+30f ;
		Tsnz_d:_FillValue = 1.e+30f ;
		Tsnz_d:cell_methods = "time: mean" ;
		Tsnz_d:time_rep = "averaged" ;

When we load it into ferret, we get,

           *** NOTE: Unspecified or unsupported ordering of axes in variable Tinz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
           *** NOTE: Unspecified or unsupported ordering of axes in variable Sinz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
           *** NOTE: Unspecified or unsupported ordering of axes in variable Tsnz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
           *** NOTE: Axis TIME marked as TI
           *** NOTE: but it has different directions in grids.
 **ERROR: inconsistent data grids: File needs axis attributes on coordinate variables indicating correct directions

and the file won't work. If we remove the three 5d variables (these are the only 5d variables in the file) with

ncks -x -v Tinz_d,Sinz_d,Tsnz_d hilat_GJRA55v1.4_TL319_t071_onyx_NR_gl.cice.h.2019-08-10.nc test.nc

then everything works fine in ferret on the new file. Did recent changes in ferret to "stop" with inconsistent grids lead to loss of some capability of higher dimension variables? We are using "FERRET v7.6 (optimized) Linux 4.12.14-197.51-default 64-bit - 07/08/21". The original file with 5d variables works fine with ferret "FERRET v7.5 (optimized) Linux 4.12.14-150.17-default 64-bit - 07/11/19".

One other thing I can report. If I extract the three 5d variables from the file alone, then ferret seems to work fine,

/p/work/apcraig/test>ferret
 	NOAA/PMEL TMAP
 	FERRET v7.6 (optimized)
 	Linux 4.12.14-197.51-default 64-bit - 07/08/21
 	19-Jul-21 13:31     

yes? use test3.nc 
           *** NOTE: Unspecified or unsupported ordering of axes in variable Sinz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
           *** NOTE: Unspecified or unsupported ordering of axes in variable Tinz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
           *** NOTE: Unspecified or unsupported ordering of axes in variable Tsnz_d
           *** NOTE: Check the dimensions: can USE/ORDER= be applied?
           *** NOTE: The default ordering will be used
yes? sho dat
     currently SET data sets:
    1> ./test3.nc  (default)
 name     title                             I         J         K         L         M         N
 NCAT     category maximum thickness       ...       ...       ...       1:5       ...       ...
 SINZ_D   ice internal bulk salinity       1:5148    1:4400    1:7       1:5       1:1       ...
 TLAT     T grid center latitude           1:5148    1:4400    ...       ...       ...       ...
 TLON     T grid center longitude          1:5148    1:4400    ...       ...       ...       ...
 TINZ_D   ice internal temperatures on CI  1:5148    1:4400    1:7       1:5       1:1       ...
 TSNZ_D   snow internal temperatures       1:5148    1:4400    1:1       1:5       1:1       ...
 VGRDI    vertical ice levels              ...       ...       1:7       ...       ...       ...
 VGRDS    vertical snow levels             ...       ...       1:1       ...       ...       ...
 TAREA    area of T grid cells             1:5148    1:4400    ...       ...       ...       ...

So it may be more complicated than just those variables alone. It may be how other variables are also defined in the same file and the ordering of the axis in them. Any ideas?

I'm happy to provide the file or the full ncdump of it if that's helpful. The file is 19 Gb.
@pourbiazar
Copy link

Is this problem fixed? I have a similar problem. I have a WRF output that works fine with the older version of Ferret but doesn't work with the recent release. It is perhaps due to changes made on Jan. 7, 2020 (#1962). It seems that the problem is due to having variables with mixed dimensions and order. I guess an option to avoid the default behavior of Ferret (by avoiding the error check) and allowing the user to use i/j/k/l/m... would be desirable.

Here is the header for the original file:
original.txt

When I remove the problematic variables using ncks:
ncks -O -x -v ZNU,ZNW,ZS,C1F,C1H,C2F,C2H,DZS,C3F,C3H,C4F,C4H,DN,DNW,FNM,FNP,RDN,RDNW orig.nc corrected.nc

Then I will be able to use Ferret.

@ACManke
Copy link

ACManke commented Apr 25, 2024

Grid definition is not explicitly defined by the CF standard, and so pyferret works through the variables, trying to determine the direction of the dimensions. For a file like this, the dimensions do not have associated coordinate variables, and so an attempt is made to use dimension names. More description of this process is here, https://www.pmel.noaa.gov/maillists/tmap/ferret_users/fu_2019/msg01013.html

I think that if you rename the dimensions in your files using ncks and the list above, you should be able to open the dataset. If you are running PyFerret v7.63 then all of the above names should indicate a direction, so you could use X for "west_east" and I for "west_east_stag"; Z for "bottom_top", and perhaps layer for "soil_layers_stag", and so forth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@apcraig @pourbiazar @ACManke