reading nc4 file

[perthsb]

I am trying to read nc4 file using MET 9.0.2 but getting following error message.
plot_data_plane gocart.inst_aod.20160728_1800z.nc4 test.ps 'name="AOD"; level="(0,1,,)";' -v 5

DEBUG 1: Opening data file: gocart.inst_aod.20160728_1800z.nc4
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_None".
ERROR :
ERROR : plot_data_plane -> file "gocart.inst_aod.20160728_1800z.nc4" not a valid data file
ERROR :

However, I am able to do ncdump and plot the variables in that file. File location in Hera:
/scratch1/NCEPDEV/global/Partha.Bhattacharjee/GOCART_Kate/C96/20160728

Is there different syntax needed to read nc4 files using MET ?

Thanks
Partha

[JohnHalleyGotway]

Hi Partha, I see you're having trouble getting MET to read data from a NetCDF file. Thanks for sending me the path. I was able to log onto Hera and grab it for testing. Fortunately, there's an easy way to get this working.

For some background, MET does read a few different flavors of NetCDF files:

• its internal format which is output from tools like gen_vx_mask or pcp_combine
• files following the Climate Forecast (CF) convention
• WRF output post-processed by the WRF-interp utility

The file you're reading is very close to the CF-convention. But the problem is that the global attributes do NOT state that it's CF-compliant. There is no global "conventions" attribute there. But we can tell MET how to process this file using the "file_type" option, as you can see below:

met-10.0.0/bin/plot_data_plane \
gocart.inst_aod.20160728_1800z.nc4 \
AOD.ps \
'name="AOD"; level="(0,1,*,*)"; file_type=NETCDF_NCCF;' \
-v 4

I ran at verbosity 4 so that'll print the timing and range of data values:

DEBUG 4: Data plane information:
DEBUG 4:       plane min: 0.000303123
DEBUG 4:       plane max: 7.3939
DEBUG 4:      valid time: 20160728_180000
DEBUG 4:       lead time: 000000
DEBUG 4:       init time: 20160728_180000
DEBUG 4:      accum time: 000000

And here's the image it creates.
AOD.pdf
So whenever processing this data, you'll need to include this in your config file, or on the command line (for plot_data_plane):

file_type = NETCDF_NCCF;

[perthsb]

Thanks John. I used the change in config file and use for Series grid stat but getting similar error while running it
location : /scratch1/NCEPDEV/global/Partha.Bhattacharjee/GOCART_Kate/series_stat_GccartC96_ICAPTAOD_20160708.sh
Any help on this is appreciated.

[JohnHalleyGotway]

@perthsb I logged onto Hera and took a look at this. While I did run into several roadblocks with the script itself, I did get a small test case to eventually run (with 2 timesteps instead of 120).

cd /home/John.H.Gotway/MET_Help/partha_data_20210802
module use -a /contrib/anaconda/modulefiles
module load intel/18.0.5.274
module load anaconda/latest
module use -a /contrib/met/modulefiles/
module load met/9.0.2
setenv FCSTDIR /scratch1/NCEPDEV/global/Partha.Bhattacharjee/GOCART_Kate/C96
setenv /scratch1/NCEPDEV/global/Partha.Bhattacharjee/GOCART_Kate/ICAP/July_2016
series_analysis -fcst $FCSTDIR/20160728/gocart.inst_aod.20160728_0600z.nc4 $FCSTDIR/20160728/gocart.inst_aod.20160728_1200z.nc4 -obs $OBSDIR/icap_2016072800_MME_modeaod550.nc $OBSDIR/icap_2016072800_MME_modeaod550.nc -out output_METseries_20160708/series_fh0624_20160708_ICAP_GcartC96_TAOD.nc -config SeriesAnlConfig_JulyAug16_ICAPTAOD_fh0624 -paired
...
DEBUG 1: Output file: output_METseries_20160708/series_fh0624_20160708_ICAP_GcartC96_TAOD.nc

So why isn't your script, as written, working?

cat /scratch1/NCEPDEV/global/Partha.Bhattacharjee/GOCART_Kate/series_stat_GccartC96_ICAPTAOD_20160708.sh

• The module use/load commands for Hera do not match the one listed on the Existing Builds page.
• Running that script on the command line doesn't load the needed modules... I can never remember how to do that on HPC machines.

So I do NOT think the problems you're experiencing are the same as the previous one that was solved by adding the "file_type = NETCDF_NCCF;" option.

That script looks pretty long and tedious to update. I'm wondering if you're interested in trying to get this call to Series-Analysis working through METplus? If so, once you get your script up and running, I could ask @georgemccabe to take a look and advise on doing this via METplus.

[perthsb]

Thanks John. I am able to generate output without error now. And taking your suggestion to try it with Metplus.