pcp_combine SUBTRACT wrapper issues and general questions/comments #1078
-
|
ERROR : I've been chasing this around for a bit... and couldn't work out what was wrong. Maybe this error message can be improved? It is a syntax error of sorts but the message does not point one to the fact that the field specification is incomplete, and that you need two parts: name and level in quotes e.g. -field 'name="stratiform_rainfall_amount"; level="(,)";' This error has arisen whilst trying to debug a pcp_combine wrapper for subtraction... I kept getting a level error Running pcp_combine off the command line (without the wrapper) then gave me the yyerror() because I had not added the level string. I had just specified the name part as -field "stratiform_rainfall_amount". Be that as it may, we can't understand why the wrapper is not taking the field information provided above and giving the lookup error, i.e. giving the level error even though it's specified and accepted by pcp_combine. First the wrapper does not appear to take note of the SUBTRACT in the wrapper and then proceeds to ignore the field info too. DEBUG 2: Since the "-field" command line option was not used, parsing the command line arguments a list of files, each followed by a configuration string. Finally, even though on the face of it pcp_combine without the wrapper ran through and produced an output file, we will check that pcp_combine SUBTRACT is working correctly because: ncdump One other we notice is that the output from pcp_combine has the variable name with a _01 appended. I hope that can be overriden somehow because this is undesirable. The long name stays the same, but we wouldn't want to have the CF compliant name changing... am I right in thinking that whilst the netcdf input is CF compliant, the output netcdf from pcp_combine is not? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
|
For whatever reason the editor removed my (,) in FCST_PCP_COMBINE_INPUT_LEVELS = "(,)"! |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mpm-meto,
I agree that the error message from parsing the configuration is confusing. I would have to defer to @JohnHalleyGotway to see if there is a good way to improve the error message in this case. From what I understand these types of errors can occur in various places in the config parsing, so it may be difficult to advise as to what exactly has gone wrong here.
From the log excerpt you provided, it looks like pcp_combine is being run twice, once using -add mode and once using -subtract mode. Could you also send the metplus_final.conf file that is generated in your OUTPUT_BASE directory so I can see what settings are being used for the run? If FCST_PCP_COMBINE_RUN = True and OBS_PCP_COMBINE_RUN = True, then the wrapper will run the tool twice for each run time. Regarding the call to pcp_combine in subtract mode, the level value is not currently passed into the command for that mode. There is a note in the wrapper to add support for this, although we have not yet had a use case that needed it. With the work for issue #1062 I was planning on cleaning up the wrapper so that the different run methods are handled more consistently. I will be sure to enable setting the level value in the way you have shown when I clean up the PCPCombine wrapper. I was under the impression that if the level value is "(*,*)" then you do not need to specify it explicitly (FYI, I put a backslash \ in front of the * characters so they are not interpreted as a block of bold text). Could you send me the input files so I can take a closer look?
The name of the output filename is controlled by [FCST/OBS]_PCP_COMBINE_OUTPUT_TEMPLATE, so the level (aka accumulation) is likely specified in the value for that variable. Adjusting the value will remove the extra info from the filename.
I'm not sure, but I didn't think that the name of the file is a requirement of CF compliance. However, you are correct to think that the output NetCDF file generated from pcp_combine is not CF compliant. The files generated by all of the MET tools are in their own format that is not currently CF compliant. There has been discussion to make the output CF compliant, but I am not sure how big of a task that would be. @JohnHalleyGotway may have more info on that as well. I added a note in issue #1062 to ensure that the level can be passed into -subtract mode. If you'd like to send the files I mentioned in this comment I can take a closer look at the use case. |
Beta Was this translation helpful? Give feedback.
-
|
@mpm-meto, the "name" and "level" strings are always required in those field strings. It is not the case that "(*,*)" is a default level string. However, I suppose we could add that as a default if needed. While switching from the MET NetCDF format to a CF compliant version is desirable, it would require a significant amount of work. It is not a quick and easy change, and is not realistic to do without identifying time and funding ahead of time for it. In particular, we'd need to create library code for writing grids in a CF-compliant way. So definitely doable, but we'd need to plan for it. |
Beta Was this translation helpful? Give feedback.
Hi @mpm-meto,
I agree that the error message from parsing the configuration is confusing. I would have to defer to @JohnHalleyGotway to see if there is a good way to improve the error message in this case. From what I understand these types of errors can occur in various places in the config parsing, so it may be difficult to advise as to what exactly has gone wrong here.
From the log excerpt you provided, it looks like pcp_combine is being…