Add the "state of ground" to bring in the extra snow depth reports #1368
Conversation
jiaruidong2017
requested review from
CoryMartin-NOAA,
ClaraDraper-NOAA and
TravisElless-NOAA
| @@ -2,3 +2,4 @@ mkdir: | |||
| - '{{ DATA }}/obs' | |||
| copy: | |||
| - ['{{PARMgfs}}/gdas/snow/obs/config/bufr_sfcsno_mapping.yaml', '{{ DATA }}/obs/'] | |||
| - ['{{PARMgfs}}/gdas/snow/obs/config/ioda_bufr_python_encoder.py', '{{ DATA }}'] | |||
There was a problem hiding this comment.
does the python script need copied? and should it go here? we might need to talk to @emilyhcliu about the progress of SPOC
There was a problem hiding this comment.
Thanks @CoryMartin-NOAA for your suggestions. Do you agree if I put this python code to sorc/gdas.cd/parm/snow/, so that this code can be used directly?
@emilyhcliu Any suggestions?
There was a problem hiding this comment.
I moved this py code to the parm/gdas/snow/ directory for a temporary solution. @emilyhcliu Please let me know if you have any other suggestions. Thanks.
There was a problem hiding this comment.
Scripts shouldn't be in parm/, until we move to SPOC, it should probably go here: https://github.com/NOAA-EMC/GDASApp/tree/develop/ush/ioda/bufr2ioda
There was a problem hiding this comment.
Understood. Wait for @emilyhcliu suggestions.
There was a problem hiding this comment.
Thanks for doing this @jiaruidong2017 . I have requested one small change.
|
|
||
| sogr = container.get('variables/groundState') | ||
| snod = container.get('variables/totalSnowDepth') | ||
| snod[(sogr <= 11.0) | (sogr == 15.0)] = 0.0 |
There was a problem hiding this comment.
Without having seen the data, I think there should be an if statement here - so that we only replace the snod when it was undefined / missing.
i.e., if snod > 0. (or some, slighty higher threshold), and sogr indictaes no snow cover, I assume we would want to use the snod value, not 0.
There was a problem hiding this comment.
Above (with an if statement) is all consistent with how the PR description.
There was a problem hiding this comment.
Good suggestions. I will investigate this.
@rmclaren Do you have any suggestions?
There was a problem hiding this comment.
Thanks again for the good findings. I modified to only replace the filled/missing snod.
| - bounding: | ||
| variable: totalSnowDepth | ||
| lowerBound: 0 | ||
| upperBound: 10000000 |
There was a problem hiding this comment.
For my understanding:
- is this introducing a 10 m limit on snow depth obs?
- if they are above 10 m, are they discarded, or set to 10.?
There was a problem hiding this comment.
I previously put this large number (10000m) here for removing the missing values. Here, we don't need this any more because we need to set the missing snod values to 0 when sogr satisfies the defined conditions.
There was a problem hiding this comment.
We might still have missing values though wouldn't we?
There was a problem hiding this comment.
The missing values will be removed after applying sogr conditions.
|
|
||
| return new_container | ||
|
|
||
| def create_obs_group(input_path): |
There was a problem hiding this comment.
Some comments here would be helpful to explain what we're doing - specifically what snogr is, and why we're selecting out the values that we are.
There was a problem hiding this comment.
Excellent suggestions. I will add the comments here later. Thanks.
There was a problem hiding this comment.
Add the comments to explain the steps.
There was a problem hiding this comment.
@jiaruidong2017 move this to https://github.com/NOAA-EMC/GDASApp/tree/develop/ush/ioda/bufr2ioda but change its name to something more descriptive/specific until we can move things to SPOC
| encoder = Encoder(YAML_PATH) | ||
| data = next(iter(encoder.encode(masked_container).values())) |
There was a problem hiding this comment.
Could be 1 liner data = next(iter(Encoder(YAML_PATH).encode(masked_container).values())). Up to you :)
There was a problem hiding this comment.
Thanks @rmclaren for your suggestions. Will change it.
| input_path: '{{ DATA }}/obs/{{ OPREFIX }}sfcsno.tm00.bufr_d' | ||
| obsfile: '{{ DATA }}/obs/{{ OPREFIX }}sfcsno.tm00.bufr_d' |
There was a problem hiding this comment.
Don't forget to add an issue so they do not assume the formatting of the YAML file (obsfile variable shoould not be necessary). Reading the backend engines YAML breaks encapsulation (external modules should not be dependent on the private configuration for a backend type. Some backends may not have any kind of file associated with them (like a db...).
There was a problem hiding this comment.
yes thanks @rmclaren I'm aware of that issue, I/we/others need to figure out how best to handle it on the workflow side.
This PR uses the python BUFR2IODA API to read the GTS bufr snow observations and adds to read the
state of ground(SOGR) for bringing in the extra snow depth reports. If the state of the ground says there is no snow, but the snow depth observation is missing, we should change the missing values to be 0 so that the reports can be assimilated.This PR also incorporates the latest changes from submodule from
NOAA-EMC/bufr-query.This PR contributes to NOAA-EMC/global-workflow#3093