Potential new use case: Use of METplus and python embedding for verification of WRF-LES using observations from the ARL tower array #2469
-
|
@DanielAdriaansen The project goal is to evaluate simulations produced by the WRF which was run in a VLES/LES configuration. The ground truth data consists of high temporal resolution measurements of air temperature, relative humidity, pressure, and the horizontal wind components produced by the tower mounted sensors at the 2m and 10m AGL levels. The observations data for each tower is contained in text files. The wind component data (m/s) from sonic anemometers collected at 20 Hz is contained in two separate files. One file for the 2m data and one file for the 10m data. The temperature (C), relative humidity (%) and pressure data (hPa) (thermodynamic data) collected at 1 Hz for the 2m and 10m levels are contained in one text file. The format of the observations is non-standard, consisting of one row of data corresponding to each time. No location or elevation data is included with the measurements. This data is available in a separate text file. The following provides examples of the data formats for the wind data and the thermodynamic data. Wind data: (no headers) Thermo data (has col headers): The WRF-LES output was post-processed using the UPP. The output is contained in GRIB files valid at 15-min intervals using UTC time. For verification, the goal is to use 15-min averaged observations of each variable which are valid at the same time as the WRF-LES output. I have done some preliminary testing to see how MET would handle typical tower data which I extracted manually from the files. I added a fictitious time stamp to match the UTC time of the post-processed WRF-LES output. I added representative elevation values in meters to fulfil the requirement for this data in the MET 11-column format. The following is an example of the data I used as input to MET ASCII2NC which was successfully ingested and converted to NetCDF: ADPSFC MSATWR10 20220705_120000 32.58 -106.67 1349.39 11 869.25781 1351.39 NA 298.97753 The following shows the NetCDF output from ASCII2NC in human readable format: #msg_type s_id valid_time lat lon elv var_name/gc level height qty value The elevation data I used for col 6 is based on the terrain elevation at the tower base. I adjusted the elevation values in col 9 to correspond to the elevation difference from the tower base to the level of the sensor which is either 2m AGL or 10m AGL. I converted the temperature values from C to K. Since I only have the pressure measurement at the 2 m level, I had to extrapolate to compute the pressure at the 10m level using a standard lapse rate. I assigned the message type ADPSFC so surface verification is performed on the standard 2m level variables (TMP, RH) and the standard 10m level wind component variables (UGRD, VGRD). I assigned the ADPUPA message type to these variables that are at non-standard levels which are 2m UGRD and VGRD wind components and 10m TMP so the verification would be done on vertical pressure levels. I ran Point-Stat using the post-processed WRF-LES output as the forecast input and the NetCDF observation file as the observation input and it successfully ran. The following shows some of the contents of the Matched Pair data text file: FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE OBS_LVL OBS_ELV FCST OBS The forecast and observations appear to be matched as needed to compute the error statistics when combined with the same data from the other towers at the same time or with the same tower at different times. There are 7 matched pairs corresponding to each of the 7 observations coming from the tower (the “TOTAL” values in the matched pair data for each observation are all 1). I would appreciate your feedback on the approach I used to encode the tower data into the 11-col format. I would also appreciate the opportunity to work with you to develop a method to use METplus and Python embedding to automate the reformatting of the tower data into the 11-col format ready for ingest into ASCII2NC and then Point-Stat. I would appreciate any suggestions you have for possibly automating the process of averaging the tower data to produce 15 min averages which are valid at the top of the hour and at 15 min intervals (UTC) to coincide with the valid times of the forecasts. I can provide additional samples of the tower data and other data on request. I cannot upload the files in the forum or use FTP, so arrangements have been made with Julie Prestopnik to receive the files from an Army fileserver. She will distribute the files as needed within the DTC to support any potential collaboration. Distribution outside of DTC is not authorized without prior coordination with Mr. John Raby. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 8 replies
-
|
Hi @jwraby, And thank you for this wealth of information. I'm going to tag @georgemccabe who is the METplus Wrappers repository lead for a little direction on this. I know some work is currently being done to ingest WRF output natively, so a WRF-LES file might be a good candidate for that (unless you want the output post-processed by the UPP). What you've laid out here sounds like the start (and then some) of a great use case. It was unclear to me from your layout if you've utilized ASCII2NC's One thing to consider if this goes on to become a use case in the METplus repository is any data necessary to run the use case will need to be made available to the public. We will not publish anything before that time, but METplus use cases need all of the scripts and input data necessary to run them from a general user. |
Beta Was this translation helpful? Give feedback.
-
|
The changes I am implementing to support WRF output natively in MET involves reading NetCDF WRF output. It sounds like this data is post-processed by the UPP and can already be read into MET without issue. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @georgemccabe and @j-opatz. Thanks for reviewing the discussion and providing preliminary inputs/clarifications. Before I provide further input to pursue this as a potential use case, I need to get approvals for proceeding given that the information would need to be released to the public. |
Beta Was this translation helpful? Give feedback.
-
|
I have approval for release of examples of the tower observations and post-processed WRF output. I will provide the files to Julie Prestopnik. If it is decided to develop a use case from this project, I will need to get ARL approval prior to public release. Based on the tip provided by @j-opatz above, I was able to use the time_summary option of ASCII2NC to compute 15-min mean values from an input file containing simulated one-min tower TMP observations. Nice feature and potentially useful for this project. |
Beta Was this translation helpful? Give feedback.
-
|
HI @jwraby. I just wanted to follow up as I have yet not received a DoD SAFE link to pull data from. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for checking. Sorry about that. I was delayed and could not get to that point yesterday. I will try to upload the data send you the SAFE link today. |
Beta Was this translation helpful? Give feedback.
-
|
@jwraby There is no rush. In fact, I'm out of the office soon, so I won't be able to grab them until tomorrow. I just thought I'd check in since I was thinking about it. |
Beta Was this translation helpful? Give feedback.
-
|
No problem. I will try to get most if not all the data and READMEs which support the project uploaded to the SAFE today since I will be off tomorrow through next Mon. Hopefully you will see the SAFE notification in your email when you com in tomorrow. |
Beta Was this translation helpful? Give feedback.
-
|
@jprestop I just uploaded all the data in 3 separate uploads. The one I sent first contains two Word docs which will hopefully explain all the data and details about project. Download these files first as they should help with understanding all the data down to file descriptions. The one I sent after that contains a test case study and results with simulated tower observations in 11-col format, ASCII2NC output, post-processed WRF output, Point-Stat output and other information described in the Word doc "20220705 test case.docx". The last one I sent contains the raw tower observations files and the post-processed WRF output as described in the Word doc "Test case data for 20220706.docx" for use in development work aimed at any assistance you can provide with automating the averaging of the observations into 15-min averages and reformatting the observations into the 11-col format. Let me know if you have any questions or if you need anything else sent. Looking forward to working with you on this project! |
Beta Was this translation helpful? Give feedback.
-
|
@jwraby Thanks! I was able to receive all three zip files. @j-opatz and @georgemccabe, I put these files on seneca at "/d1/projects/METplus/discussions/2469" in subdirectories based on @jwraby's description. Please let me know if you have any questions or problems accessing these files. |
Beta Was this translation helpful? Give feedback.
-
|
We've opened a GitHub issue to track this work in #2487. Please continue any additional conversation there. Thank you! |
Beta Was this translation helpful? Give feedback.
We've opened a GitHub issue to track this work in #2487. Please continue any additional conversation there. Thank you!