pb2nc bufr file with varying reference times #2642
-
|
I was running a bufr file through pb2nc when I got these WARNING messages
This was brought up in the METplus Users Telecon on 7/22/2024. It was mentioned that it may be due to the reference time information in the bufr file changing. I put the file at in /incoming/irap/met_help/row_data. |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments
-
|
@malloryprow thanks for providing the sample file. I ran it through pb2nc and was able to reproduce the warning messages you describe: Let me explain why we originally classified this situation as a warning. In the original PrepBUFR files we used during development, the PrepBUFR reference time remained constant for all messages in the file. Because of this, we used the same configuration option to define the matching time window in both PB2NC and Point-Stat: In Point-Stat, the matching time window is defined as seconds of offset relative to the forecast valid time, and in PB2NC it is defined relative to the PrepBUFR center time. If that center time changes from message to message, suddenly it's no longer a useful way to define the matching time window since it just uses the FIRST center time it encounters. FYI, you can see the matching time window printed in the log output: Fortunately, PB2NC already supports the I propose that we add the following logic to address this situation...
You would do this by setting the PB2NC_VALID_BEG and PB2NC_VALID_END METplus config options. Does that update to the logic make sense to you and @BinbinZhou-NOAA, who was also seeing similar warning messages? |
Beta Was this translation helpful? Give feedback.
-
|
That makes sense! Thanks @JohnHalleyGotway! |
Beta Was this translation helpful? Give feedback.
-
|
Mallory,
My sample file is:
/lfs/h2/emc/vpppg/noscrub/binbin.zhou/split_by_subset/rap.t02z.prepbufr.tm00.split
The original entire prepbufr file is
/lfs/h2/emc/vpppg/noscrub/binbin.zhou/split_by_subset/rap.t02z.prepbufr.tm00
PB2NC has no WARNING ot process the second one
Could you please send these 2 files to DTC?
Thanks!
Binbin
…On Mon, Jul 22, 2024 at 1:37 PM John Halley Gotway ***@***.***> wrote:
@malloryprow <https://github.com/malloryprow> thanks for providing the
sample file. I ran it through pb2nc and was able to reproduce the warning
messages you describe:
WARNING: process_pbfile() -> The observation time should remain the same for all Bufr messages
WARNING: process_pbfile() -> 80151 messages with different reference time (20240102_000000):
WARNING: process_pbfile() -> - 20240102_010000
WARNING: process_pbfile() -> - 20240102_030000
... and so on
Let me explain why we originally classified this situation as a warning.
In the original PrepBUFR files we used during development, the *PrepBUFR
reference time* remained constant for all messages in the file. Because
of this, we used the same configuration option to define the matching time
window in both PB2NC and Point-Stat:
obs_window = {
beg = -5400;
end = 5400;
}
In Point-Stat, the matching time window is defined as seconds of offset
relative to the *forecast valid time*, and in PB2NC it is defined
relative to the *PrepBUFR center time*. If that center time changes from
message to message, suddenly that it's a very useful way to define a
matching time window.
Fortunately, PB2NC already supports the -valid_beg and -valid_end command
line options to manually define the matching time window. This overrides
the obs_window setting.
I propose the following logic to address this situation...
- PB2NC should still check to see if the PrepBUFR message center time
is changing.
- If it does not change, DO NOT PRINT a warning message.
- If it does change, but the -valid_beg and -valid_end command line
options have been used to manually define the time window, DO NOT PRINT a
warning message.
- If it does change, but obs_window was used, DO PRINT a warning
message instructing the user to specify the matching time window using the
-valid_beg and -valid_end command line options.
You would do this by setting the PB2NC_VALID_BEG
<https://metplus.readthedocs.io/en/develop/Users_Guide/glossary.html#term-PB2NC_VALID_BEGIN>
and PB2NC_VALID_END
<https://metplus.readthedocs.io/en/develop/Users_Guide/glossary.html#term-PB2NC_VALID_END>
METplus config options.
Does that update to the logic make sense to you and @BinbinZhou-NOAA
<https://github.com/BinbinZhou-NOAA>, who was also seeing similar warning
messages?
—
Reply to this email directly, view it on GitHub
<#2642 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQBMPFPSRG4W64N6FQN5TVTZNU7MRAVCNFSM6AAAAABLIWRS5CVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMJRG42TAOA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Binbin Zhou
Physical Scientist
Lynker at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
***@***.***
301-683-3683
|
Beta Was this translation helpful? Give feedback.
-
|
@JohnHalleyGotway I put the files Binbin mentioned in /incoming/irap/met_help/row_data. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the feedback and sample data @malloryprow and @BinbinZhou-NOAA. I used this info to write up a new dtcenter/MET#2938 issue to change the logic as described above. I'll go ahead and mark this discussion as being answered. Please feel free to add comments to dtcenter/MET#2938 for any further discussion on this topic. |
Beta Was this translation helpful? Give feedback.
@malloryprow thanks for providing the sample file. I ran it through pb2nc and was able to reproduce the warning messages you describe:
Let me explain why we originally classified this situation as a warning. In the original PrepBUFR files we used during development, the PrepBUFR reference time remained constant for all messages in the file. Because of this, we used the same configuration opti…