Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 187/287 METAR config templates #218

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

spanNOAA
Copy link
Collaborator

@spanNOAA spanNOAA commented Nov 5, 2024

Finalize yamls for 187/287 metar observations with reasonable results.

See issue #46 for details.

Comment on lines +58 to +63
# Temporal thinning filter
- filter: Temporal Thinning
min_spacing: PT30M
seed_time: *analysisDate
category_variable:
name: MetaData/stationIdentification
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spanNOAA, are you finding that this is needed to match ob counts and increments? I found that for mesonet that this was actually detrimental for my results. Are you mainly using this to remove duplicate observations? If so, there could be other things we could try to handle that. I don't think the data is temporally thinned in GSI, so this "might" not be the best. Just something to consider.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found that removing duplicate METAR observations actually improved my results. In my case, eliminating them improved the analysis, as the duplicates may add additional value at the same location even if the information was outdated or from a significantly later time.

I believe the detrimental results when applying temporal thinning to mesonet observations may be due to the smaller observation window used. For mesonet data, a ±6 minute window is applied, while for METAR data, a larger ±15 minute window is used. This is because some METAR stations have less frequent observations. As a result, applying a smaller window may cause some stations to be dropped, while others might have multiple observations within the ±15 minute window.

Furthermore, GSI applies a form of "temporal thinning" when the l_closeobs option is enabled. This method retains only the observation closest to the analysis time and discards the others. Below is the relevant code:

if (l_closeobs) then
    if(abs(data(itime,k)-hr_offset)<abs(data(itime,l)-hr_offset)) then
        muse(l)=.false.
    else
        muse(k)=.false.
    endif
else
    tfact=min(one,abs(data(itime,k)-data(itime,l))/dfact1)
    dup(k)=dup(k)+one-tfact*tfact*(one-dfact)
    dup(l)=dup(l)+one-tfact*tfact*(one-dfact)
endif

Temporal thinning works well for temperature, humidity, and wind observations. However, I've had an issue with temporal thinning for surface pressure in my case. Specifically, surface observations in Canada were being dropped when temporal thinning was enabled. Theoretically, there should always be at least one observation at each station. I'm still investigating the cause of this issue.

30M spacing
image
0M spacing
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should try to remove the duplicated observations from surface stations. @delippi Could you provide more details on how this filter detriment your MESONET results?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are some runs I just made for mesonet using the temporal filter. The time window is +/-6 min. "No filter" gives better observation counts (counts are in the image; lower left corner), but increments look largely the same. The difference (between no filter and PT01M) is hard to spot but mainly just over west Texas. No temporal filter just seems better to me. PT30M definitely seems not good for this case. It could be argued that PT00M or PT01M would be fine.

No Temporal filter: (This gives different result from PT00M)
fv3jedi_vs_gsi_increment_airTemperature_msonet_airTemperature_188_None
PT01M:
fv3jedi_vs_gsi_increment_airTemperature_msonet_airTemperature_188_PT01M
PT30M:
fv3jedi_vs_gsi_increment_airTemperature_msonet_airTemperature_188_PT30M

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants