Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mixing temporal resolutions when merging files with combine_first #297

Open
BaptisteVandecrux opened this issue Aug 27, 2024 · 1 comment

Comments

@BaptisteVandecrux
Copy link
Member

If an hourly STM table covering 2022-2024 is retrieved from the field and is added after a raw 10 min file covering 2022-2023, then the merging of L1 datasets into the L1A dataset will have a strange behavior:

self.L1A = reduce(xr.Dataset.combine_first, reversed(self.L1))

The latest hourly data ds_latest_hourly = AWS.L1[-1] will be merged with the previous raw dataset ds_previous_10min = AWS.L1[-2] with :
ds_merged = ds_latest_hourly.combine_first(ds_previous_10min )

In that statement, the round-hours lines from the hourly file will be unchanged, but 10 min data will be inserted between these values.

@BaptisteVandecrux
Copy link
Member Author

Due to this this, winter daily tx have been injected at KAN_U's hourly data:
billede

Again, it all comes from the fact that combine_first only looks at the order at which the files come and is not setting different priorities to the different data types (10 min > STM > tx).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant