Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statics processing not correct #8

Open
ilaflott opened this issue Aug 7, 2024 · 2 comments
Open

statics processing not correct #8

ilaflott opened this issue Aug 7, 2024 · 2 comments

Comments

@ilaflott
Copy link
Member

ilaflott commented Aug 7, 2024

The requested static components in the README do not get processed correctly, example:
Screen Shot 2024-08-07 at 12 59 11 PM

no jobs should be running at this point, other than stage-history jobs, as those are what make the data available for later steps. Therefore, from the screenshot, what we can see is that e.g. for years 0001 and 0002, before the files are done being staged (stage-history:submitted), we have jobs attempting to run on those files (00020101T0000Z/remap-pp-components-ts-P2Y_land_static:running and 00020101T0000Z/remap-pp-components-av-P2Y_land_static:failed).
Additionally, given that static data is time-indep, no time-series no time-averages should be being created.

@ilaflott ilaflott mentioned this issue Aug 7, 2024
@ceblanton
Copy link
Contributor

Thanks Ian, I agree this is not right. How does FRE know that the land_static component contains only statics? It doesn't, you're finding.

There was a clunky way to do this using the remap-pp-component's rose-app.conf, but not yet in the pp.yaml. In the rose-app.conf settings, there was an optional freq setting, that could be used to only remap a subset of the source history files (e.g. monthly but not daily). I used that (freq=P0Y) to handle the static processing of those cases.

@ceblanton
Copy link
Contributor

One way we could solve this is using the diag_out.yaml "history manifest" file during the Install step.

If the history files mapped to the land_static component are all static, then tasks for timeseries and timeaverages wouldn't be created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants