Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an archive task to GEFS system to archive files locally #2816

Merged
merged 93 commits into from
Sep 7, 2024
Merged

Add an archive task to GEFS system to archive files locally #2816

merged 93 commits into from
Sep 7, 2024

Conversation

AntonMFernando-NOAA
Copy link
Contributor

@AntonMFernando-NOAA AntonMFernando-NOAA commented Aug 7, 2024 •
edited
Loading

Description

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • forecast-only test on Hera/Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

DavidHuber-NOAA
DavidHuber-NOAA reviewed Aug 8, 2024
View reviewed changes
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this been tested on any platforms yet? I don't think that this will run until there is an archiving template in place. Marking as draft for now.

@DavidHuber-NOAA DavidHuber-NOAA marked this pull request as draft August 8, 2024 11:26
@AntonMFernando-NOAA
Copy link
Contributor Author

Has this been tested on any platforms yet? I don't think that this will run until there is an archiving template in place. Marking as draft for now.

@DavidHuber-NOAA Thanks. Yes. This is a draft.

@EricSinsky-NOAA
Copy link
Contributor

@AntonMFernando-NOAA I can help construct the archiving templates for GEFS.

@AntonMFernando-NOAA
Copy link
Contributor Author

@AntonMFernando-NOAA I can help construct the archiving templates for GEFS.

@EricSinsky-NOAA I will set up a meeting next week to further discuss this.

@AntonMFernando-NOAA
Copy link
Contributor Author

AntonMFernando-NOAA commented Aug 17, 2024 •
edited
Loading

@DavidHuber-NOAA @EricSinsky-NOAA I was able to archive a test file locally. CI test passed on Hercules.

@AntonMFernando-NOAA
Copy link
Contributor Author

@WalterKolczynski-NOAA @DavidHuber-NOAA Ready to go!

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Sep 6, 2024
DavidHuber-NOAA
DavidHuber-NOAA previously approved these changes Sep 6, 2024
View reviewed changes
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just one comment suggestion.

parm/archive/gefs_arcdir.yaml.j2 Outdated

{% set COMIN_ATMOS_ENSSTAT_1p00 = COM_ATMOS_GRIB_GRID_TMPL | replace_tmpl(tmpl_dict) %}

# test files
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# test files
# Select ensstat files to copy to the arcdir

@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Sep 6, 2024
@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Sep 6, 2024
@emcbot
Copy link

emcbot commented Sep 6, 2024

CI Passed on Hera in Build# 4
Built and ran in directory /scratch1/NCEPDEV/global/CI/2816


Experiment C48mx500_3DVarAOWCDA_692de265 Completed 2 Cycles: *SUCCESS* at Fri Sep  6 20:02:35 UTC 2024
Experiment C48_ATM_692de265 Completed 1 Cycles: *SUCCESS* at Fri Sep  6 20:14:44 UTC 2024
Experiment C96_atm3DVar_692de265 Completed 3 Cycles: *SUCCESS* at Fri Sep  6 21:40:25 UTC 2024
Experiment C96C48_hybatmDA_692de265 Completed 3 Cycles: *SUCCESS* at Fri Sep  6 21:40:31 UTC 2024
Experiment C48_S2SWA_gefs_692de265 Completed 1 Cycles: *SUCCESS* at Fri Sep  6 21:47:10 UTC 2024
Experiment C96C48_ufs_hybatmDA_692de265 Completed 2 Cycles: *SUCCESS* at Fri Sep  6 22:17:02 UTC 2024
Experiment C48_S2SW_692de265 Completed 1 Cycles: *SUCCESS* at Fri Sep  6 22:23:10 UTC 2024
Experiment C96C48_hybatmaerosnowDA_692de265 Completed 3 Cycles: *SUCCESS* at Fri Sep  6 22:35:22 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 2816c3b into NOAA-EMC:develop Sep 7, 2024
5 checks passed
@AntonMFernando-NOAA AntonMFernando-NOAA mentioned this pull request Sep 7, 2024
Open
7 tasks
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Sep 9, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into feature/ss_180
386a940 
* origin/develop:
  Create JEDI class (NOAA-EMC#2805)
  Restructure the bufr sounding job    (NOAA-EMC#2853)
  Add an archive task to GEFS system to archive files locally (NOAA-EMC#2816)
  Reenable Orion Cycling Support (NOAA-EMC#2877)
  Eliminate race conditions and remove DATAROOT last in cleanup (NOAA-EMC#2893)
  Update aerosol climatology to 2013-2024 mean (NOAA-EMC#2888)
  Add ability to run CI test C96_atm3DVar.yaml to Gaea-C5 (NOAA-EMC#2885)
  Support global-workflow GEFS C48 on Google Cloud (NOAA-EMC#2861)
  Add 3 and 9 hr increment files to IC staging (NOAA-EMC#2876)
  Add diffusion/diag B for aerosol DA and some other needed changes (NOAA-EMC#2738)
  Correct ocean `MOM.res_#` stage copy (NOAA-EMC#2868)
  Support coupling on AWS (NOAA-EMC#2859)
  Add JEDI ATM lgetkf observer and solver jobs (NOAA-EMC#2833)
  Fix gdas build on Gaea and add Gaea to available CI list (NOAA-EMC#2857)
  Support ATM forecast only on Google (NOAA-EMC#2832)
  Add GEFS C48 support on AWS (NOAA-EMC#2818)
  Update omega calculation (NOAA-EMC#2751)
  Add snow DA update and recentering for the EnKF forecasts (NOAA-EMC#2690)
  support ATM forecast only on Azure (NOAA-EMC#2827)
  Convert staging job to python and yaml (NOAA-EMC#2651)
  Fixed test on UNAVAILBLE in python Rocoto check (NOAA-EMC#2842)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DavidHuber-NOAA DavidHuber-NOAA DavidHuber-NOAA approved these changes

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA left review comments

@EricSinsky-NOAA EricSinsky-NOAA Awaiting requested review from EricSinsky-NOAA

@aerorahul aerorahul Awaiting requested review from aerorahul

Assignees
No one assigned
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Hercules-Passed **Bot use only** CI testing on Hercules for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add archive task to GEFS workflow
6 participants
@AntonMFernando-NOAA @EricSinsky-NOAA @WalterKolczynski-NOAA @DavidHuber-NOAA @emcbot @aerorahul