Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix ocean file locations in archive job #1716

Merged
merged 2 commits into from
Jun 28, 2023
Merged

Fix ocean file locations in archive job #1716

merged 2 commits into from
Jun 28, 2023

Conversation

WalterKolczynski-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA commented Jun 28, 2023 •
edited
Loading

Description

During the COM refactor, a location for the ocn/ice daily files was defined within the products directory. However, the daily files are output directly by the model and land in the history directory. The archive job was looking in the otherwise unused daily product directory for the files and not finding them.

The 2D and 3D ocean files had the opposite problem: archive was looking in the history directory, but these files have their own directories in products.

This update now:

  • Removes the COM variable for the daily files
  • Updates the archive job to:
    • pull the daily ocean files from history
    • pull 2D ocean files from the 2D product directory
    • pull 3D ocean files from the 3D product directory

Fixes #1709

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Forecast-only S2S on Orion with local archiving

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes

@WalterKolczynski-NOAA
Fix daily file location in archive job
8e25dc3 
During the COM refactor, a location for the ocn/ice daily files was
defined within the products directory. However, the daily files are
output directly by the model and land in the history directory. The
archive job was looking in the otherwise unused daily product
directory for the files and not finding them. This update now:
- Removes the COM variable for the daily files
- Updates the archive job to pull the daily files from history

Fixes NOAA-EMC#1709
@WalterKolczynski-NOAA WalterKolczynski-NOAA self-assigned this Jun 28, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title Fix daily file location in archive job Fix ocean file locations in archive job Jun 28, 2023
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Jun 28, 2023
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Jun 28, 2023
@emcbot
Copy link

emcbot commented Jun 28, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Wed Jun 28 06:37:44 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jun 28 06:41:04 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jun 28 07:58:05 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:10 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:14 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:17 UTC 2023 for experiment C96_atm3DVar

@emcbot
Copy link

emcbot commented Jun 28, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Wed Jun 28 06:37:44 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jun 28 06:41:04 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jun 28 07:58:05 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:10 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:14 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:17 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Wed Jun 28 08:42:11 UTC 2023
with 18 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 28, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Wed Jun 28 06:37:44 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jun 28 06:41:04 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jun 28 07:58:05 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:10 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:14 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:17 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Wed Jun 28 08:42:11 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Wed Jun 28 10:21:15 UTC 2023
with 148 successfully completed jobs

@emcbot
Copy link

emcbot commented Jun 28, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Wed Jun 28 06:37:44 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jun 28 06:41:04 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jun 28 07:58:05 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:10 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:14 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:17 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Wed Jun 28 08:42:11 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Wed Jun 28 10:21:15 UTC 2023
with 148 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Wed Jun 28 11:03:11 UTC 2023
with 86 successfully completed jobs

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jun 28, 2023
@emcbot
Copy link

emcbot commented Jun 28, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Wed Jun 28 06:37:44 UTC 2023 on hfe05
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jun 28 06:41:04 UTC 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jun 28 07:58:05 UTC 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:10 UTC 2023 for experiment C48_S2S
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:14 UTC 2023 for experiment C96C48_hybatmDA
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jun 28 07:58:17 UTC 2023 for experiment C96_atm3DVar
Experiment C48_S2S completed: *SUCCESS*
Experiment C48_S2S Completed at Wed Jun 28 08:42:11 UTC 2023
with 18 successfully completed jobs
Experiment C96C48_hybatmDA completed: *SUCCESS*
Experiment C96C48_hybatmDA Completed at Wed Jun 28 10:21:15 UTC 2023
with 148 successfully completed jobs
Experiment C96_atm3DVar completed: *SUCCESS*
Experiment C96_atm3DVar Completed at Wed Jun 28 11:03:11 UTC 2023
with 86 successfully completed jobs

@JessicaMeixner-NOAA
Copy link
Contributor

Thanks @WalterKolczynski-NOAA !

aerorahul
aerorahul approved these changes Jun 28, 2023
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
@jiandewang please verify the contents of the hpss archive to ensure the required files are present.

@jiandewang
Copy link
Contributor

will make an end-to-end run to verify the fixing

@jiandewang
Copy link
Contributor

my end-to-end run finished, archiving step works as expected now.

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 279b38d into NOAA-EMC:develop Jun 28, 2023
8 checks passed
@WalterKolczynski-NOAA WalterKolczynski-NOAA deleted the hotfix/fix_daily branch June 28, 2023 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aerorahul aerorahul aerorahul approved these changes

@jiandewang jiandewang jiandewang approved these changes

Assignees

@WalterKolczynski-NOAA WalterKolczynski-NOAA

Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

wrong directory name for ocean in archiving step
5 participants
@WalterKolczynski-NOAA @emcbot @JessicaMeixner-NOAA @jiandewang @aerorahul