Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for forecast-only runs on AWS #2711

Merged
merged 71 commits into from
Aug 13, 2024
Merged

Add support for forecast-only runs on AWS #2711

merged 71 commits into from
Aug 13, 2024

Conversation

weihuang-jedi
Copy link
Contributor

Description

The purpose of this PR it to merge code that allowing global-workflow run on AWS, first focus on ATM forecast only.

Resolves: #2709

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? YES

How has this been tested?

  • Clone and build on AWS
  • ATM Forecast-only on AWS

Checklist

  • [x ] Any dependent changes have been merged and published
  • [x ] My code follows the style guidelines of this project
  • [x ] I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • [ x] My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

@weihuang-jedi weihuang-jedi marked this pull request as ready for review June 23, 2024 18:47
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 24, 2024
View reviewed changes
ush/forecast_postdet.sh Fixed Show resolved Hide resolved
aerorahul
aerorahul requested changes Jun 24, 2024
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments regarding mpmd_opt and wall time for atmos_products.

env/AWSPW.env Outdated Show resolved Hide resolved
modulefiles/module_gwsetup.noaacloud.lua Show resolved Hide resolved
parm/config/gfs/config.resources Outdated Show resolved Hide resolved
sorc/link_workflow.sh Outdated Show resolved Hide resolved
versions/run.noaacloud.ver Outdated Show resolved Hide resolved
workflow/rocoto/tasks.py Outdated Show resolved Hide resolved
ush/forecast_postdet.sh Fixed Show resolved Hide resolved
workflow/rocoto/workflow_xml.py Outdated Show resolved Hide resolved
workflow/rocoto/workflow_xml.py Outdated Show resolved Hide resolved
workflow/rocoto/workflow_xml.py Outdated Show resolved Hide resolved
@DavidHuber-NOAA DavidHuber-NOAA mentioned this pull request Jun 25, 2024
Closed
@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title Make global-workflow run on AWS -- focus on ATM forecast only first Add support for forecast-only runs on AWS Jun 27, 2024
@weihuang-jedi
Copy link
Contributor Author

@aerorahul I have replied your comments, and make most of the changes you requested.
There are few left:

  1. remove NCL, since which cause compile error, and others have NCL as well, can we leave it as is?
  2. git version, that goes with spack-stack, can we just follow spack-stack?
  3. the "mpmd_opt" did not make negative impacts so far for forecast, do you want remove it from forecast and only keep in atmos-products?
  4. in tasks.py, and workflow_xml.py we use "pw_csp" local variable, some used twice and more. do you prefer not use local variable, but use "os.get_environ_get" directly.

Thanks,

Wei

@emcbot emcbot added CI-Wcoss2-Failed **Bot use only** CI testing on WCOSS for this PR has failed and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Aug 6, 2024
@emcbot
Copy link

emcbot commented Aug 6, 2024

Experiment C96_atm3DVar_extended_06093aff FAIL on Wcoss2 at 08/06/24 08:57:51 PM

Error logs:

/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f000.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f003.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f006.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f009.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f012.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f015.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f018.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f021.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f024.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f027.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f030.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f033.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f036.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f039.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f042.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f045.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f048.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f051.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f054.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f057.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakgrb2spec_f060.log
/lfs/h2/emc/global/noscrub/globalworkflow.ci/GFS_CI_ROOT/PR/2711/RUNTESTS/COMROOT/C96_atm3DVar_extended_06093aff/logs/2021122100/gfsgempakmeta.log

Follow link here to view the contents of the above file(s): (link)

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Wcoss2-Failed **Bot use only** CI testing on WCOSS for this PR has failed label Aug 8, 2024
@weihuang-jedi
Copy link
Contributor Author

@WalterKolczynski-NOAA Is there anything more that I need to do to get this PR merged?

@weihuang-jedi weihuang-jedi mentioned this pull request Aug 8, 2024
Merged
12 tasks
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS label Aug 12, 2024
@emcbot emcbot added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS and removed CI-Wcoss2-Ready **CM use only** PR is ready for CI testing on WCOSS labels Aug 12, 2024
@emcbot
Copy link

emcbot commented Aug 12, 2024 

CI Update on Wcoss2 at 08/12/24 05:30:58 PM
============================================
Cloning and Building global-workflow PR: 2711
with PID: 219881 on host: clogin03

@emcbot emcbot added CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Aug 12, 2024
@emcbot
Copy link

emcbot commented Aug 12, 2024

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Mon Aug 12 17:36:25 UTC 2024 on clogin03
---------------------------------------------------
Build: Completed at 08/12/24 06:40:40 PM
Case setup: Completed for experiment C48_ATM_bd1c9549
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_bd1c9549
Case setup: Skipped for experiment C48_S2SWA_gefs_bd1c9549
Case setup: Completed for experiment C48_S2SW_bd1c9549
Case setup: Completed for experiment C96_atm3DVar_extended_bd1c9549
Case setup: Skipped for experiment C96_atm3DVar_bd1c9549
Case setup: Completed for experiment C96_atmaerosnowDA_bd1c9549
Case setup: Completed for experiment C96C48_hybatmDA_bd1c9549
Case setup: Completed for experiment C96C48_ufs_hybatmDA_bd1c9549

@emcbot emcbot added CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Aug 13, 2024
@emcbot
Copy link

emcbot commented Aug 13, 2024

All CI Test Cases Passed on Wcoss2:

Experiment C48_ATM_bd1c9549 *** SUCCESS *** at 08/12/24 07:49:14 PM
Experiment C48_S2SW_bd1c9549 *** SUCCESS *** at 08/12/24 07:56:13 PM
Experiment C96C48_hybatmDA_bd1c9549 *** SUCCESS *** at 08/12/24 08:42:24 PM
Experiment C96_atmaerosnowDA_bd1c9549 *** SUCCESS *** at 08/12/24 09:35:24 PM
Experiment C96C48_ufs_hybatmDA_bd1c9549 *** SUCCESS *** at 08/12/24 09:56:17 PM
Experiment C96_atm3DVar_extended_bd1c9549 *** SUCCESS *** at 08/13/24 05:21:32 AM

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit eba813f into NOAA-EMC:develop Aug 13, 2024
5 checks passed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 13, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into feature/unavailabl…
7e7fbc6 
…e_rocoto

* origin/develop:
  Jenkins Pipeline Updates (NOAA-EMC#2815)
  Add Gaea C5 to CI (NOAA-EMC#2814)
  Add support for forecast-only runs on AWS (NOAA-EMC#2711)
  Add fixes to products for when REPLAY IC's are used  (NOAA-EMC#2755)
  Add capability to run forecast in segments (NOAA-EMC#2795)
This was referenced Aug 13, 2024
Closed
Merged
Merged
@weihuang-jedi weihuang-jedi deleted the aws-forecast-only branch August 22, 2024 13:35
WalterKolczynski-NOAA pushed a commit that referenced this pull request Aug 26, 2024
@weihuang-jedi
Add GEFS C48 support on AWS (#2818)
1231c9a 
Changes to make GEFS C48 case run on AWS.

After C48 ATM forecast only runs on AWs, the next step is to make GEFS
C48 run on AWS.
Changes to AWS env, and yaml files.

Resolves #2817
Refs #2711
WalterKolczynski-NOAA pushed a commit that referenced this pull request Aug 26, 2024
@weihuang-jedi
Support ATM forecast only on Google (#2832)
7a724e0 
Support global-workflow ATM forecast only runs on Google.

Add/Modify env, yaml, and python scripts changes to make global-workflow
ATM forecast only runs on GSP.

  Resolves #2831
  Refs #2826
  Refs #2711
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@aerorahul aerorahul aerorahul left review comments

@DavidHuber-NOAA DavidHuber-NOAA Awaiting requested review from DavidHuber-NOAA

Assignees
No one assigned
Labels
CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make global-workflow available on AWS -- focus on ATM forecast only first
5 participants
@weihuang-jedi @aerorahul @emcbot @WalterKolczynski-NOAA @DavidHuber-NOAA