Move to contrib spack-stack on Jet #978

InnocentSouopgui-NOAA · 2024-08-28T18:52:04Z

DESCRIPTION OF CHANGES:

Problem: Following the failure of the storage lfs4 on Jet, spack stack moved to another storage space: Contrib. This makes all softwares relying on spack stack libraries on Jet to fail.

Solution: Update jet modulefiles to point to the new installation of spack stack.

TESTS CONDUCTED:

The changes affects only Jet as all modifications are localized in Jet modulefile.

Compile branch Jet using Intel
Extract Initial conditions for Global Workflow cycled experiment on Jet.
Global Workflow at resolution
- C96/48
- C192/96
- C384/192
Run relevant consistency tests locally on Jet.

Optional test.
Cycle Global workflow at resolutions 96/48, 192/96 and 384/192

DEPENDENCIES:

None

DOCUMENTATION:

Not needed.

ISSUE:

If this PR is resolving or referencing one or more issues, in this repository or elsewhere, list them here. For example, "Fixes issue mentioned in #123" or "Related to bug in https://github.com/NOAA-EMC/other_repository/pull/63"

Fixes issue mentioned in #977
Refs NOAA-EMC/global-workflow#2841

InnocentSouopgui-NOAA · 2024-09-06T00:14:07Z

@GeorgeGayno-NOAA Can you review this so that we move it forward? The PR on global workflow depends on this.

GeorgeGayno-NOAA · 2024-09-06T12:29:48Z

I think your branch in behind the 'develop' branch. Please merge in the latest updates.

InnocentSouopgui-NOAA · 2024-09-06T13:13:11Z

I think your branch in behind the 'develop' branch. Please merge in the latest updates.

Done

GeorgeGayno-NOAA · 2024-09-06T14:15:25Z

To get the ice_blend consistency test to run, I made these changes to the driver script (load the grib-util module, then remove the definitions of COPYGB, COPYGB2, and CNVGRIB).

diff --git a/reg_tests/ice_blend/driver.jet.sh b/reg_tests/ice_blend/driver.jet.sh
index 721b6bb6..2746d759 100755
--- a/reg_tests/ice_blend/driver.jet.sh
+++ b/reg_tests/ice_blend/driver.jet.sh
@@ -32,6 +32,7 @@ source ../../sorc/machine-setup.sh > /dev/null 2>&1
 module use ../../modulefiles
 module load build.$target.intel
 module load wgrib2/2.0.8
+module load grib-util/1.3.0
 set +x
 module list
 set -x
@@ -51,9 +52,6 @@ if [ "$UPDATE_BASELINE" = "TRUE" ]; then
 fi

 export WGRIB=/apps/wgrib/1.8.1.0b/bin/wgrib
-export COPYGB=/lfs5/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/copygb
-export COPYGB2=/lfs5/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/copygb2
-export CNVGRIB=/mnt/lfs5/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/cnvgrib

InnocentSouopgui-NOAA · 2024-09-06T14:47:24Z

@GeorgeGayno-NOAA thanks for catching that.
It should be /contrib/spack-stack/spack-stack-1.6.0 for the path to spack-stack. I completely missed that because I did not run it.

While we are at it, what are all the consistencies tests to run?
I caught everything that is used by global workflow and the making of ICS for global workflow.

Are there regression tests that can catch problems in most components of UFS_UTILS?

I need to do another round of checks/modification.

GeorgeGayno-NOAA · 2024-09-06T15:09:04Z

@GeorgeGayno-NOAA thanks for catching that. It should be /contrib/spack-stack/spack-stack-1.6.0 for the path to spack-stack. I completely missed that because I did not run it.

While we are at it, what are all the consistencies tests to run? I caught everything that is used by global workflow and the makin of ICS for global workflow.

Are there regression tests that can catch problems in most components of UFS_UTILS?

I need to do another round of checks/modification.

The consistency/regression tests are in ./reg_tests. Most tests have a 'jet' driver script. Other tests have a single script that works for all machines. For instructions on how to run, see each scripts prolog. I am running the tests now. I recommend you run them as well. Let me know if you have questions.

…OAA/UFS_UTILS into migration-jet-contrib

InnocentSouopgui-NOAA · 2024-09-09T15:10:45Z

@GeorgeGayno-NOAA , when you get the chance, have another look at the changes.

GeorgeGayno-NOAA · 2024-09-06T12:38:43Z

fix/link_fixdirs.sh

@@ -47,7 +47,7 @@ pwd=$(pwd -P)
 if [ $machine = "hera" ]; then
    FIX_DIR="/scratch1/NCEPDEV/global/glopara/fix"
 elif [ $machine = "jet" ]; then


The Jet directory does not exist.

GeorgeGayno-NOAA · 2024-09-06T12:50:17Z

util/gdas_init/set_fixed_files.sh

@@ -36,7 +36,7 @@ if [ "$machine" = 'hera' ] ; then
 elif [ "$machine" = 'wcoss2' ] ; then
  FIX_ORO_INPUT=/lfs/h2/emc/global/noscrub/emc.global/FIX/fix/orog/20230615
 elif [ "$machine" = 'jet' ] ; then
-  FIX_ORO_INPUT=/lfs4/HFIP/hfv3gfs/glopara/git/fv3gfs/fix/orog/20230615
+  FIX_ORO_INPUT=/lfs5/HFIP/hfv3gfs/glopara/git/fv3gfs/fix/orog/20230615


Same issue here. This directory does not exist.

This should be /lfs5/HFIP/hfv3gfs/glopara/FIX/fix/orog/20230615.

GeorgeGayno-NOAA · 2024-09-06T13:22:13Z

reg_tests/chgres_cube/driver.jet.sh

@@ -54,7 +54,7 @@ fi

 export HOMEufs=$PWD/../..

-export HOMEreg=/lfs4/HFIP/hfv3gfs/emc.nemspara/role.ufsutils/ufs_utils/reg_tests/chgres_cube
+export HOMEreg=/lfs5/HFIP/hfv3gfs/emc.nemspara/role.ufsutils/ufs_utils/reg_tests/chgres_cube


This directory does not exist yet (I am creating them now). Did you run the consistency tests?

GeorgeGayno-NOAA · 2024-09-06T15:04:08Z

reg_tests/cpld_gridgen/rt.sh

-    export MOM6_FIXDIR=/lfs4/HFIP/hfv3gfs/glopara/git/fv3gfs/fix/mom6/20220805
-    BASELINE_ROOT=/lfs4/HFIP/hfv3gfs/emc.nemspara/role.ufsutils/ufs_utils/reg_tests/cpld_gridgen/baseline_data
+    STMP=${STMP:-/lfs5/HFIP/h-nems/$USER}
+    export MOM6_FIXDIR=/lfs5/HFIP/hfv3gfs/glopara/git/fv3gfs/fix/mom6/20220805


MOM6_FIXDIR should be /lfs5/HFIP/hfv3gfs/glopara/FIX/fix/mom6/20220805

GeorgeGayno-NOAA · 2024-09-09T12:52:05Z

reg_tests/ice_blend/driver.jet.sh

-export COPYGB=/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/copygb
-export COPYGB2=/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/copygb2
-export CNVGRIB=/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-hrqavdi/bin/cnvgrib
+export COPYGB=/contrib/spack-stack/spack-stack-1.6.0/envs/unified-env-rocky8/install/intel/2021.5.0/grib-util-1.3.0-74mdurc/bin/copygb


You can remove the export of COPYGB, COPYGB2 and CNVGRIB if you add module load grib-util/1.3.0 near the top of the script.

GeorgeGayno-NOAA · 2024-09-09T13:01:56Z

fix/link_fixdirs.sh

@@ -47,7 +47,7 @@ pwd=$(pwd -P)
 if [ $machine = "hera" ]; then
    FIX_DIR="/scratch1/NCEPDEV/global/glopara/fix"
 elif [ $machine = "jet" ]; then
-    FIX_DIR="/lfs4/HFIP/hfv3gfs/glopara/git/fv3gfs/fix"
+    FIX_DIR="/lfs5/HFIP/hfv3gfs/glopara/git_lfs4/fv3gfs/fix"


@KateFriedman-NOAA - I see multiple copies of the 'fixed' data on Jet. Is this the correct directory?

No, that is a copy from /lfs4. The official fix path on Jet is now: /lfs5/HFIP/hfv3gfs/glopara/FIX/fix

Move to contrib spack-stack on Jet

646a973

InnocentSouopgui-NOAA mentioned this pull request Aug 29, 2024

Move to contrib installation of spack-stack on Jet NOAA-EMC/global-workflow#2878

Merged

23 tasks

Adjust paths to use /lfs5 instead on /lfs4

d3da723

GeorgeGayno-NOAA self-requested a review August 30, 2024 16:37

InnocentSouopgui-NOAA marked this pull request as ready for review September 3, 2024 18:43

Merge branch 'ufs-community:develop' into migration-jet-contrib

d8e9815

InnocentSouopgui-NOAA added 7 commits September 6, 2024 16:06

Add path to new install of spack stack on Jet

bc9a1e0

Update the jet regression test log for cpld_gridgen

40497d0

Update the jet regression test log for ocnice_prep

cbf40b6

Update the jet location for test data for cpld_gridgen

f095b10

Update the jet location for local data

872651c

Merge branch 'migration-jet-contrib' of github.com:InnocentSouopgui-N…

bd8a6b1

…OAA/UFS_UTILS into migration-jet-contrib

set local path for Jet

5a84549

InnocentSouopgui-NOAA mentioned this pull request Sep 9, 2024

Spack-stack moved from /lfs4 to /contrib on Jet #977

Closed

GeorgeGayno-NOAA approved these changes Sep 9, 2024

View reviewed changes

GeorgeGayno-NOAA merged commit 06eec5b into ufs-community:develop Sep 9, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move to contrib spack-stack on Jet #978

Move to contrib spack-stack on Jet #978

InnocentSouopgui-NOAA commented Aug 28, 2024 •

edited

Loading

InnocentSouopgui-NOAA commented Sep 6, 2024

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 6, 2024

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 6, 2024 •

edited

Loading

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 9, 2024

GeorgeGayno-NOAA Sep 6, 2024

GeorgeGayno-NOAA Sep 6, 2024

KateFriedman-NOAA Sep 9, 2024

GeorgeGayno-NOAA Sep 6, 2024

GeorgeGayno-NOAA Sep 6, 2024

GeorgeGayno-NOAA Sep 9, 2024

GeorgeGayno-NOAA Sep 9, 2024

KateFriedman-NOAA Sep 9, 2024

Move to contrib spack-stack on Jet #978

Move to contrib spack-stack on Jet #978

Conversation

InnocentSouopgui-NOAA commented Aug 28, 2024 • edited Loading

DESCRIPTION OF CHANGES:

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

InnocentSouopgui-NOAA commented Sep 6, 2024

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 6, 2024

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 6, 2024 • edited Loading

GeorgeGayno-NOAA commented Sep 6, 2024

InnocentSouopgui-NOAA commented Sep 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

InnocentSouopgui-NOAA commented Aug 28, 2024 •

edited

Loading

InnocentSouopgui-NOAA commented Sep 6, 2024 •

edited

Loading