Add fv3-jedi and its dependencies (SkyLab 8 version) #395

rhoneyager-tomorrow · 2024-01-24T20:28:54Z

Description

The previous PR (#306) has been around for a while. I have updated it for the new SkyLab v7 release.

This provides the same as before, with the addition of mom6 and soca packages.

Errata

I've excluded updates/additions of mpas-model and mpas-jedi from this PR. The mpas package on Spack is entirely incompatible with the JCSDA fork. CMake packaging logic vs a custom Makefile. I tested production of a CMake-ified version mpas-model package, but there is a bug in the CMake package export logic in the JCSDA fork, and this prevents use with mpas-jedi.

Checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have run the unit tests before creating the PR

…kylab7_components

climbfuji · 2024-01-25T15:43:24Z

@rhoneyager-tomorrow This replaces #306, correct?

rhoneyager-tomorrow · 2024-01-25T15:44:09Z

Yes. I can close the old version.

climbfuji · 2024-03-11T16:21:07Z

@rhoneyager-tomorrow Let's see if we can get this PR merged/resolved. My preference would be to apply all the patches that you submitted here to the Skylab v8 release and then do away with anything prior to that in the spack PR. I would also prefer to pick apart this PR and add/update packages individually. For packages that are also in spack mainline, I would prefer submitting the updates to the upstream repo first and then bring them down to our fork.

Other notes:

Hopefully we don't need to maintain our GFDL_atmos_cubed_sphere fork for much longer and can just use the FV3 dycore code from GFDL directly (without the ecbuild stuff on top).
The spack-stack package "spack repo" that used to be var/spack/repos/jcsda_emc/...is now part of the spack-stack GitHub repository in subdirectoryspack-ext`

I can assist with or take care of some or most of this work, but hopefully we can split the work somewhat. Thoughts?

climbfuji · 2024-03-11T16:32:12Z

@rhoneyager-tomorrow I am going to submit this to femps (note that I removed the HINTS since the spack-package adds the external dependency on ecbuild:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6b2e041..9f570cb 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -4,6 +4,8 @@

 cmake_minimum_required( VERSION 3.3.2 FATAL_ERROR )

+find_package( ecbuild 3.6 REQUIRED )
+
 project( femps VERSION 1.2.0 LANGUAGES Fortran )

 set( CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake;${CMAKE_MODULE_PATH})
diff --git a/src/femps/CMakeLists.txt b/src/femps/CMakeLists.txt
index f7c2e65..3852a11 100644
--- a/src/femps/CMakeLists.txt
+++ b/src/femps/CMakeLists.txt
@@ -25,3 +25,11 @@ ecbuild_add_library( TARGET          femps

 target_include_directories( femps INTERFACE $<BUILD_INTERFACE:${CMAKE_Fortran_MODULE_DIRECTORY}>
                                             $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
+
+# Fortran module output directory for build and install interfaces
+set( MODULE_DIR module/${PROJECT_NAME}/${CMAKE_Fortran_COMPILER_ID}/${CMAKE_Fortran_COMPILER_VERSION} )
+set_target_properties( ${PROJECT_NAME} PROPERTIES Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR}/${MODULE_DIR} )
+install( DIRECTORY ${CMAKE_BINARY_DIR}/${MODULE_DIR}/ DESTINATION ${MODULE_DIR} )
+target_include_directories( femps INTERFACE
+                            $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/${MODULE_DIR}>
+                            $<INSTALL_INTERFACE:${MODULE_DIR}> )

Does that look ok for you?

climbfuji · 2024-03-11T16:48:57Z

Same for fv3-jedi-linearmodel:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index ffa8887..1d4f7f1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -15,6 +15,8 @@ set( CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake;${CMAKE_MODULE_PATH} )

 set( CMAKE_DIRECTORY_LABELS "fv3-jedi-lm" )

+find_package( ecbuild 3.6 REQUIRED )
+
 set( ECBUILD_DEFAULT_BUILD_TYPE Release )
 set( ENABLE_OS_TESTS           OFF CACHE BOOL "Disable OS tests" FORCE )
 set( ENABLE_LARGE_FILE_SUPPORT OFF CACHE BOOL "Disable testing of large file support" FORCE )
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 2d320ef..f4e4406 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -40,3 +40,11 @@ endif()
 if(CMAKE_Fortran_COMPILER_ID MATCHES GNU AND CMAKE_Fortran_COMPILER_VERSION VERSION_GREATER_EQUAL 10)
     target_compile_options(fv3jedilm PRIVATE $<$<COMPILE_LANGUAGE:Fortran>:-fallow-argument-mismatch>)
 endif()
+
+# Fortran module output directory for build and install interfaces
+set( MODULE_DIR module/${PROJECT_NAME}/${CMAKE_Fortran_COMPILER_ID}/${CMAKE_Fortran_COMPILER_VERSION} )
+set_target_properties( ${PROJECT_NAME} PROPERTIES Fortran_MODULE_DIRECTORY ${CMAKE_BINARY_DIR}/${MODULE_DIR} )
+install( DIRECTORY ${CMAKE_BINARY_DIR}/${MODULE_DIR}/ DESTINATION ${MODULE_DIR} )
+target_include_directories( fv3jedilm INTERFACE
+                            $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/${MODULE_DIR}>
+                            $<INSTALL_INTERFACE:${MODULE_DIR}> )

climbfuji · 2024-03-11T16:58:57Z

var/spack/repos/jcsda-emc/packages/fv3-jedi-linearmodel/package.py

+    variant("mpi", default=True, description="Support for MPI distributed parallelism")
+    variant("openmp", default=True, description="Build with OpenMP support")
+
+    conflicts("forecast_model=GEOS", msg="FV3-JEDI-LINEARMODEL: GEOS to be implemented.")


I think we are using the current fv3-jedi-linearmodel with GEOS and UFS - @shlyaeva can you comment on this?

You're using both, but at time of writing this PR GEOS wasn't fully buildable in Spack. Likewise, the ufs-bundle depended on JCSDA-internal, so I didn't have much insight there.

I think we can merge it as-is now, and update later as needed

climbfuji · 2024-03-11T17:00:10Z

var/spack/repos/jcsda-emc/packages/gsw/package.py

+    depends_on("ecbuild", type=("build"))
+    depends_on("[email protected]:", type=("build"), when="@3.0.7:")
+
+    version("3.0.7", commit="1a02ebaf6f7a4e9f2c2d2dd973fb050e697bcc74")


Note the non-standard notation of the tag. In the repo, it's v3.07

climbfuji · 2024-03-11T17:01:38Z

var/spack/repos/jcsda-emc/packages/saber/CMakeLists.txt.patch

+++ b/CMakeLists.txt
+@@ -42,7 +42,7 @@ find_package( atlas 0.33.0 REQUIRED COMPONENTS OMP_Fortran )
+ if( ENABLE_MKL )
+     find_package( MKL )


In fact, this should rather be

if( ENABLE_MKL ) find_package( MKL REQUIRED )

(at least that is what we decided at JCSDA - if you request a feature then it is an error if it is not available)

Actually, we already have the change that you made in develop, ~~and your version makes more sense than what I was thinking initially. That's because it's easier going to use Intel MKL or LAPACK, but if both are missing it will abort.~~ I am going to revert that comment. I will make it a hard requirement if someone asks for MKL.

My idea doesn't work, fixing the underlying problem requires changes across several repositories.

rhoneyager-tomorrow · 2024-03-11T17:18:22Z

@rhoneyager-tomorrow Let's see if we can get this PR merged/resolved. My preference would be to apply all the patches that you submitted here to the Skylab v8 release and then do away with anything prior to that in the spack PR. I would also prefer to pick apart this PR and add/update packages individually. For packages that are also in spack mainline, I would prefer submitting the updates to the upstream repo first and then bring them down to our fork.

SkyLab 8 isn't released yet, but I expect that it will take some time for all the patches to be applied, so I'm quite fine with everything that you propose.

Other notes:

Hopefully we don't need to maintain our GFDL_atmos_cubed_sphere fork for much longer and can just use the FV3 dycore code from GFDL directly (without the ecbuild stuff on top).

Great news! Hopefully you can do the same with mpas eventually.

The spack-stack package "spack repo" that used to be var/spack/repos/jcsda_emc/...is now part of the spack-stack GitHub repository in subdirectoryspack-ext`

I can assist with or take care of some or most of this work, but hopefully we can split the work somewhat. Thoughts?

Splitting works for me. Let me know once you finish reviewing.

climbfuji · 2024-03-11T19:38:24Z

I created the PRs for femps and fv3-jedi-linearmodel so far based on what I posted above

climbfuji · 2024-03-11T22:26:08Z

Regarding the ecmwf-atlas update: version 0.36.0 is already in spack develop, and I just opened spack#43133 to add the missing [email protected]: dependency. Once that's merged, I can bring both updates down to our fork.

climbfuji · 2024-03-12T16:56:55Z

@rhoneyager-tomorrow We merged the femps and fv3-jedi-linearmodel build system bug fixes. Do you want to check the public develop branches? That will be in the Skylab v8 releases.

climbfuji · 2024-03-12T18:58:31Z

var/spack/repos/builtin/packages/ecmwf-atlas/package.py

@@ -20,6 +20,7 @@ class EcmwfAtlas(CMakePackage):

    version("master", branch="master")
    version("develop", branch="develop")
+    version("0.36.0", sha256="39bf748aa7b22df80b9791fbb6b4351ed9a9f85587b58fc3225314278a2a68f8")


Changes to this file will no longer be needed. We merged the missing [email protected]: dependency upstream (spack#43133); spack develop already had 0.36.0. We'll bring both updates to our spack fork shortly.

climbfuji · 2024-03-12T19:03:20Z

var/spack/repos/jcsda-emc/packages/femps/package.py

+    depends_on("netcdf-c")
+    depends_on("netcdf-fortran")
+
+    # find_package(ecbuild REQUIRED) is needed when using ecbuild.


Patches should no longer be required if we target sklyab v8

climbfuji · 2024-03-12T19:04:18Z

var/spack/repos/jcsda-emc/packages/fv3-jedi-linearmodel/package.py

+        res = [self.define_from_variant("FV3_FORECAST_MODEL", "forecast_model")]
+        return res
+
+    # find_package(ecbuild REQUIRED) is needed when using ecbuild.


Patches should no longer be required if we target sklyab v8

climbfuji · 2024-03-13T02:03:28Z

var/spack/repos/jcsda-emc/packages/GFDL_atmos_cubed_sphere/CMakeLists.txt.patch

@@ -0,0 +1,11 @@
+--- a/CMakeLists.txt


No longer required for Skylab v8 because of JCSDA/GFDL_atmos_cubed_sphere#3

climbfuji · 2024-03-14T14:34:07Z

@rhoneyager-tomorrow Here is an update on the PR: I've merged the following patches (w/ slight modifications into the develop code, to be released in Skylab v8 next month):

var/spack/repos/jcsda-emc/packages/GFDL_atmos_cubed_sphere/CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/femps/CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/femps/src.femps.CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi-linearmodel/CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi-linearmodel/src.CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi/CMakeLists.txt.16.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi/CMakeLists.txt.18.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi/cmake.fv3jedi_extra_macros.cmake.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi/test.CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/saber/CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/saber/quench.src.Fields.cc.patch
var/spack/repos/jcsda-emc/packages/saber/saber-import.cmake.in.patch

I also updated the following package recipes in our spack fork:

var/spack/repos/builtin/packages/ecmwf-atlas/package.py (and synced it with spack develop)

What needs to remain in this PR (but updated to only cover the Skylab v8 versions):

var/spack/repos/builtin/packages/crtm/package.py
var/spack/repos/jcsda-emc/packages/GFDL_atmos_cubed_sphere/package.py
var/spack/repos/jcsda-emc/packages/femps/package.py
var/spack/repos/jcsda-emc/packages/fv3-jedi-linearmodel/CMakeLists.txt.patch
var/spack/repos/jcsda-emc/packages/fv3-jedi/package.py
var/spack/repos/jcsda-emc/packages/gsw/package.py (but note that different tag name)
var/spack/repos/jcsda-emc/packages/ioda/package.py
var/spack/repos/jcsda-emc/packages/mom6/package.py
var/spack/repos/jcsda-emc/packages/oops/package.py
var/spack/repos/jcsda-emc/packages/saber/package.py
var/spack/repos/jcsda-emc/packages/soca/package.py
var/spack/repos/jcsda-emc/packages/ufo/package.py
var/spack/repos/jcsda-emc/packages/vader/package.py

rhoneyager-tomorrow and others added 10 commits August 21, 2023 10:51

Add fv3-jedi for Skylab v5.

1939921

Merge branch 'jcsda_emc_spack_stack' into feature/fv3_jedi

6923f36

Updates for spack-stack 1.5 and the SkyLab 6 release.

401763e

black formatting

fa07cbe

Merge branch 'jcsda_emc_spack_stack' into feature/fv3_jedi

30dfeef

flake8 checks

561f9e1

Merge remote-tracking branch 'origin/feature/fv3_jedi' into feature/s…

bbb04d9

…kylab7_components

Updating through fv3-jedi to SkyLab v7

93a01d9

Adding soca to spack build.

c812238

black formatting

a965a72

climbfuji self-requested a review January 25, 2024 15:43

climbfuji self-assigned this Jan 25, 2024

climbfuji added the INFRA JEDI Infrastructure label Jan 25, 2024

climbfuji reviewed Mar 11, 2024

View reviewed changes

climbfuji reviewed Mar 12, 2024

View reviewed changes

This was referenced Mar 12, 2024

[email protected]: depend on [email protected]: (spack PR #43133) #413

Merged

Build system update: add missing ecbuild dependency JCSDA/GFDL_atmos_cubed_sphere#3

Merged

climbfuji reviewed Mar 13, 2024

View reviewed changes

climbfuji changed the title ~~Add fv3-jedi and its dependencies (SkyLab 7 version)~~ Add fv3-jedi and its dependencies (SkyLab 8 version) Apr 12, 2024

climbfuji assigned eap and srherbener Aug 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fv3-jedi and its dependencies (SkyLab 8 version) #395

Add fv3-jedi and its dependencies (SkyLab 8 version) #395

rhoneyager-tomorrow commented Jan 24, 2024

climbfuji commented Jan 25, 2024

rhoneyager-tomorrow commented Jan 25, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji Mar 11, 2024

rhoneyager-tomorrow Mar 11, 2024 •

edited

Loading

climbfuji Mar 12, 2024

climbfuji Mar 11, 2024

climbfuji Mar 11, 2024

climbfuji Mar 12, 2024 •

edited

Loading

climbfuji Mar 12, 2024

rhoneyager-tomorrow commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 12, 2024

climbfuji Mar 12, 2024

climbfuji Mar 12, 2024

climbfuji Mar 12, 2024

climbfuji Mar 13, 2024

climbfuji commented Mar 14, 2024 •

edited

Loading

Add fv3-jedi and its dependencies (SkyLab 8 version) #395

Are you sure you want to change the base?

Add fv3-jedi and its dependencies (SkyLab 8 version) #395

Conversation

rhoneyager-tomorrow commented Jan 24, 2024

Description

Errata

Checklist

climbfuji commented Jan 25, 2024

rhoneyager-tomorrow commented Jan 25, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

Choose a reason for hiding this comment

rhoneyager-tomorrow Mar 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

climbfuji Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhoneyager-tomorrow commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 11, 2024

climbfuji commented Mar 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

climbfuji commented Mar 14, 2024 • edited Loading

rhoneyager-tomorrow Mar 11, 2024 •

edited

Loading

climbfuji Mar 12, 2024 •

edited

Loading

climbfuji commented Mar 14, 2024 •

edited

Loading