Add sub-regioan diag output example in 1D case #39

yichengt900 · 2024-05-14T23:44:19Z

This PR inclues the following changes:

enhances the build script to include the libyaml build process;
added an example diag_table YAML file for sub-region output in the 1D case;
added an example xml about how to use new diag manager as well as how to convert legacy diag_table to diag_table.yaml;
added short paragraph for the new yaml style of diag_table in our online documentation.

Keep in mind the old diag_manager code has been kept intact and will be used by default, so these updates should not have impact to your current experiments. If you want to try the new indices subsetting (comes with the new re-written diag manager), you will have to first build the model system with FMS2 as well as -Duse_yaml flag, and set use_modern_diag = .true. in your diag_manager_nml. You also have to prepare a yaml format of diag_table.

The sub-region diagnostic works ok with indices for both NEP and Arctic domains (see below example plot from Arctic domain). The NWA domain works with either latlon or indices subsetting.

Also, if you prefer FMS1 and still want to use index subsetting, you can try turning on USE_INDEX_DIAGNOSTIC_AXES = True in MOM6. This should allow you to specify the index instead of latlon for sub-region diagnostic output.

* Clean up of the light and phytoplankton growth calculations * update baseline answers --------- Co-authored-by: Charles Stock <[email protected]> Co-authored-by: yuchengt900 <[email protected]>

theresa-morrison · 2024-05-22T19:48:52Z

This will be relevant for the NEP and Arctic domains
@wchengpmel @kshedstrom

yichengt900 · 2024-05-29T18:48:14Z

Following our discussion this morning, I'd appreciate it if someone could review this PR when there's a chance.

kshedstrom · 2024-05-29T18:58:32Z

My understanding is that the latest FMS is not compatible with the gnu compiler and I therefore can't test this at home. I hope to get to this on gaea next week.

yichengt900 · 2024-05-29T19:07:47Z

@kshedstrom, Yes, FMS2 does not work with the latest GCC. It only works with previous versions of GCC.

yichengt900 · 2024-06-14T20:13:08Z

@uwagura, it would be nice if you could review this PR when you get chance, especially the new XML file, which is related to the topic of YAML-based diagnostics that @charliestock mentioned. Any comments are more than welcome!

kshedstrom · 2024-06-18T00:01:35Z

Following the NWA xml example, I can compile with yaml and get it to run. However, trying for FMS 2024.01.02 causes it to look for that version of ice_param here:

<codeBase version="$(CPL_GIT_TAG)"> ice_param.git </codeBase>

and it doesn't find it. I made a separate ICE_PARAM_TAG for it.

Then I got bold and asked for my favorite vertical sections and it failed on me:

FATAL from PE  1037: axis_utils2::nearest_index array is NOT monotonously ordered

The mess is here: /gpfs/f5/cpo_rmom6/scratch/Katherine.Hedstrom/work/run75_1x1m0d_1648x1o.o135017531

kshedstrom · 2024-06-18T00:35:11Z

My bad - I needed to use USE_INDEX_DIAGNOSTIC_AXES = True. Why did you say it's for FMS1?

yichengt900 · 2024-06-18T00:54:38Z

@kshedstrom, thanks for your testing. I fixed the ice_param tag issue in the XML.

The USE_INDEX_DIAGNOSTIC_AXES = True is a trick from MOM6, not FMS. The index subsetting comes naturally with the FMS2 newly rewritten diag-manager and should not need to set USE_INDEX_DIAGNOSTIC_AXES = True.

I took a quick look at your config and noticed that the sub_region grid type in your diag_table.yaml is still latlon, not index. My first guess is this might be an issue from the diag-table-to-yaml tool. Since the legacy diag_table does not indicate whether the sub-region is in latlon or index, the tool probably has a problem identifying that at this moment. @thomas-robinson, can you confirm this is the case?

For Artic we can either set USE_INDEX_DIAGNOSTIC_AXES = True or add an extra step in xml to replace latlon with index in the diag_table.yaml.

thomas-robinson · 2024-06-18T11:27:17Z

Tagging @uramirez8707 here just in case he is needed. The diag table to diag yaml tool translates the diag table, and the diag table only supports latlon. Yes you're correct @yichengt900 . If your diag_table file has a subregion with local indicies and not latlon, then then output produced is probably not what you are expecting,

uwagura

Sucessfully completed regression test for layout

kshedstrom · 2024-06-28T00:57:54Z

I recompiled outside of my conda environment and things went rather better (FMS 2024.01.02). Everything is fine as long as I use the old style diag format. Without my small cross-sections, the new format still fails:

FATAL from PE    43: freq units is required. Check your entry for file:ocean_hourly

#0  0x31eb527 in __mpp_mod_MOD_mpp_error_basic
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/mpp/include/mpp_util_mpi.inc:74
#1  0x3b09506 in parse_key
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/diag_manager/fms_diag_yaml.F90:801
#2  0x3b0c140 in fill_in_diag_files
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/diag_manager/fms_diag_yaml.F90:551
#3  0x3b119eb in __fms_diag_yaml_mod_MOD_diag_yaml_object_init
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/diag_manager/fms_diag_yaml.F90:488
#4  0x3835798 in __fms_diag_object_mod_MOD_fms_diag_object_init
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/diag_manager/fms_diag_object.F90:130
#5  0x345fb1b in __diag_manager_mod_MOD_diag_manager_init
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/FMS2/diag_manager/diag_manager.F90:4213
#6  0xfec2ac in coupler_init
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/coupler/full/coupler_main.F90:1566
#7  0xfdb646 in coupler_main
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/coupler/full/coupler_main.F90:590
#8  0xff1607 in main
        at //import/c1/AKWATERS/kate/ESMG/ESMG-configs/src/coupler/full/coupler_main.F90:335

uramirez8707 · 2024-06-28T15:36:17Z

@kshedstrom Can you point me to your diag_table.yaml?

kshedstrom · 2024-06-28T17:57:05Z

I just copied it to /ncrc/home2/Katherine.Hedstrom/ESMG/ESMG-configs/Arctic6/mid_12
on gaea.

uramirez8707 · 2024-07-02T17:48:11Z

/ncrc/home2/Katherine.Hedstrom/ESMG/ESMG-configs/Arctic6/mid_12

@kshedstrom It looks like I don't have permissions to view the file

-rw------- 1 Katherine.Hedstrom esrl 10653 Jun 28 13:56 /ncrc/home2/Katherine.Hedstrom/ESMG/ESMG-configs/Arctic6/mid_12/diag_table.yaml

kshedstrom · 2024-07-02T18:40:54Z

Oops, sorry, fixed.

uramirez8707 · 2024-07-02T20:20:02Z

It looks like your table was generated using an older version of the converter. Before this update which combined the freq and freq_units keys: NOAA-GFDL/fms_yaml_tools#26

On gaea, you can do

module load fre/bronx-21
diag-table-to-yaml -f /ncrc/home2/Katherine.Hedstrom/ESMG/ESMG-configs/Arctic6/mid_12/diag_table -o diag_table.yaml

on your system, please updated your install of fms_yaml_tools.

kshedstrom · 2024-07-03T23:19:39Z

Yes, that works better. Thanks!
The subregion output works for the Bering Sea domain in latlon mode.
I do not have the subregion output working for the Arctic domain in any mode. It seems to accept it in index mode, but doesn't output anything. I assume the tile numbering starts at 0? The attempt in /autofs/ncrc-svm1_home2/Katherine.Hedstrom/ESMG/ESMG-configs/Arctic6/mid_12/diag_table.yaml
is for 24 cores.

yichengt900 and others added 8 commits May 14, 2024 11:32

add src/libyaml

5d95fa4

update build script for yaml support

49de077

fix build script for libyaml link

df00d93

update 1d case with regional diag output example

95c2dec

update build script

2c6f777

debugging

6818ca0

forget add diag_table.yaml

289e705

Update docker scirpt to include FRE NC Tool; Update docs

7c6a569

yichengt900 added the CEFI_MOM6_RT_container label May 15, 2024

github-actions bot added pass_CEFI_MOM6_RT and removed CEFI_MOM6_RT_container labels May 15, 2024

yichengt900 marked this pull request as ready for review May 15, 2024 17:56

yichengt900 self-assigned this May 15, 2024

yichengt900 added 4 commits May 16, 2024 09:37

Update MOM6 tag

2fa51ac

Add example xml for new diag manager

96ab007

Merge branch 'main' into fix/fms2_nudge

7dc39c4

remove unused GHA workflow

7b8e606

yichengt900 requested review from andrew-c-ross, gabyneg, charliestock, theresa-morrison and uwagura May 16, 2024 18:20

Update MOM6 tag for USE_GRID_SPACE_DIAGNOSTIC_AXES fix

b0a3e79

update MOM6 tag to inlclude diag_send_complete commit

84c0e5a

yichengt900 removed the pass_CEFI_MOM6_RT label May 31, 2024

yichengt900 added 2 commits June 14, 2024 19:18

Merge remote-tracking branch 'origin' into fix/fms2_nudge

624a4a2

Update CEFI_NWA12_cobalt_fms2_yaml.xml

82dcc39

yichengt900 added CEFI_MOM6_RT_container and removed pass_CEFI_MOM6_RT labels Jun 14, 2024

Merge branch 'main' into fix/fms2_nudge

aa9f848

github-actions bot added pass_CEFI_MOM6_RT and removed CEFI_MOM6_RT_container labels Jun 15, 2024

fix ice_param tag in xml

28fd9f8

Merge branch 'main' into fix/fms2_nudge

db1b1ab

yichengt900 added CEFI_MOM6_RT_container and removed pass_CEFI_MOM6_RT labels Jun 24, 2024

github-actions bot added pass_CEFI_MOM6_RT and removed CEFI_MOM6_RT_container labels Jun 24, 2024

yichengt900 added 2 commits June 26, 2024 11:10

Update diag_table_COBALT_2023-04

32b39e5

Merge branch 'main' into fix/fms2_nudge

c15aff9

uwagura approved these changes Jun 27, 2024

View reviewed changes

uwagura merged commit 2d8cc03 into main Jun 27, 2024
2 checks passed

yichengt900 deleted the fix/fms2_nudge branch July 1, 2024 11:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sub-regioan diag output example in 1D case #39

Add sub-regioan diag output example in 1D case #39

yichengt900 commented May 14, 2024 •

edited

Loading

theresa-morrison commented May 22, 2024

yichengt900 commented May 29, 2024

kshedstrom commented May 29, 2024

yichengt900 commented May 29, 2024

yichengt900 commented Jun 14, 2024

kshedstrom commented Jun 18, 2024

kshedstrom commented Jun 18, 2024

yichengt900 commented Jun 18, 2024 •

edited

Loading

thomas-robinson commented Jun 18, 2024

uwagura left a comment

kshedstrom commented Jun 28, 2024 •

edited

Loading

uramirez8707 commented Jun 28, 2024

kshedstrom commented Jun 28, 2024

uramirez8707 commented Jul 2, 2024

kshedstrom commented Jul 2, 2024

uramirez8707 commented Jul 2, 2024

kshedstrom commented Jul 3, 2024

Add sub-regioan diag output example in 1D case #39

Add sub-regioan diag output example in 1D case #39

Conversation

yichengt900 commented May 14, 2024 • edited Loading

theresa-morrison commented May 22, 2024

yichengt900 commented May 29, 2024

kshedstrom commented May 29, 2024

yichengt900 commented May 29, 2024

yichengt900 commented Jun 14, 2024

kshedstrom commented Jun 18, 2024

kshedstrom commented Jun 18, 2024

yichengt900 commented Jun 18, 2024 • edited Loading

thomas-robinson commented Jun 18, 2024

uwagura left a comment

Choose a reason for hiding this comment

kshedstrom commented Jun 28, 2024 • edited Loading

uramirez8707 commented Jun 28, 2024

kshedstrom commented Jun 28, 2024

uramirez8707 commented Jul 2, 2024

kshedstrom commented Jul 2, 2024

uramirez8707 commented Jul 2, 2024

kshedstrom commented Jul 3, 2024

yichengt900 commented May 14, 2024 •

edited

Loading

yichengt900 commented Jun 18, 2024 •

edited

Loading

kshedstrom commented Jun 28, 2024 •

edited

Loading