Errors when running iLAMB

[daleihao]

When using zppy to run ILAMB in chrysalis, I encountered the issues related to file path and variable match.

1. The auto-generated E3SM time-series files path are not correct.
2. Most of variable names in EAM/ELM outpur donot match with the variable name in the iLAMB configuration file.

[chengzhuzhang]

@daleihao could you please share the zppy configuration file and make the zppy run directory accessible for trouble shooting? iLAMB support CMIP convention, there is a intermediate task to convert E3SM variables to CMIP-like, I suspect something was wrong in this conversion.

[daleihao]

Thanks @chengzhuzhang. The zppy configuration file is in /lcrc/group/e3sm/ac.dalei.hao/E3SMv3_dev/post.20230907.v3alpha02_TOP.F20TR.chrysalis_iLAMB.cfg. The generated zppy script files are /lcrc/group/e3sm/ac.dalei.hao/E3SMv3_dev/20230907.v3alpha02_TOP.F20TR.chrysalis/post/scripts.

[chengzhuzhang]

@daleihao, thanks for the files. It does look like the cmip conversion is not turned on. In this example .cfg file https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/test_complete_run_chrysalis.cfg
there is a ts_fmt parameter,which needs to be turned on for generating additional CMIP-format time series files. For the ilamb case, you will need to set ts_fmt = "cmip" for both monthly atmosphere and land variables. For variables that e3sm to cmip conversion supported for now, they can be find here:

zppy/zppy/templates/ts.bash

Line 151 in 8a33b70

srun -N 1 e3sm_to_cmip \

[daleihao]

Got you. Thanks @chengzhuzhang ! I will have a try.

[daleihao]

Hi @chengzhuzhang , After adding ts_fmt in the script, it can run successfully. However, in the generated iLAMB results (https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.dalei.hao/E3SMv3_dev/v3alpha02_TOP.F20TR/ilamb/180x360_aave_1985-2014/), just few forcing ATM variables and RUNOFF are evaluated. All the land variables are not evaluted, although they are included in ts chunk, e.g., FSH, EFLX_LH_TOT, FSA.

[daleihao]

Although the status file says OK, but in the log file, I see "Errors occurred in the run, please consult ./_build/ILAMB01.log for more detailed information

Completed in 0:04:44"

[chengzhuzhang]

Yes, this does seem strange. I'm looking at the ilamb viewer and none of the plots are generated ...I also checked the test case generated by @forsyth2 , where a few variables has map generated: https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_test_complete_run_www/v2.LR.historical_0201/ilamb/180x360_aave_1850-1851/, not sure if @forsyth2 has some insights here.

[acme-y9s]

@daleihao @chengzhuzhang I gave it a try a few days ago. Here is my link for the maps https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xnshi/20221127.v2.LR.BGC-LNDATM.CONTRL.ne30pg2_r05_EC30to60E2r2.chrysalis/ilamb/1960_2010/v2.LR.BGC-LNDATM.CONTRL/ilamb/180x360_aave_1960-2009/

[chengzhuzhang]

@acme-y9s thank you! Your results are good. At this point, I'm not sure why Dalei's run did not generate plots. Xiaoying, could you please share your zppy config file for Dalei to compare?

[daleihao]

Xiaoying ran the ELM-BGC mode. My run is at ELM-SP mode. So I am curious about why all the hydrology- and energy-related variables are not mapped.

[daleihao]

I find just few variables are formated as cmip style by e3sm_to_cmip in https://github.com/E3SM-Project/zppy/blob/8a33b7014a01234a1adc7895f4cb3ffd2523f74c/zppy/templates/ts.bash#L155C3-L157C12. Dose this cause the issue?

[chengzhuzhang]

hmm, I don't know about the ELM-SP mode...but it does seem like the below list of files has CMIP format generated:

/lcrc/group/e3sm/ac.dalei.hao/E3SMv3_dev/20230907.v3alpha02_TOP.F20TR.chrysalis/post/lnd/180x360_aave/cmip_ts/monthly
(e3sm_unified_1.9.0_login) [ac.zhang40@chrlogin1 monthly]$ ll
total 185664
-rw-rw----+ 1 ac.dalei.hao E3SM 27119971 Sep 15 13:59 evspsblveg_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc
-rw-rw----+ 1 ac.dalei.hao E3SM 28680605 Sep 15 13:59 mrros_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc
-rw-rw----+ 1 ac.dalei.hao E3SM 28330198 Sep 15 13:59 mrsos_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc
-rw-rw----+ 1 ac.dalei.hao E3SM 35742016 Sep 15 13:59 mrro_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc
-rw-rw----+ 1 ac.dalei.hao E3SM 35119952 Sep 15 13:59 evspsblsoi_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc
-rw-rw----+ 1 ac.dalei.hao E3SM 35071741 Sep 15 13:59 tran_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc

We should at least see these variables being evaluated in ilamb.

The e3sm_to_cmip handlers are built based on the list of variables E3SM contribute to CMIP, if more variables are needed. New handlers needs to be added to e3sm_to_cmip script`.

[daleihao]

Oh, I see. We need to add more land variables into the lists. Now most variables are related to carbon cycle. My iLAMB results are in https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.dalei.hao/E3SMv3_dev/v3alpha02_TOP.F20TR/ilamb//180x360_aave_1985-2014/. You can see that runoff was mapped. Jill Chengzhu Zhang ***@***.***> 于2023年9月15日周五 13:35写道：

…

hmm, I don't know about the ELM-SP mode...but it does seem like the below list of files has CMIP format generated: /lcrc/group/e3sm/ac.dalei.hao/E3SMv3_dev/20230907.v3alpha02_TOP.F20TR.chrysalis/post/lnd/180x360_aave/cmip_ts/monthly (e3sm_unified_1.9.0_login) ***@***.*** monthly]$ ll total 185664 -rw-rw----+ 1 ac.dalei.hao E3SM 27119971 Sep 15 13:59 evspsblveg_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc -rw-rw----+ 1 ac.dalei.hao E3SM 28680605 Sep 15 13:59 mrros_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc -rw-rw----+ 1 ac.dalei.hao E3SM 28330198 Sep 15 13:59 mrsos_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc -rw-rw----+ 1 ac.dalei.hao E3SM 35742016 Sep 15 13:59 mrro_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc -rw-rw----+ 1 ac.dalei.hao E3SM 35119952 Sep 15 13:59 evspsblsoi_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc -rw-rw----+ 1 ac.dalei.hao E3SM 35071741 Sep 15 13:59 tran_Lmon_E3SM-1-0_piControl_r1i1p1f1_gr_198501-201412.nc We should at least see these variables being evaluated in ilamb. The e3sm_to_cmip handlers are built based on the list of variables E3SM contribute to CMIP <https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/925500501/Lmon+variable+conversion+table>, if more variables are needed. New handlers needs to be added to e3sm_to_cmip script`. — Reply to this email directly, view it on GitHub <#507 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMSGUORR2DJJEJHVTSEEPXTX2S3YJANCNFSM6AAAAAA42DXODE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[daleihao]

@chengzhuzhang I guess some variables with the cimp format are not included in ilamb datatsets and thus they are not mapped.

[chengzhuzhang]

Yes, this one looks better. @acme-y9s, @BunnyVon and I had a thread to discuss the ilamb support needed for supporting ELM component. We can plan to expand e3sm_to_cmip supported variable list for additional variables can be evaluated with ilamb.

[daleihao]

Thanks a lot, @chengzhuzhang! More hydrology- and energy-related variables can be added in the list if it is possible.

[chengzhuzhang]

Yes, let's plan to expand list. As a start point, could you please take the lead to add these variables to the table (here starting row 28) We can have @acme-y9s and @BunnyVon for a review and then get the formula to coded into e3sm_to_cmip.

[daleihao]

Sure. I will first discuss with @bishtgautam and then fill in the table. Thanks a lot!

[chengzhuzhang]

That's great! Thanks for checking with @bishtgautam.

One relevant note: @minxu74 developed a script to convert ELM output to CMIP format, but that was before e3sm_to_cmip is created. And I'm sure there have been updates in ELM output. I think the variable list we can refer to.