Investigating CTSM5.2 surface datasets

[wwieder]

New surface datasets have a number of changes that we can start discussing. I've created this slide deck to start illustrating changes between cases that we're running. So far, these are i2000 CTSMsp simulations with 5.2 vs. 5.1 using GSWP3 forcing. Both from a common CTSM5.1_dev090 tag. Diagnostics from the SP case are posted here with associated raw data here /glade/u/home/slevis/cases_sp

• @lawrencepj1 do you have a script that can quickly compare differences in surface dataset fields? If not, maybe @slevisconsulting can help? These include:

  • pft distributions,
  • special land units like lakes, glacier, and urban
  • soil properties (ORGANIC, CLAY and SAND + maybe the derived constants that getting used in the model like WATSAT or HKSAT that are written out to history files).

• Land over fractions, especially for forests / grasses, see the PFT fractions (e.g., forests in China ). @lawrencepj1 can you check these changes are as intended?

• Lake, urban, and land ice datasets have changed for this 2000 case. The biggest thing that stands out here is the distribution of lakes , evident by the upper plots, but nothing big really stands out here.

• Soil properties have changed quite a bit, which changed soil thermal and hydraulic properties, especially in the Arctic. This makes summer soil conditions in the new simulations warmer and wetter, with less ice across the Arctic.

  • @olyson will provide a look at ALTMAX

• @slevisconsulting will run parallel BGC cases and post additional diagnostics and ILAMB plots

[olyson]

ALTMAX is only output for BGC simulations. So we'll have to use the BGC transient simulations Sam is running.

[slevis-lmwg]

@slevisconsulting will run parallel BGC cases and post additional diagnostics

I have encountered a problem with the new transient bgc crop simulation; suggestions welcome:

• Case ihist_bgccrop with default datasets has run 1850-1973 this far and continues, so no problem here.
• The parallel case with the new datasets ran from 1850-1920 with datm cycling 1901-1920 BUT...
  I have tried to continue this as a hybrid or a startup using restart files from 1921/1/1 or 1920/1/1 and I get a methane balance error.

The new datasets seem like the logical culprit, except for the fact that 1920 completed fine as a one-year restart in the cycling part of the simulation (1850-1920), while it triggers the methane balance error as a hybrid or startup for the next part (1920-2014). That's all the info I have for now. I will be troubleshooting.

[slevis-lmwg]

The transient case with the new datasets still gives me a methane balance error as follows:

• No problems running/restarting 1850-1920 with datm cycling 1901-1920.
• Hybrid or startup runs that I tried from 1901/1/1 or 1920/1/1 or 1921/1/1 gave methane balance errors.

I will continue troubleshooting. Suggestions are welcome.

[olyson]

During the initial development of dynamic urban we encountered a methane error. See more details in doc/design/dynamic_urban.rst. But this occurred the first time PCT_URBAN changed. PCT_URBAN shouldn't change until year 2001 and we had fixed that error anyway. So I don't think this is relevant, just wanted to make you aware of it. I guess you could check to see what column type(s) are contributing to that gridcell error.

[slevis-lmwg]

#43 describes a ch4 balance error with transient lakes. My ch4 balance error may be the same, although:

• I have yet to confirm that my problem is due to transient lakes and
• My balance error appears in hybrid/startup cases (e.g. 1901, 1920, 1921) though not in 1850 where the whole simulation started and not in restart/branch cases.

#43 suggests this workaround:
Comment out the gridcell balance check when running with transient lakes, maybe only for the first timestep of every year. The way the problem manifests for me, we may even just comment it out for the first timestep of the run.

[slevis-lmwg]

I plan to troubleshoot and address the ch4 balance check issue in #1873
For now I see this code already in place in subr. ch4:

      ! Gricell level balance
      !

      ! Skip the check if it's the beginning of a new year and dynamic lakes are on
      ! See (https://github.com/ESCOMP/CTSM/issues/1356#issuecomment-905963583)
      !
      if ( is_beg_curr_year() .and. get_do_transient_lakes() )then
         ch4_first_time_grc(begg:endg) = .true.
      end if

So it seems likely that the error will go away if I add a comparable flag for the first timestep of hybrid/startup simulations. I will try that next.

[slevis-lmwg]

Hybrid and Startup cases that were giving the ch4 balance error are now working.
See 858539a in #1873

[billsacks]

Awesome, great work on this @slevisconsulting !

[slevis-lmwg]

Thanks @billsacks and also thank you and @ekluzek for making suggestions at last wk's Standup.

[slevis-lmwg]

@slevisconsulting will run parallel BGC cases and post additional diagnostics

Diagnostics posted here.

[slevis-lmwg]

In "line plots" the TOTRUNOFF difference plot may suggest some kind of problem a little before 1950.

[olyson]

I think we've noticed this before, it follows the GSWP3V1 precip trend.

[olyson]

ILAMB diagnostics here:

https://webext.cgd.ucar.edu/I20TR/ihist_bgccrop/lnd/_build_fsurdat/

[lawrencepj1]

@slevisconsulting Thanks for running and posting all of this. I found a pretty crazy spike in Landuse Flux in the diagnostics which I will look into.

[slevis-lmwg]

Thank you @lawrencepj1

[wwieder]

From our meeting I think we identified this as an issue with the lakes dataset.

[olyson]

I've added a new soil moisture storage plot to set3.
Also added ALTMAX to set2, hopefully with a better color scale. See also the updated slides for ALTMAX:

https://docs.google.com/presentation/d/1fmReSb6XKQjChSjMTAxAp-p0BAR4HibsHya1y43udeg/edit#slide=id.g1630bf08395_0_25

The SP and BGC diagnostics pages have been updated with ALTMAX and soil moisture storage.

[wwieder]

@lawrencepj1 have you been able to make plots comparing soil datasets for CTSM5.1 and 5.2 simulations?

[lawrencepj1]

Sorry Will totally spaced on this. Will get on it today. -- Dr Peter Lawrence Terrestrial Science Section National Center for Atmospheric Research 1850 Table Mesa Drive Boulder Colorado 80305 Work: 1-303-497-1727 Cell: 1-303-956-6932

…

On Thu, Oct 27, 2022 at 10:22 AM will wieder ***@***.***> wrote: @lawrencepj1 <https://github.com/lawrencepj1> have you been able to make plots comparing soil datasets for CTSM5.1 and 5.2 simulations? — Reply to this email directly, view it on GitHub <#1868 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC3OJOJRUM5YA6GS5QLLDFTWFKT4FANCNFSM6AAAAAAQ733474> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wwieder]

Not to worry Peter. I know you were involved with the workshop this week too, hope it went well.

[wwieder]

@lawrencepj1 have you been able to make plots from the first item in this discussion, or should someone else dive into this? Especially related to differences in tree and grass cover with the new surface dataset?

[lawrencepj1]

Hi Will

I just checked and while some of these changes in trees are due to the new lake dataset much of the changes shown in this plot are coming from updates to the raw pft/cft data from redoing the crop and pasture data. I have put in a PCT_TREE plot for the CTSM52 raw data vs the CLM5 data for the year 2015. For turn around time the plot is from ncview with the legend running over similar ranges as the ncl diagnostic difference plot.

[lawrencepj1]

@wwieder realized I didn't tag you on this update

[wwieder]

excellent. Thanks Peter. Changes in China are really big! Maybe we can post differences in soil properties here too for completeness?

[wwieder]

Hello, I wanted to follow up on a few points from our conversation on this a few weeks ago:

• Shifting cultivation @lawrencepj1 this needs to be added? What's an appropriate timeline to implement this?
• Lake dataset @slevisconsulting were you going to investigate the lake area issues around 1940?
• Soil properties @lawrencepj1 have you been able to compare 'ORGANIC' values in surface soils from the two surface data?
• Permafrost @olyson made new difference plots in ALT. @dlawrenncar is this enough to evaluate permafrost dynamics?
• Wetlands @billsacks has opened a PR for this.

Are there other topics to follow up on?

[lawrencepj1]

Hi Will

Yes we talked about the timelines for the first two component being the end of next week for 1. and by the end of the week after for 2. The remaining tasks will be over the next month to two months. We will be having a follow up meeting next week to discuss post CTSM5.2 issues. This will include the FATES / LUH2(3) developments required to include primary and secondary forests and non-forests as well as pasture and rangeland. It will also include the ability to represent some data currently on the surface and landuse timeseries files on streams. This may involve updating the current climatology LAI/SAI/Canopy height data along with harvest and fertilizer data.

Peter 

components 1 and 2:

@lawrencepj1 will generate the new 850-2100 pft data plus lai and soilcolor data this month. He needs a new mesh file to complete this work; @lawrencepj1 will reach out to @ekluzek (or @slevisconsulting ) in the next week or so to generate the new mesh file.
@lawrencepj1 will generate corresponding timeslice data for the standard high-res resolution. Decision is pending whether to offer another high-res resolution and whether to offer timeseries data for the high-res resolution(s).

[wwieder]

Thanks for these additional details.

I see from @slevisconsulting's notes above that @ekluzek is also supposed to organize a post CTSM5.2 discussion. Is this something we should do at our weekly CLM meeting on Thursday, or does the conversation need a smaller, more focused group?

[slevis-lmwg]

@wwieder
Erik will set up the post-ctsm5.2 discussion most likely for 1/10 (tomorrow).

[slevis-lmwg]

Update of TODOs in this discussion:

• @lawrencepj1 will generate the new 850-2100 pft data plus lai and soilcolor data this month.

DONE out to 2300

• @lawrencepj1 will generate corresponding timeslice data for the standard high-res resolution.

PEND

• @lawrencepj1 will place new files in the /glade/p/cesmdata/cseg/inputdata/lnd/clm2/rawdata space

DONE for 850-2300
PEND for high-res

@slevisconsulting (or @ekluzek ) will ./rimport the files to the svn repo

TODO after testing as part of PR #1946

• @lawrencepj1 will move his codes/workflow to github.com/ESCOMP and will teach others (@slevisconsulting @ekluzek ...) how to use them

PEND

@ekluzek is supposed to look at #309 before reviewing with @slevisconsulting and merging

@slevisconsulting will update #309 to main and perform testing; @ekluzek will merge when ready
#1920 got merged

I generated the 1.9x2.5 transient surface data using the new raw pft data as requested by @samsrabin. SamR is testing. I posted a bit more detail in #1946.

[lawrencepj1]

Thanks for the update on the pending and completed tasks @slevisconsulting

[dlawrenncar]

I think the permafrost simulation is fine. There are some changes, but nothing so big that it would be a big concern.

…

On Thu, Nov 3, 2022 at 6:16 PM Samuel Levis ***@***.***> wrote: - *Lake dataset* @slevisconsulting <https://github.com/slevisconsulting> were you going to investigate the lake area issues around 1940? I was in touch with @Ivanderkelen <https://github.com/Ivanderkelen> roughly 10 days ago, and she agreed to send updated raw files for 1940 and 1941 soon. Are there other topics to follow up on? - For completeness, I am expecting more new raw datasets from @lawrencepj1 <https://github.com/lawrencepj1> i.e. for 850-1849 and for the SSP/RCPs. - Also from @lawrencepj1 <https://github.com/lawrencepj1> I'm expecting new hi-res raw datasets, some for the old hi-res resolution and all for the new hi-res resolution. Then I'm supposed to test the hi-res option. - @ekluzek <https://github.com/ekluzek> is looking at #309 <#309> and then we are supposed to merge to the ctsm5.2 branch. — Reply to this email directly, view it on GitHub <#1868 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFABYVHTLJIKH3JYDWB7GFTWGRIUBANCNFSM6AAAAAAQ733474> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[slevis-lmwg]

New minus old ORGANIC [kg m-3]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

org_matt_dens_by_soil_lev_-200to200_kg_m-3_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New ORGANIC [kg m-3]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

org_new_by_soil_lev_0to300_kg_m-3_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

Old ORGANIC [kg m-3]. Same colorbar as the new (directly above).
The video starts with layer 1 and loops through the 10 soil layers multiple times. The old organic goes to 0 everywhere in the bottom two layers.

org_old_by_soil_lev_0to300_kg_m-3_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New minus old SAND [-50% to 50%]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

sand_by_soil_lev_-50to50._surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New SAND [0 to 100%]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

sand_new_by_soil_lev_0to100._surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

Old SAND [0 to 100%]. Same colorbar as the new (directly above). The video starts with layer 1 and loops through the 10 soil layers multiple times.

sand_old_by_soil_lev_0to100._surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New minus old CLAY [-50% to 50%]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

clay_by_soil_lev_-50to50_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New CLAY [0 to 100%]. The video starts with layer 1 and loops through the 10 soil layers multiple times.

clay_new_by_soil_lev_0to100_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

Old CLAY [0 to 100%]. Same colorbar as the new (directly above). The video starts with layer 1 and loops through the 10 soil layers multiple times.

clay_old_by_soil_lev_0to100_surfdata_0.9x1.25_hist_78pfts_CMIP6_simyr1850_c221003_minus_c190214.mp4

[slevis-lmwg]

New minus old HKSAT [-0.27 to 0.27 mm s-1]. Same colorbar colorscheme as prev. variables. The video starts with layer 1 and loops through the 25 ground levels a few times.

hksat_by_grnd_lev_-027to027_mm_s-1_ihist_bgccrop_jan1983_minus_ihist_bgccrop_old_fsurdat_jan2010.mp4

[slevis-lmwg]

New HKSAT [0 to 0.2744 mm s-1]. Same colorbar colorscheme as prev. variables. The video starts with layer 1 and loops through the 25 ground levels a few times.

hksat_new_0to0.2744_mm_s-1_ihist_bgccrop.clm2.h0.1983-01.mp4

[slevis-lmwg]

Old HKSAT [0 to 0.2744 mm s-1]. Same colorbar colorscheme as prev. variables. The video starts with layer 1 and loops through the 25 ground levels a few times.

hksat_old_0to02744_mm_s-1_ihist_bgccrop_old_fsurdat.clm2.h0.2010-01.mp4

[wwieder]

At the Feb 28th co-chairs meeting we're discussing developments in component models with an eye to the next round of coupled model testing that's scheduled to start in March. Can we consider using a 5.2 surface dataset on this timeframe (just for a single model resolution), or would this be too soon to be ready? My guess is too soon, as we also need to provide initial conditions for coupled model runs, but I thought I'd ask.

[samsrabin]

Depending on when the revised 5.2 datasets become available at "2-degree" resolution, I may rerun the experiments for my crop calendar paper with them. This will include a "CLM Default" run starting with the 1850 surface dataset, then time-varying land use.

That said, I'm not set up to do analysis of anything other than crop-related outputs.

[lawrencepj1]

Hi @slevisconsulting and @samsrabin Yes the timeframe for the datasets should be fine. The issues on crop distribution we found with Sam Rabin have been resolved so the raw data will be updated in the next week for the February 28th meeting. Thanks Peter

[slevis-lmwg]

A missing piece still is #309 which is close but not done. Assuming today's test-suite passes, I have two more updates before we merge to main, update the ctsm5.2 branch, and test again. Other than waiting for test-suites, the slowest part of the process is debugging conflicts that don't get caught by git.

Having said all that, I think it makes sense to generate the needed fsurdat now, and then again when #309 is in. I could do that if you let me know what resolution we need for the coupled testing.

[wwieder]

Thanks @slevisconsulting I'll start an email thread with those involved in the coupled model tests to ask about resolution. I like the idea of providing a beta version of a 5.2 surface dataset to testing.

How time consuming is it going to be to generate initial conditions to run in the coupled model, @olyson?

Is it worth this effort for testing coupled configurations? (I'm pretty naive regarding how to proceed with these coupled model tests).

[olyson]

As you suggested, we should have a science discussion on this. I think there were some discussions about spinning up CLM using forcing from a CAM-CLM F-case simulation in which CAM and CLM are as close to the fully coupled configuration being used for CESM development simulations. We used to do this type of spinup using coupler history files but I thought I saw somewhere that nuopc didn't support that kind of configuration? So there may be a bit of time involved in understanding how to do this.

[billsacks]

I saw somewhere that nuopc didn't support that kind of configuration? So there may be a bit of time involved in understanding how to do this.

I believe this will be possible once we integrate #1929

[billsacks]

Actually, I think my last comment may have been wrong. @adrifoster reminded me of #1844 , and I think that may be the remaining problem. If so, I'm not sure that #1929 is strictly needed for that. (I'm having deja vu: I know we've gone around a few times about what #1929 is and isn't actually needed for.) On the other hand, I think that ESCOMP/CAM#735 is needed: Once CAM always passes NDEP, we can generate new cplhist files from nuopc/cmeps, then use those new cplhist files to force datm using CDEPS. At that point, it would probably make sense to also change CTSM to actually listen to the NDEP sent from ATM. (The need for the CAM update is because DATM expects NDEP on its forcing files, even though CTSM doesn't actually use NDEP from ATM yet.)

[billsacks]

See #1951 (comment) for more details.

[klindsay28]

change CTSM to actually listen to the NDEP sent from ATM.

I thought that was already happening. Is it not done by the following block in the NUOPC cap?

CTSM/src/cpl/nuopc/lnd_import_export.F90

Lines 668 to 677 in 05fb17b

	! Atmosphere ndep
	if (fldchk(importState, Faxa_ndep)) then
	! The mediator is sending ndep in units of kgN/m2/s - and ctsm
	! uses units of gN/m2/sec so the following conversion needs to happen
	call state_getimport_2d(importState, Faxa_ndep, forc_ndep(begg:,:), rc=rc)
	if (ChkErr(rc,__LINE__,u_FILE_u)) return
	do g = begg, endg
	atm2lnd_inst%forc_ndep_grc(g) = (forc_ndep(g,1) + forc_ndep(g,2))*1000._r8
	end do
	end if

[olyson]

Ok, sorry for mis-stating this, I was thinking of this discussion:

NCAR/amwg_dev#188

in which Adam and Mariana were debugging the cpl history output option in nuopc.
The last entry states that "Quick update - we've resolved the issue of generating the streams w/ nuopc. We are meeting on wednesday to iron out setting those streams up for forcing clm standalone."

I'm not sure if this is associated with an issue/pull request that has been implemented already.

[billsacks]

@klindsay28 and @olyson - sorry, I feel like my comments have been more misleading / wrong than helpful: they were based on my recollections of conversations from a few months ago rather than any first-hand knowledge of the current state. I think you both have a better sense of the current state of affairs than I do.

[wwieder]

@olyson completed a historical using the datasets @slevisconsulting created. Diagnostics compared to a CLM5 run are here:

https://webext.cgd.ucar.edu/I20TR/ctsm51_ctsm51d118_ne30pg3g17_GSWP3V1_hist/lnd/ctsm51_ctsm51d118_ne30pg3g17_GSWP3V1_hist.1994_2013-clm50_ctsm51d118_ne30pg3g17_GSWP3V1_hist.1994_2013/setsIndex.html

I don't see anything unexpected. For example, the spike in the landuse flux that @lawrencepj1 noted previously (due to a lake dataset problem) isn't there.

[lawrencepj1]

Hi @slevisconsulting @wwieder @ekluzek @samsrabin

Just making a quick note that the distributions of individual CFTs has been updated following suggestions from Sam Rabin and as presented at the 2023 LMWG meeting. The updated files are now available on

/glade/p/cesmdata/cseg/inputdata/lnd/clm2/rawdata/

pftcftdynharv.0.25x0.25.LUH2.histsimyr0850-1850.c20230226
pftcftdynharv.0.25x0.25.LUH2.histsimyr1850-2015.c20230226

pftcftdynharv.0.25x0.25.SSP1-1.9.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP1-2.6.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP2-4.5.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP3-7.0.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP4-3.4.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP4-6.0.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP5-3.4.simyr2015-2100.c20230226
pftcftdynharv.0.25x0.25.SSP5-8.5.simyr2015-2100.c20230226

pftcftdynharv.0.25x0.25.SSP1-2.6.simyr2100-2300.c20230226
pftcftdynharv.0.25x0.25.SSP5-3.4.simyr2100-2300.c20230226
pftcftdynharv.0.25x0.25.SSP5-8.5.simyr2100-2300.c20230226

Note the directories and file names now for 2023 02 26.

These updated raw files should be considered the CTSM 5.2 land cover and land use data rather than the previous version. We should remove the 2023 01 16 version once we have this finalized.

Thanks
Peter

[slevis-lmwg]

Thank you @lawrencepj1

[samsrabin]

My f19 crop runs with the 5.2 datasets are complete, and I've made some simple global timeseries figures to illustrate the difference. Overall, area looks much better, and the production (Mt) values look either improved or similar. Yield (t/ha) is either similar or slightly worsened (overestimate increased, for corn and wheat).

Here, cyan is with the 5.0 datasets and pink is with 5.2. "Total (grains)" means everything except cotton and sugarcane.

Area

Production

Yield

Numbers in subplot titles are mean bias over 1980–2009, 5.0 → 5.2.