Where to save input files and scripts for new functionality?

[samsrabin]

The prescribed crop calendars update (#1863) is nearing completion, so thinking some more about logistics. In order to run with prescribed calendars, there are a few steps involved:

1. From raw GGCMI calendar files, generate CLM sowing and harvest date netCDFs at the resolution of interest (or trust CLM to interpolate correctly from files of a different resolution).
2. Perform a "GDD-generating" run, forced with sowing dates from (1), to save daily GDD accumulation.
3. Run a Python script to generate prescribed maturity requirements from that output.

I was wondering:

• Where should I put the netCDF files required by (1)? These include both the original 0.5° resolution files from GGCMI as well as the CLM versions I've made. Probably somewhere in /glade/p/cesmdata/cseg/inputdata/lnd/clm2/, right?
• Where should I put the Python scripts required by (3) and optionally (1)? Somewhere in $SRCROOT/python/ctsm/?

Thanks in advance!

[ekluzek]

Yes, I think this data should go under inputdata under the lnd/clm2 directory. The prescribed_data subdirectory could make sense, or you could create a new directory for it. The rawdata directory also makes sense, but that's for data that's input to surface dataset generation, so probably wouldn't be one to use.

The guts of the python scripts would preferentially be under python/ctsm and have testing for them as well as being able to do pylint and black checking of them. The top level script that users would use would be under the tools directory, and we probably need a new subdirectory for this type of tool. Look at the site_and_regional subdirectory for example scripts that have a small top level script with the guts put under the python/ctsm subdirectory.

[samsrabin]

Thanks, Erik!

It might be that rawdata (or a new, similarly-named directory) would be best for the raw GGCMI crop calendar files, to indicate that they're not CLM-ready. Or maybe a structure like

crop_calendars
|--- raw_ggcmi3
|--- processed

[ekluzek]

Maybe it should be a more broad subdirectory (so cropdata)? And then raw_ggcmi3_calendars/GDD_for_processed_calendars?

It's OK to assume it's a current feature, even though it's in process. We don't have a process of elevating data from experimental to main-line use. So just assume the data directory that we want to see when this is a mainline feature of the model.

[samsrabin]

@ekluzek, I think I'm ready for this to happen. Let's use the following structure:

cropdata/
|---calendars/
|    |---raw_ggcmi3_v1.01/
|    |---processed/

Could you please copy the following?

• Contents of (targets of symlinks in) /glade/work/samrabin/crop_dates_blessed/ to …/cropdata/calendars/processed/
• Contents of /glade/work/samrabin/isimip3_cropcals/ to …/cropdata/calendars/raw_ggcmi3_v1.01/

Thanks!

[ekluzek]

@samsrabin I like that directory structure. I'm happy to copy files around if needed, but since you are now official we should give you the ability to do this yourself. There's some groups we need to add you into, and some guidance that we should probably meet to chat about. So let me get that process started...