Generalizing FV3Forecast(Driver)

[benjamin-cash]

Hey everyone,

I've been reading through some of the code and had a couple of comments. After chatting with @christinaholtNOAA at AMS I'm putting them here so they are more available than just slack DMs. :) In FV3Forecast, there are a couple of places where things are hard coded - the required subdirectories (INPUT, RESTART) and the required config files in _prepare_config_files. Both of those are configuration dependent, and will need to be changed for the coupled model (for example). How difficult would it be to move those hard coded lists out into a configuration file and make that code reusable for other versions of the model?

[maddenp-noaa]

@benjamin-cash Sorry that this discussion has languished here for so long -- time flies. I think that some things have changed w.r.t. to the uwtools FV3 driver since you posted, and I wonder if we can revisit this and see if we need to distill some action items to address any shortcomings with the current code. You might need to refresh my memory about the underlying issues, too.

Looking at the current FV3 driver, it looks like it will create an INPUT directory and populate it with lateral-boundary-condition files when configured for a regional domain (and not otherwise). It will also create a RESTART directory. Here's where I need a memory refresh: Is the concern that these are not inherent to FV3 itself and potentially should not be created for some non-UFS (or even non-SRW) use cases?

One development since your initial post is that we've added some support for external (i.e. not defined in the uwtools codebase) and coupled drivers, and have a repository for developing these here. We haven't done a lot with it yet, but there's a nascent driver for a coupled Coastal app here. The coastal.py module demonstrates the approach: Drivers for components -- CDEPS and SCHISM in this case -- are imported from the uwtools API and called upon to provision run-directory items (namelist files, stream definition files, etc.) that they are responsible for. The coupled driver can then assemble a provisioned run directory from both other drivers' material and from unique tasks that it defines for itself. So I wonder if this addresses any concerns: An external coupled driver can request, à la carte, only the items it wants from a built-in uwtools driver -- so, of course, a coupled model that includes FV3 would not ask for a runscript from the FV3 driver, as it will define its own, but it might call on e.g. the FV3 driver's namelist_file task to provision an appropriate namelist file.

Please let me know your thoughts on the above.

[benjamin-cash]

Hi @maddenp-noaa. It's not quite "Is the concern that these are not inherent to FV3 itself and potentially should not be created for some non-UFS (or even non-SRW) use cases?" and more that other components might need those directories independent of FV3. For example, INPUT also contains files for MOM6. I haven't tried running MOM6 independent of FV3, but I strongly suspect it will need those directories even when it is used with a data atmosphere.

I guess one option would just be to have some redundancy and have MOM6 and FV3 drivers check against their respective lists of required directories and create them if they don't exist.

[maddenp-noaa]

I don't see a problem with different components in a coupled driver requiring the same directory and creating it if it doesn't exist.

For example, several of the internal utility tasks in uwtools, like here or here, include a .parent.mkdir(parents=True, exist_ok=True) call on the file/symlink they're trying to create the required containing directory, and it's not an error if it already exists.

I don't immediately see a benefit to trying to coordinate tasks so that only one of them is responsible for this. In more complicated cases, where there could be a race condition and if uwtools drivers were multithreaded, coordination would be necessary, but that's not the case here.

[maddenp-noaa]

I'll close this for now, but please feel free to re-open if more discussion is needed.