MDTF-diagnostics preprocessor update #509
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
add routine to check the conda env specified by the POD for the required packages
… is the correct way to inherit from the modified Singleton definition
…in PODs that require the python environment, since packages for NCL are not explicitly imported in the driver scripts
… to handle objects in the Fieldlist files
add calls to register methods for abstract atts back to data_model.py
… a single standard name working on the call to Fieldlist.from_struct
add try except block to catch duplicates to _DMDimensionsMixin build_axes method
move VariableTranslator class and modifier check from data_model to translation.FieldlistEntry __post_init method to avoid circular import
change modeling_realm to realm in fieldlists
… string in DMDependentvariable because this class is used to define coordinates, which do not have realm atts, and variables, which do
…ifier and realm entries when defining fieldlist lookup tables remove mandatory modifier and ndim units from fieldlist class atts since they are not needed there
… the change in class names in varlist_util.py
… attribute to data_model._dmcoordinatshared class which is inherited by the same child classes in varlist_util
define pod_dims using the _pod_dims_from_struct function in Varlist from_struct call
comment out convention match check for varlist setup development in pod_setup
add date_range attribute to datasourcebase and a method to set the date range add call to set_date_range to pod_setup change varlist_util.setup_var to accept date_range and convention parameters to pass to called methods
add dummy attributes required by abstract methods in data_model clean up formatting add realm to from_CF method in translation
add module loader procedure to import custom preprocessing python scripts to DaskMultFilePreprocessor init method
remove unused deactivate_data_key method from VarlistEntry
debug the custom module load method
update example_multicase config and environment yamls
still need to debug calling custom module on xr ds and variable
…t_translation modules
change user_pp_scripts preprocessor attribute to just be a list of script names and not the full paths
…arlistentries or xarray datasets whether they perform operations or act as dummy functions
wrongkindofdoctor
requested review from
jkrasting and
aradhakrishnanGFDL
as code owners
wrongkindofdoctor
added a commit
that referenced
this pull request
Mar 8, 2024
MDTF-diagnostics preprocessor update (#509)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Preliminary updates to MDTF-diagnostics package with support for intake-esm catalogs and custom preprocessing scripts
Associated issue #81, #51
How Has This Been Tested?
This branch has only undergone basic functionality tests with a CMIP dataset. It is still under development.
Checklist:
conda_env_setup.sh