-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify conda environments #198
Conversation
- Loosen up constraints to improve upgradeability - Remove `readthedocs.yml`
Hi @TonyB9000, I'm tagging you for review so you can familiarize yourself with the changes. You can adopt a similar approach when you open up a PR to simplify the datasm conda-env yaml files. Please approve once you are good to go. |
@@ -35,32 +35,61 @@ jobs: | |||
- if: ${{ steps.skip_check.outputs.should_skip != 'true' }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can ignore the changes in this file. This is purely DevOps work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahh. DevOps work. (complete mystery).
@@ -4,14 +4,14 @@ fail_fast: true | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ignore the changes in this file as well.
# Conda e3sm_to_cmip CI/CD environment (used in GH Actions). | ||
name: e3sm_to_cmip_ci | ||
channels: | ||
- conda-forge | ||
- defaults | ||
dependencies: | ||
# Base | ||
# ================== | ||
- python>=3.9 | ||
- pip | ||
- cmor>=3.6.0 | ||
# Base - required for building the repository as an Anaconda package | ||
# ==================== | ||
- python >=3.9 | ||
- cdms2=3.1.5 | ||
- cdutil=8.2.1 | ||
- cmor >=3.7.0 | ||
- dask | ||
- nco | ||
- nco >=5.1.4 | ||
- netcdf4 | ||
- numpy | ||
- numpy >=1.23.0 # This version of numpy includes support for Python 3.11. | ||
- pyyaml | ||
- tqdm | ||
- xarray | ||
# Required for phalf and pfull handlers. | ||
- cdms2=3.1.5 | ||
- cdutil=8.2.1 | ||
- xarray >=2022.02.0 # This version of Xarray drops support for Python 3.8. | ||
# Testing | ||
# ================== | ||
- pytest | ||
- pytest-cov | ||
# Documentation | ||
# ================= | ||
- sphinx | ||
- sphinx_rtd_theme | ||
prefix: /opt/miniconda3/envs/e3sm_to_cmip_ci |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
File of interest
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have no problem with these changes. But I have no good idea how you knew to make them, nor how to apply anything similar to the datasm proc/pub.yml specs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reasoning behind constraints:
- python - we want to support 3.9 or higher
- cdms2 and cdutil - these are the latest versions and we need them for pfull/phalf handlers. We don't want old versions because there are probably bugs in them and dependency compatibility issues
- nco - I presumed we want 5.1.4 or greater based on our past experience working with this package in e2c (this will pick up the latest version)
- numpy - there is a comment in the yml file
- xarray - there is a comment in the yml file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tomvothecoder Thanks Tom, I've already (sort of) modified my datasm proc.yml based on this guidance. There is some overlap. Yet to test datasm (will wait until e3sm_to_cmip testing is completed.)
# Base - required for building the repository as an Anaconda package | ||
# ==================== | ||
- python >=3.9 | ||
- cdms2=3.1.5 | ||
- cdutil=8.2.1 | ||
- cmor >=3.7.0 | ||
- dask | ||
- nco >=5.1.4 | ||
- netcdf4 | ||
- numpy >=1.23.0 # This version of numpy includes support for Python 3.11. | ||
- pyyaml | ||
- tqdm | ||
- xarray >=2022.02.0 # This version of Xarray drops support for Python 3.8. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
File of interest
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See previous comment.
I merged. I did a pull in my repo that brought down the changes to master.
I issued I issued I issued I issued
Not sure how to proceed from here. |
@TonyB9000, presumably If some warning is happening that you want to suppress, you will have to find another way. |
@xylar Thanks Xylar. I hope to add an option to suppress all of the "progress bar" output - in batch mode these just create MB of non-print chars in log files. |
Sounds good but that's an entirely unrelated issue |
I can now get "e3sm_to_cmip --version" to report Fixed now, I think. I am getting the expected
If it works with mode 3 (checking for variable support in supplied data-path), I'll push this. But should be in another issue somewhere (not #198). I'll use #197 I think. |
Thanks Xylar. I added a FIXME line there 5 months ago according to the git blame. It looks like this has caught up now as an actual issue (#196). |
ci.yml
anddev.yml
constraints to improve upgradeabilityconda-env/readthedocs.yml
and useci.yml
instead inreadthedocs.yml
config