Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use caching to speed up testdata fetching, fix upstream builds, scipy suggestions #1906

Merged
merged 31 commits into from
Sep 17, 2024

Conversation

Zeitsperre
Copy link
Collaborator

@Zeitsperre Zeitsperre commented Sep 5, 2024

Pull Request Checklist:

  • This PR addresses an already opened issue (for bug fixes / features)
  • Tests for the changes have been added (for bug fixes / features)
    • (If applicable) Documentation has been added / updated (for bug fixes / features)
  • CHANGELOG.rst has been updated (with summary of main changes)
    • Link to issue (:issue:number) and pull request (:pull:number) has been added

What kind of change does this PR introduce?

  • Employs caching of pip dependencies, the tox environments and the xclim-testdata cache to speed up the CI setup on subsequent builds.
  • Deals with an issue that was affecting the upstream build dependencies, causing their installation to silently fail.

Does this PR introduce a breaking change?

No.

Other information:

https://github.com/actions/cache

Preliminary results suggest that this will shave at least 30 seconds off of the setup times for most builds (except for prefetch-based tox environments, as this build is designed to test the fetching of the test data).

Also worth noting: Caches are immutable once created!

@Zeitsperre Zeitsperre self-assigned this Sep 5, 2024
@github-actions github-actions bot added the CI Automation and Contiunous Integration label Sep 5, 2024
Copy link

github-actions bot commented Sep 5, 2024

Note

It appears that this Pull Request modifies the main.yml workflow.

On inspection, the XCLIM_TESTDATA_BRANCH environment variable is set to the most recent tag (v2024.8.23).

No further action is required.

@Zeitsperre
Copy link
Collaborator Author

This is ready. Will perform a squash when merging. Thanks!

@github-actions github-actions bot added the approved Approved for additional tests label Sep 16, 2024
@Zeitsperre Zeitsperre changed the title Use caching to speed up testdata fetching Use caching to speed up testdata fetching, fix upstream builds Sep 16, 2024
Zeitsperre and others added 6 commits September 16, 2024 15:24
### What kind of change does this PR introduce?

* Applies several linting suggestions adopted by the `scipy` community
* Replaces `isort` with `ruff` formatting
* Adds Markdown file formatting
* Adds a few strongly encouraged `ruff` checks (`bugbear`, `pyupgrade`)
* Adjust `mypy` checks

### Does this PR introduce a breaking change?

Yes. `isort` is no longer required. Code checks are marginally stricter.

### Other information:

https://github.com/scientific-python/repo-review

If we wanted to replace `black` as well, this is feasible with `ruff`,
as it nearly replicates its functionality 1-to-1. To be revisited.

I've staged the `python-check-blanket-noqa` and
`python-check-blanket-type-ignore` hooks since there are several
instances where these won't work (`numba`-based code blocks). These
statements should be followed up on at some point, likely when we
address more things with `mypy`.
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions github-actions bot added sdba Issues concerning the sdba submodule. API Interfacing and User Concerns docs Improvements to documenation labels Sep 17, 2024
@Zeitsperre Zeitsperre changed the title Use caching to speed up testdata fetching, fix upstream builds Use caching to speed up testdata fetching, fix upstream builds, scipy suggestions Sep 17, 2024
@coveralls
Copy link

Coverage Status

coverage: 89.394% (-0.008%) from 89.402%
when pulling 004b95a on use-more-caching
into 08656c1 on main.

@Zeitsperre Zeitsperre merged commit 61a5274 into main Sep 17, 2024
22 of 23 checks passed
@Zeitsperre Zeitsperre deleted the use-more-caching branch September 17, 2024 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Interfacing and User Concerns approved Approved for additional tests CI Automation and Contiunous Integration docs Improvements to documenation sdba Issues concerning the sdba submodule.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upstream deps test not testing upstream deps
4 participants