Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Install pyarrow-core instead of pyarrow from conda-forge #3698

Merged
merged 4 commits into from
Dec 19, 2024

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Dec 19, 2024

Description of proposed changes

Reduce unnecessary optional dependencies when installing pyarrow from conda-forge. Xref conda-forge/pyarrow-feedstock#111

From https://github.com/conda-forge/pyarrow-feedstock/blob/381a88b6f8957204e34047151c4ca8a3f1e54204/recipe/meta.yaml#L157-L164, this means we won't be installing libarrow-acero, libarrow-dataset, libarrow-substrait and libparquet.

Fixes #

Preview:

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
  • If adding new functionality, add an example to docstrings or tutorials.
  • Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash command is:

  • /format: automatically format and lint the code

Reduce unnecessary optional dependencies when installing `pyarrow` from conda-forge. Xref conda-forge/pyarrow-feedstock#111
@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Dec 19, 2024
@weiji14 weiji14 added this to the 0.14.0 milestone Dec 19, 2024
@weiji14 weiji14 self-assigned this Dec 19, 2024
@weiji14 weiji14 marked this pull request as ready for review December 19, 2024 02:16
@@ -73,7 +73,7 @@ jobs:
numpy-version: '1.25'
pandas-version: '=2.0'
xarray-version: '=2023.04'
optional-packages: ' contextily geopandas<1 ipython pyarrow rioxarray sphinx-gallery'
optional-packages: ' contextily geopandas<1 ipython pyarrow-core rioxarray sphinx-gallery'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also need to update line 82.

Do we also want to update doc/install.md?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done in commits 3e59046 and 0cef357

@seisman
Copy link
Member

seisman commented Dec 19, 2024

Aslo need to note that pyarrow-core is available since pyarrow v16 (released in April, 2024). Installing pyarrow-core in ci_tests_legacy.yaml may cause issues in #3639 (comment).

@weiji14
Copy link
Member Author

weiji14 commented Dec 19, 2024

Aslo need to note that pyarrow-core is available since pyarrow v16 (released in April, 2024). Installing pyarrow-core in ci_tests_legacy.yaml may cause issues in #3639 (comment).

Yes, thanks for the reminder, I think we can use pyarrow-core here, and I'll revert to using pyarrow in #3639. That PR is still a little experimental. Edit: Actually, #3639 is installing pyarrow from PyPI (via uv) instead of conda-forge, so it doesn't really matter.

@weiji14 weiji14 merged commit 387c69f into main Dec 19, 2024
19 checks passed
@weiji14 weiji14 deleted the deps/pyarrow-core branch December 19, 2024 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants