Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump poetry from 1.1.11 to 1.2.0rc2 and refresh other conda/python dependencies #343

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

weiji14
Copy link
Owner

@weiji14 weiji14 commented Oct 17, 2022

Refresh to poetry 1.2.0rc2 and taking the opportunity to bump up lots of other dependencies like GMT and RAPIDS AI libraries. Supersedes #326.

TODO:

  • Update environment.yml file:
    • Bump poetry to 1.2.0rc2, Pip to 22.3 (3bb50b5)
    • Bump gmt from 6.2.0 to 6.3.0 and RAPIDS AI libraries from 21.10 to 22.02 (21e0e99, 94dcff6)
  • Update poetry.lock file:
    • Update dependencies in lock file by changing the ^ to == in pyproject.toml, running rm poetry.lock && poetry lock, change == back to ^, and then running poetry lock --no-update to update the hash
    • Fix broken tests due to dependency updates

@weiji14 weiji14 added dependencies Pull requests that update a dependency file maintenance 🧰 Keeping things tidy labels Oct 17, 2022
@weiji14 weiji14 added this to the v0.5.0 milestone Oct 17, 2022
@weiji14 weiji14 self-assigned this Oct 17, 2022
Bumps [gmt](https://github.com/GenericMappingTools/gmt) from 6.2.0 to 6.4.0.
  - [Release notes](https://github.com/GenericMappingTools/gmt/releases)
  - [Commits](GenericMappingTools/gmt@6.2.0...6.4.0)

Bumps [cuml](https://github.com/rapidsai/cuml) from 21.10.00 to 22.10.00.
- [Release notes](https://github.com/rapidsai/cuml/releases)
- [Changelog](https://github.com/rapidsai/cuml/blob/branch-22.10/CHANGELOG.md)
- [Commits](rapidsai/cuml@v21.10.00...v22.10.00)

Bumps [cuspatial](https://github.com/rapidsai/cuspatial) from 21.10.00 to 22.10.00.
- [Release notes](https://github.com/rapidsai/cuspatial/releases)
- [Changelog](https://github.com/rapidsai/cuspatial/blob/branch-22.10/CHANGELOG.md)
- [Commits](rapidsai/cuspatial@v21.10.00...v22.10.00)

Geospatial packages all aligned to GDAL 3.5. Also bumped gxx_linux-64 from 11.2.0 to 11.3.0, geos from 3.9.1 to 3.11.0, proj from 8.0.1 to 9.0.1.
Bumps [cupy-cuda11x](https://github.com/cupy/cupy) from 9.5.0 to 11.2.0.
- [Release notes](https://github.com/cupy/cupy/releases)
- [Commits](cupy/cupy@v9.5.0...v11.2.0)

Bumps [dask](https://github.com/dask/dask) from 2021.10.0 to 2022.9.2.
- [Release notes](https://github.com/dask/dask/releases)
- [Changelog](https://github.com/dask/dask/blob/master/docs/release-procedure.md)
- [Commits](dask/dask@2021.10.0...2022.9.2)

Bumps [dask-cuda](https://github.com/rapidsai/dask-cuda) from 21.12.0a211025 to 22.10.0.
- [Release notes](https://github.com/rapidsai/dask-cuda/releases)
- [Changelog](https://github.com/rapidsai/dask-cuda/blob/branch-21.12/CHANGELOG.md)
- [Commits](rapidsai/dask-cuda@v21.12.00a...v22.10.00)

Note pinning to cupy-cuda11x instead of cupy-cuda117. Reordered conda channel priority to put nvidia and rapidsai channels first, so that cudatoolkit comes from nvidia channel. Also bumped dask-labextension from 5.1.0 to 5.3.0.
@weiji14 weiji14 force-pushed the dependencies/poetry-1.2.0rc2 branch from ff364a1 to 6e9affc Compare October 18, 2022 02:10
@weiji14
Copy link
Owner Author

weiji14 commented Oct 18, 2022

Segmentation fault on CI is caused by import cudf, bug reported upstream at rapidsai/cudf#11941.

Downgrading to RAPIDS AI 22.02 to prevent segmentation fault when doing `import cudf` on RAPIDS AI 22.06+. Need to sync to GDAL 3.3 which is why gmt and proj was downgraded too. Partial revert of 21e0e99.
To prevent `TypeError: C function cuda.ccudart.cudaStreamSynchronize has wrong signature (expected __pyx_t_4cuda_7ccudart_cudaError_t (__pyx_t_4cuda_7ccudart_cudaStream_t), got cudaError_t (cudaStream_t))`. See rapidsai/cudf#11185. Also downgraded cupy from 11.2 to 10.6 and switched from cupy-cuda11x back to cupy-cuda117.
Fix `ValueError: The truth value of a Array is ambiguous. Use a.any() or a.all()` by converting the `diff_grid` variable to a string format (it can be either a filename or xarray.DataArray). Updated the docstring to clarify this too.
@weiji14
Copy link
Owner Author

weiji14 commented Oct 19, 2022

Next problem may or may not have something to do with dask. Crashing on https://github.com/weiji14/deepicedrain/actions/runs/3276952377/jobs/5393616432#step:9:48, deepicedrain/tests/test_subglacial_lake_crossover.py::test_subglacial_lake_anomalies somewhere along this chunk:

for track1_track2, indexes in tqdm.tqdm(crossovers.indices.items()):
df_ = df.loc[indexes].sort_values(by=time_var)
dhdt, _ = np.polyfit(
x=df_[time_var].astype(np.int64), y=df_[elev_var], deg=1
) * (365.25 * 24 * 60 * 60 * 1_000_000_000)
if abs(dhdt) > elev_filter: # Plot only > 1 metre height change
track1, track2 = track1_track2.split("x")
fig.plot(
x=df_[time_var],
y=df_[elev_var],
zvalue=dhdt,
style="c0.1c",
cmap=True,
pen="thin+z",
)
# Plot line connecting points
fig.plot(
x=df_[time_var],
y=df_[elev_var],
zvalue=dhdt,
pen=f"faint,+z,-",
cmap=True,
)
# Get x, y and color (elev) for inset map
if outline_points:
_x, _y = df_.iloc[0][["x", "y"]]
xover["x"].append(_x)
xover["y"].append(_y)
xover["dhdt"].append(dhdt)

@weiji14 weiji14 removed their assignment Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file maintenance 🧰 Keeping things tidy
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant