Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc/v3 migration guide #2102

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Zarr-Python
:hidden:

getting_started
migration
tutorial
api/index
spec
Expand Down
2 changes: 1 addition & 1 deletion docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ latest GitHub main::

$ pip install git+https://github.com/zarr-developers/zarr-python.git

To work with Zarr source code in development, see `Contributing <contributing.html>`_.
To work with Zarr source code in development, see `Contributing <contributing.html>`_.
91 changes: 91 additions & 0 deletions docs/migration.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
Zarr-Python 3 Migration Guide
=============================

Zarr-Python 3 introduces a number of changes to the API, including a number
of significant breaking changes and pending deprecations.

This page provides a guide highlighting the most notable changes to help you
migrate your code from version 2 to version 3.

Compatibility target
--------------------

Zarr-Python 3 represents a major refactor of the Zarr-Python codebase. Some of the goals motivating this refactor included:

- adding support for the Zarr V3 specification (alongside the Zarr V2 specification)
- cleaning up internal and user facing APIs
- improving performance (particularly in high latency storage environments like cloud object store)

These goals necessitated some breaking changes to the API (hence the major version update), but we have tried to maintain
backwards compatibility in the most widely used parts of the API including the `Array` and `Group` classes and the top-level
API (e.g. `zarr.open_array` and `zarr.open_group`).

Getting ready for 3.0
---------------------

Ahead of the 3.0 release, we suggest projects that depend on Zarr-Python take the following actions:

1. Pin the supported Zarr-Python version to ``zarr>=2,<3``. This is a best practice and will protect your users from any incompatibilities that may arise during the release of Zarr-Python 3.0.
2. Limit your imports from the Zarr-Python package. Most of the primary API ``zarr.*`` will be compatible in 3.0. However, the following breaking API changes are planned:
jhamman marked this conversation as resolved.
Show resolved Hide resolved

- ``numcodecs.*`` will no longer be available in ``zarr.*``. (Suggested action: transition to importing codecs from ``numcodecs`` directly.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- ``numcodecs.*`` will no longer be available in ``zarr.*``. (Suggested action: transition to importing codecs from ``numcodecs`` directly.)
- ``zarr.numcodecs.*`` will no longer be available. These imports can be replaced by importing ``numcodecs`` directly.

We should be doing more than "suggesting", we should be providing concrete fixes/code updates!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish it were that easy! Unfortunately, the top level namespace is littered with * imports

from zarr.codecs import *

from numcodecs import *

If there is a reasonable way for us to deprecate folks using these imports, I'm all ears.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if there isn't a reasonable way to deprecate the imports, as a minimum this migration guide should provide a complete list of all imports that are disappearing and how to replace them.

- The ``zarr.v3_api_available`` feature flag is being removed. In zarr-python v3 the v3 API is always available, so you shouldn't need to use this flag.
- The following internal modules are being removed or significantly changed. If your application relies on imports from any of the below modules, you will need to either a) modify your application to no longer rely on these imports or b) vendor the parts of the specific modules that you need.

- ``zarr.attrs``
- ``zarr.codecs``
- ``zarr.context``
- ``zarr.core``
- ``zarr.hierarchy``
- ``zarr.indexing``
- ``zarr.meta``
- ``zarr.meta_v1``
- ``zarr.storage``
- ``zarr.sync``
- ``zarr.types``
- ``zarr.util``
- ``zarr.n5``

jhamman marked this conversation as resolved.
Show resolved Hide resolved
3. Test that your package works with v3. You can start testing against version 3 now (pre-releases are being published to PyPI weekly).
4. Update the pin to zarr >=3

Continue using Zarr-Python 2
jhamman marked this conversation as resolved.
Show resolved Hide resolved
----------------------------

Zarr-Python 2.x is still available, though we recommend migrating to Zarr-Python 3 for its improvements and new features.
Security and bug fixes will be made to the 2.x series for at least 6 months following the first Zarr-Python 3 release.

If you need to use the latest Zarr-Python 2 release, you can install it with:

$ pip install "zarr==2.*"
jhamman marked this conversation as resolved.
Show resolved Hide resolved


Migration Guide
---------------

The following sections provide details on the most important changes in Zarr-Python 3.

Changes to the Array class
~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Disallow direct construction

Changes to the Group class
~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Disallow direct construction

Changes to the Store class
~~~~~~~~~~~~~~~~~~~~~~~~~~

Some of the biggest changes in Zarr-Python 3 are found in the `Store` class. The most notable changes to the Store API are:

1. Dropped the ``MutableMapping`` base class in favor of a custom abstract base class (``zarr.abc.store.Store``).
2. Switched to a primarily Async interface.

TODO

Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~

TODO