xbatcher: Batch Generation from Xarray Datasets

Xbatcher is a small library for iterating Xarray DataArrays and Datasets in batches. The goal is to make it easy to feed Xarray objects to machine learning libraries such as PyTorch or TensorFlow. View the for more info.

Installation

Xbatcher can be installed from PyPI as:

python -m pip install xbatcher

Or via Conda as:

conda install -c conda-forge xbatcher

Or from source as:

python -m pip install git+https://github.com/xarray-contrib/xbatcher.git

Note

The required dependencies installed with Xbatcher are Xarray, Dask, and NumPy. You will need to separately install TensorFlow or PyTorch to use those data loaders or Xarray accessors. Review the installation instructions for more details.

Documentation

Documentation is hosted on ReadTheDocs: https://xbatcher.readthedocs.org

License

Apache License 2.0, see LICENSE file.

Acknowledgements

This work was funded in part by:

NASA ACCESS19-0049: Pangeo ML: Open Source Tools and Pipelines for Scalable Machine Learning Using NASA Earth Observation Data

This work was motivated by many conversations in the Pangeo community and Pangeo ML working group