From e37ea2f2066d1c52c258e464a3224a07d71f8aa0 Mon Sep 17 00:00:00 2001 From: Francesc Alted Date: Thu, 29 Aug 2024 13:57:38 +0200 Subject: [PATCH] Add a reference to the notebook --- posts/ndim-reductions.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/posts/ndim-reductions.rst b/posts/ndim-reductions.rst index b756f8f..510c75c 100644 --- a/posts/ndim-reductions.rst +++ b/posts/ndim-reductions.rst @@ -155,8 +155,8 @@ Conclusion ---------- Understanding the balance between space savings and the additional time required to process the data is important. Testing different compression settings can help finding the method that offers the best trade-off between reduced size and processing time. The fact that Blosc2 automatically chooses the chunk shape, makes it easy for the user to get a decently good performance, without having to worry about the details of the CPU cache. In addition, as we have shown, we can fine tune the chunk shape in case the default one does not fit our needs (e.g. we need more uniform performance along all axes). -Moreover, it is also possible to use reductions even for very large arrays that are stored on disk. This opens the door to a wide range of possibilities for data analysis and science, allowing for efficient reductions on large datasets that are compressed on-disk and with minimal memory usage. We will explore this in a forthcoming blog. +Besides the sum() reduction exercised here, Blosc2 supports a fair range of reduction operators (mean, std, min, max, all, any, etc.), and you are invited to `explore them `_. Moreover, it is also possible to use reductions even for very large arrays that are stored on disk. This opens the door to a wide range of possibilities for data analysis and science, allowing for efficient reductions on large datasets that are compressed on-disk and with minimal memory usage. We will explore this in a forthcoming blog. -Besides the sum() reduction exercised here, Blosc2 supports a fair range of reduction operators (mean, std, min, max, all, any, etc.), and you are invited to `explore them `_. +Finally, you can find the code for this blog on a `notebook in the Blosc2 repository `_. Feel free to experiment with different parameters and share your results with us! We would like to thank `ironArray `_ for supporting the development of the computing capabilities of Blosc2. Then, to NumFOCUS for recently providing a small grant that is helping us to improve the documentation for the project. Last but not least, we would like to thank the Blosc community for providing so many valuable insights and feedback that have helped us to improve the performance and usability of Blosc2.