Provide an API using numpy arrays in python

[martinResearch]

When working with OpenEXR files in Python, I've noticed that the process can feel somewhat verbose compared to using APIs like OpenCV or ImageIO for writing and reading EXR files.

Unfortunately, both ImageIO and OpenCV come with their own limitations when dealing with EXR files:

• opencv-python allows saving and reading EXR images, but it requires setting up an environment variable, OPENCV_IO_ENABLE_OPENEXR, before the first import of OpenCV on Windows. This violates PEP8 import rules and can be tricky to ensure if OpenCV is imported in other modules. You can find more details in issue Enabling OpenEXR after python module import. See issue Enabling OpenEXR after python module import opencv/opencv#24470.

• imageio relies on either freeimage or OpenCV under the hood to load and read EXR images. FreeImage doesn't have a permissive license and thus is not installed by default with ImageIO and cannot be installed with pip (see support freeimage plugin instalaltion using pip install imageio[freeimage] imageio/imageio#809). This requires a manual step to install, which deviates from the traditional Python environment setup process using pip only. Additionally, it modifies the system by adding the FreeImage DLL to the system path, making it visible to all Python environments and potentially altering the behavior of other Python environments that use ImageIO on the machine. Similarly, using OpenCV under the hood also has limitations and requires setting up an environment variable, as mentioned above.

For these reasons, I created a small Python package called openexr-numpy that adds a layer on top of OpenEXR to simplify reading or writing EXR files to or from NumPy arrays.

Would it be possible or sensible to provide something similar to this as part of the official OpenEXR Python package? Doing so would likely enhance adoption and increase trust in this functionality being maintained in the long term.

[lgritz]

Use OpenImageIO (not to be confused with "imageio"). That's what almost everybody in VFX does to read image files from Python. It does read right into Numpy arrays.

Unfortunately, there isn't yet a pip install for it, sorry.

[martinResearch]

Although it seems available on anaconda.org (OpenImageIO) I am afraid not having it on pypi.org is a no-go for my use-case and seems quite limiting. Do you know if publishing to pypi.org Is it in the roadmap? and if not I would be curious about the potential blockers.

[lgritz]

It is on the roadmap, yes, and it's even being worked on. I'm not sure what specifically is blocking it at the moment.

AcademySoftwareFoundation/OpenImageIO#4011

[martinResearch]

should I understand that the response to "Would it be possible or sensible to provide something similar to this as part of the official OpenEXR Python package" is no, or could that still be considered despite the existence of openimageio ?
If the recommended solution is to use openimageio instead of the OpenEXR package, I believe it would improve the discoverability of openimageio if we can mention it on the page OpenEXR pypi page.

[lgritz]

It's sensible and we would like this, it's mainly a question of needing somebody to step forward to do the work.

One reason that there hasn't been a rush to get it done is because most people who need it (and probably all the people who would be able and inclined to do the work) are already using the OpenImageIO Python bindings to read openexr files.

But if somebody made a PR for OpenEXR that did a long-needed overhaul of the python bindings, convert to pybind11, and add Numpy compatibility, it would very happily be accepted.

[martinResearch]

I tried to install openimageio from anaconda.org but it looks like it is not available for windows :(.

[lgritz]

How about conan? Or VcPkg?

[martinResearch]

Conan and VcPkg seems to be mostly C and C++ package managers, would I then need to compile the python binding? If so that sounds quite involved...and not easily applicable to many python environment setups (I think about setup for continuous integration for example). Not having a package on Pypi is a very strong blocker for adoption in the larger python community in my opinion.

" OpenEXR that did a long-needed overhaul of the python bindings, convert to pybind11, and add NumPy compatibility, it would very happily be accepted." I will probably not volunteer for this as I am not sure how much work that would be and I am not proficient at pybin11. Does it have to be done in the C++ side ? Could we imagine instead keeping the curent C++ binding as it is and simply add a python layer similar to what I wrote? Also could adding NumPy as a dependency be considered as a problem/regression for some of the current user of the OpenEXR python package? in which case making the NumPy dependency optional might be desired, using "pip install openexr[numpy]" for example, which might be easier to setup by having the NumPy numpy layer on the python side

[lgritz]

Conan and VcPkg seems to be mostly C and C++ package managers, would I then need to compile the python binding?

The python binding is a wrapper around a C++ library, so yes, you need to build the library in order to have a meaningful Python wrapper for it. When you build the library, it builds the python bindings for it, too.

Not having a package on Pypi is a very strong blocker

We know; that's why it's on the roadmap.

Does it have to be done in the C++ side ?
Could we imagine instead keeping the current C++ binding as it is and simply add a python layer similar to what I wrote?

I'm not sure what you mean.

Like with OpenImageIO, OpenEXR is at its core a C/C++ library.

The C++ bindings are the existing C++ API that lets C++ programs use the library. We have no intention to change that.

The Python binding lets Python programs use the library, but it's still implemented in C/C++.

For OpenEXR, the current Python binding wasn't written by the OpenEXR project team, nor was it originally part of the project at all. It was written by somebody else, spent years in disrepair and being minimally or not at all maintained and not keeping up with changes in the C++ bindings (that is, the real OpenEXR). Recently, it was gifted to the OpenEXR project. So it exists and is part of the main project now, which is a big step forward to a better future. But it's still far behind, not well understood by anybody on the team, and in dire need of (in my opinion) a total rewrite. If we did that rewrite using Pybind11, support for numpy would more or less come along for free (as it does with OpenImageIO's python bindings, which are based on pybind11).

[cary-ilm]

As @lgritz says, the "official" OpenEXR python bindings ("pip install openexr") have only recently been adopted by the OpenEXR project and made official, the API was not our design and to be truly functional it needs an overhaul. I'm cautiously optimistic that it's not that much work, the basics should be straightforward, but we don't have an ETA.

[martinResearch]

for the reference, in addition to openexr-numpy, there is also pyexr that provides a numpy interface by wrapping OpenEXR