Crash with h5py3.0rc1 on macos

[t20100]

Tested on macos with python3.7, h5py and hdf5plugin installed from wheels.

Some filters crash with h5py 3.0rc1:

import h5py, hdf5plugin, numpy 
f = h5py.File("test.h5", "a")

f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='uint8'), **hdf5plugin.Blosc())                                                                            
Python(1757,0x7fffafcbe3c0) malloc: *** error for object 0x7f8404b04c30: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
...
f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='uint8'), **hdf5plugin.Bitshuffle())                                                                       
Python(1742,0x7fffafcbe3c0) malloc: *** error for object 0x7fbff9424430: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
...
f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='float32'), **hdf5plugin.Zfp())                                                                            
Python(1772,0x7fffafcbe3c0) malloc: *** error for object 0x7f8befc88e30: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

LZ4 and FciDecomp works for fine under same conditions.

I cannot reproduce the same with a python3.8 virtualenv on ubuntu20.04.

[vasole]

I suspect a bad build. This is what I find on actual MacOS El Capitan: Symbol not found: ____chkstk_darwin

Vs-MacBook-Pro:test sole$ ls
bitshuffle.h5	ficdecomp.h5	test.py		zfp_052.h5
blosc.h5	lz4.h5		zfp_050.h5	zfp_054.h5
Vs-MacBook-Pro:test sole$ python3 test.py 
Traceback (most recent call last):
  File "test.py", line 34, in <module>
    import h5py
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/h5py/__init__.py", line 25, in <module>
    from . import _errors
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/h5py/_errors.cpython-38-darwin.so, 2): Symbol not found: ____chkstk_darwin
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/h5py/.dylibs/libsz.2.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/h5py/.dylibs/libsz.2.dylib

[t20100]

Symbol not found: ____chkstk_darwin issue is solved in h5py: h5py/h5py#1707

[vasole]

Yes with your wheel I get the same error as you had. Trying to build everything on an old Mac.

[vasole]

Still crashing with the same error with HDF5 and h5py built on the same mac.

[t20100]

Similar issue on h5py with lzf filter: h5py/h5py#1711
Behavior is the same here, commenting the free in the blosc filter avoids the crash.

[t20100]

Related question on HDF5 forum: https://forum.hdfgroup.org/t/issue-with-compression-filters-and-hdf5-1-12-0-on-macos/7640

[t20100]

It works when building current h5py master with libHDF5 1.10.6, so it seems to be due to some changes in HDF5 1.12.0 regarding memory management.

[t20100]

Related PR on hdf5-blosc: Blosc/hdf5-blosc#21

[t20100]

Using H5free_memory instead of free looks to fix the issue.

[vasole]

Does it need to replace all the malloc and free?

[t20100]

Discussion is going on here: https://forum.hdfgroup.org/t/issue-with-compression-filters-and-hdf5-1-12-0-on-macos/7640

[vasole]

I can confirm the problem disappears setting configure flags.

HDFGroup/hdf5#53

Not an hdf5plugin issue.