dll.version symbol not accessible starting with libMagPlus 4.7.0

[amotl]

Hi there,

when invoking python -m Magics selfcheck on our recent installation [1] on macOS, it slightly croaks like:

$ python -m Magics selfcheck
Found: Magics 'You are using an old version of magics ( < 4.0.0)'.
Library: /usr/local/opt/magics-4.9.3/lib/libMagPlus.dylib
Magics home: /usr/local/opt/magics-4.9.3
Your system is ready.

Other than this, the library appears to be fully functional, so we just wanted to bring this to your attention. Also, #37 and #38 are two outcomes from us approaching this issue, maybe you like them.

With kind regards,
Andreas.

[1] Python 3.9.7, MagPlus 4.9.3, Magics HEAD

[amotl]

We've investigated this a bit, and indeed, the dll.version symbol appears to be missing completely.

export MAGPLUS_HOME=/usr/local/opt/magics-4.9.3
$ python
Python 3.9.7 (default, Oct 13 2021, 06:44:56)
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

When invoking this code

import findlibs, ctypes
lib = findlibs.find("MagPlus")
dll = ctypes.CDLL(lib)
dll.version

it croaks like

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(0x7f7f59e05120, version): symbol not found

while that works

>>> dll.py_grib
<_FuncPtr object at 0x10c6cd340>

We are building MagPlus from source like outlined at [2]. Maybe we are doing something wrong?

[2] https://github.com/earthobservations/gribmagic/blob/1b64033/gribmagic.mk#L31-L43

[amotl]

We can confirm that everything works flawlessly on Linux, at least with MagPlus version 4.5.3.

docker run -it --rm python:3.9-slim bash
apt-get update

MagPlus 4.5.3, Magics 1.5.5 (dist)

Install distribution packages only.

apt-get install --yes python3-magics++
Setting up python3-magics++ (2:1.5.5-1) ...

Invoke selfcheck.

export PYTHONPATH=/usr/lib/python3/dist-packages
python -m Magics selfcheck
Found: Magics 'Magics 4.5.3'.
Your system is ready.

Clean up.

apt-get remove --yes python3-magics++
unset PYTHONPATH

MagPlus 4.5.3, Magics 1.5.6 (pip)

Install distribution LibMagPlus and Magics binding using pip.

apt-get install --yes libmagplus3v5
pip install Magics

Invoke selfcheck.

python -m Magics selfcheck
Found: Magics 'Magics 4.5.3'.
Your system is ready.

Clean up.

pip uninstall --yes Magics

MagPlus 4.5.3, Magics HEAD

Install the HEAD version.

apt-get install --yes git
git clone https://github.com/ecmwf/magics-python
pip install --editable=.

Invoke selfcheck.

python -m Magics selfcheck
Found: Magics 'Magics 4.5.3'.
Library: libMagPlus.so.3
Magics home: /usr
Your system is ready.

[amotl]

Oh. After building MagPlus 4.9.3 on Linux, we are observing the same problem.

docker run -it --rm python:3.9-slim bash

Install MagPlus to /usr/local/opt/magics-4.9.3.

apt-get update
apt-get install --yes build-essential cmake libeccodes-dev libeccodes-tools libproj-dev libexpat-dev wget
ln -s /usr/lib/x86_64-linux-gnu/libopenjp2.so.7 /usr/lib/x86_64-linux-gnu/libopenjp2.so
wget https://raw.githubusercontent.com/earthobservations/gribmagic/1b64033/Makefile
wget https://raw.githubusercontent.com/earthobservations/gribmagic/1b64033/gribmagic.mk
make magics-install

Install Magics.

pip install Magics

Invoke selfcheck.

export MAGPLUS_HOME=/usr/local/opt/magics-4.9.3
python -m Magics selfcheck
Found: Magics 'You are using an old version of magics ( < 4.0.0)'.
Your system is ready.

[amotl]

After some iterations over the available source packages on the official releases page, we can confirm it stopped working with Magics 4.7.0, available from the snapshot releases page.

export MAGPLUS_HOME=/usr/local/opt/magics-4.5.3/
python -m Magics selfcheck
Found: Magics 'Magics 4.5.3'.
Your system is ready.

export MAGPLUS_HOME=/usr/local/opt/magics-4.6.0/
python -m Magics selfcheck
Found: Magics 'Magics 4.6.0'.
Your system is ready.

export MAGPLUS_HOME=/usr/local/opt/magics-4.7.0/
root@dcac37fcca72:/# python -m Magics selfcheck
Found: Magics 'You are using an old version of magics ( < 4.0.0)'.
Your system is ready.

export MAGPLUS_HOME=/usr/local/opt/magics-4.8.0/
root@dcac37fcca72:/# python -m Magics selfcheck
Found: Magics 'You are using an old version of magics ( < 4.0.0)'.
Your system is ready.

[amotl]

Can we humbly ask you to carry this observation [1] forward to the appropriate authors and maintainers of the main Magics software package? Maybe someone reading here is already better accustomed with reporting a bug or issue than us and would like to do it?

[1] That the version symbol of libMagPlus seems to have gone missing starting with version 4.7.0.

[amotl]

Hi again,

we just submitted a patch at ecmwf/magics#47. With this, selfcheck works flawlessly again.

$ python -m Magics selfcheck
Found: Magics 'Magics 4.9.3'.
Your system is ready.

With kind regards,
Andreas.