Library not loaded: @rpath/libarchive.13.dylib

[maresb]

Issue

I'm seeing a sporadic failure in the conda-lock CI when running with micromamba on osx-arm64. I would estimate the frequency of failure to be around 10%.

An example of a failing run is here.

The relevant output is:

+ micromamba run -n gdal-test-mm-lock gdalinfo --version
+ local cmd=run
+ case "${cmd}" in
+ __mamba_exe run -n gdal-test-mm-lock gdalinfo --version
+ /Users/runner/micromamba-bin/micromamba run -n gdal-test-mm-lock gdalinfo --version
dyld[2884]: Library not loaded: @rpath/libarchive.13.dylib
  Referenced from: <AF2F33F0-47DF-3AD6-B501-D354725FEBD8> /Users/runner/micromamba/pkgs/libgdal-core-3.9.2-hfd0b032_7/lib/libgdal.35.3.9.2.dylib
  Reason: tried: '/Users/runner/micromamba/pkgs/libgdal-core-3.9.2-hfd0b032_7/lib/libarchive.13.dylib' (no such file), '/Users/runner/micromamba/pkgs/libgdal-core-3.9.2-hfd0b032_7/bin/../lib/libarchive.13.dylib' (no such file), '/Users/runner/micromamba/pkgs/libgdal-core-3.9.2-hfd0b032_7/bin/../lib/libarchive.13.dylib' (no such file), '/usr/local/lib/libarchive.13.dylib' (no such file), '/usr/lib/libarchive.13.dylib' (no such file, not in dyld cache)
Error: Process completed with exit code 134.

I originally reported this error in mamba-org/mamba#1826 (comment). I got a response from @jjerphan in the second half of mamba-org/mamba#1826 (comment). In particular, he wrote,

@maresb: the problem that you describe is rather due to libgdal-core which improperly relatively searches for libarchive's the shared library. I would recommend opening an issue on https://github.com/conda-forge/gdal-feedstock.

I don't understand much about how shared libraries work, but my interpretation is that libgdal-core should not be searching for libarchive, but should instead directly find the one provided by conda-forge.

Installed packages

Expand for environment creation logs

+ __mamba_exe create -y -n gdal-test-mm-lock -f lockfiles/conda-lock.yml
+ /Users/runner/micromamba-bin/micromamba create -y -n gdal-test-mm-lock -f lockfiles/conda-lock.yml
Empty environment created at prefix: /Users/runner/micromamba/envs/gdal-test-mm-lock
Transaction

  Prefix: /Users/runner/micromamba/envs/gdal-test-mm-lock



  No specs added or removed.

  Package                 Version  Build                 Channel         Size
───────────────────────────────────────────────────────────────────────────────
  Install:
───────────────────────────────────────────────────────────────────────────────

  + bzip2                   1.0.8  h99b78c6_7            conda-forge         
  + c-ares                 1.33.1  hd74edd7_0            conda-forge         
  + ca-certificates     2024.8.30  hf0a4a13_0            conda-forge         
  + giflib                  5.2.2  h93a5062_0            conda-forge         
  + icu                      75.1  hfee45f7_0            conda-forge         
  + json-c                   0.18  he4178ee_0            conda-forge         
  + libcxx                 19.1.1  ha82da77_0            conda-forge         
  + libdeflate               1.22  hd74edd7_0            conda-forge         
  + libev                    4.33  h93a5062_2            conda-forge         
  + libexpat                2.6.3  hf9b8971_0            conda-forge         
  + libffi                  3.4.2  h3422bc3_5            conda-forge         
  + libiconv                 1.17  h0d3ecfb_2            conda-forge         
  + libjpeg-turbo           3.0.0  hb547adb_1            conda-forge         
  + libwebp-base            1.4.0  h93a5062_0            conda-forge         
  + libzlib                 1.3.1  h8359307_2            conda-forge         
  + llvm-openmp            19.1.0  hba312e6_0            conda-forge         
  + lzo                      2.10  h93a5062_1001         conda-forge         
  + ncurses                   6.5  h7bae524_1            conda-forge         
  + python_abi                3.8  5_cp38                conda-forge         
  + xz                      5.2.6  h57fd34a_0            conda-forge         
  + openssl                 3.3.2  h8359307_0            conda-forge         
  + xerces-c                3.2.5  h92fc2f4_2            conda-forge         
  + uriparser               0.9.8  h00cdb27_0            conda-forge         
  + snappy                  1.2.1  hd02b534_0            conda-forge         
  + lz4-c                   1.9.4  hb7217d7_0            conda-forge         
  + lerc                    4.0.0  h9a09cb3_0            conda-forge         
  + geos                   3.13.0  hf9b8971_0            conda-forge         
  + zstd                    1.5.6  hb46c0d2_0            conda-forge         
  + zlib                    1.3.1  h8359307_2            conda-forge         
  + tk                     8.6.13  h5083fa2_1            conda-forge         
  + pcre2                   10.44  h297a79d_2            conda-forge         
  + libsqlite              3.46.1  hc14010f_0            conda-forge         
  + libpng                 1.6.44  hc14010f_0            conda-forge         
  + libgfortran5           13.2.0  hf226fd6_3            conda-forge         
  + readline                  8.2  h92ec313_1            conda-forge         
  + libedit          3.1.20191231  hc8eb9b7_2            conda-forge         
  + libxml2                2.12.7  h01dff8b_4            conda-forge         
  + libssh2                1.11.0  h7a5bd25_0            conda-forge         
  + libnghttp2             1.58.0  ha4dd798_1            conda-forge         
  + libkml                  1.3.0  he250239_1021         conda-forge         
  + librttopo               1.1.0  ha2cf0f4_17           conda-forge         
  + minizip                 4.0.7  h27ee973_0            conda-forge         
  + libtiff                 4.7.0  hfce79cd_1            conda-forge         
  + blosc                  1.21.6  h5499902_0            conda-forge         
  + libgfortran             5.0.0  13_2_0_hd922786_3     conda-forge         
  + sqlite                 3.46.1  h3b4c4e4_0            conda-forge         
  + python                 3.8.20  h7d35d02_2_cpython    conda-forge         
  + krb5                   1.21.3  h237132a_0            conda-forge         
  + libarchive              3.7.4  h83d404f_0            conda-forge         
  + freexl                  2.0.0  hfbad9fb_0            conda-forge         
  + libopenblas            0.3.27  openmp_h517c56d_1     conda-forge         
  + wheel                  0.44.0  pyhd8ed1ab_0          conda-forge         
  + setuptools             75.1.0  pyhd8ed1ab_0          conda-forge         
  + libcurl                8.10.1  h13a7ad3_0            conda-forge         
  + libblas                 3.9.0  24_osxarm64_openblas  conda-forge         
  + pip                      24.2  pyh8b19718_1          conda-forge         
  + proj                    9.5.0  h61a8e3e_0            conda-forge         
  + liblapack               3.9.0  24_osxarm64_openblas  conda-forge         
  + libcblas                3.9.0  24_osxarm64_openblas  conda-forge         
  + libspatialite           5.1.0  hffd3212_11           conda-forge         
  + geotiff                 1.7.3  h82bf549_3            conda-forge         
  + numpy                  1.24.4  py38ha84db1f_0        conda-forge         
  + libgdal-core            3.9.2  hfd0b032_7            conda-forge         
  + gdal                    3.9.2  py38hf88e1e0_0        conda-forge         

  Summary:

  Install: 64 packages

  Total download: 0 B

───────────────────────────────────────────────────────────────────────────────



Transaction starting

Transaction finished

To activate this environment, use:

    micromamba activate gdal-test-mm-lock

Or to execute a single command in this environment, use:

    micromamba run -n gdal-test-mm-lock mycommand


Installing pip --no-deps packages: toolz @ https://files.pythonhosted.org/packages/03/98/eb27cc78ad3af8e302c9d8ff4977f5026676e130d28dd7578132a457170c/toolz-1.0.0-py3-none-any.whl#sha256=292c8f1c4e7516bf9086f8850935c799a874039c8bcf959d47b600e4c44a6236
Collecting toolz@ https://files.pythonhosted.org/packages/03/98/eb27cc78ad3af8e302c9d8ff4977f5026676e130d28dd7578132a457170c/toolz-1.0.0-py3-none-any.whl#sha256=292c8f1c4e7516bf9086f8850935c799a874039c8bcf959d47b600e4c44a6236 (from -r /Users/runner/work/conda-lock/conda-lock/lockfiles/mambaf3f914kJcNc (line 1))
  Downloading toolz-1.0.0-py3-none-any.whl (56 kB)
Installing collected packages: toolz
Successfully installed toolz-1.0.0