On installing Julia on a Cray

[marcodelapierre]

Hi @dipietrantonio @pelahi ,
I am reporting on some attempts I have done today to install [email protected] with the latest Spack develop.

On a VM with Ubuntu 22.04, it succeeds.

On a Cray, it fails. Note here I have used the develop branch in conjunction with the yaml config files from this repo.
There are two distinct issues:

1. the edited llvm recipe we have gives issues with the variants required by Spack; to try and progress, I have reverted to the llvm recipe from the develop branch
2. when 1. is applied, an error arises early during the build of the julia package, concerning pkg-config, with multiple repeats of the following:

     >> 10    Error invoking pkg-config!
    11    Package dsmml was not found in the pkg-config search path.
    12    Perhaps you should add the directory containing `dsmml.pc'
    13    to the PKG_CONFIG_PATH environment variable
    14    No package 'dsmml' found
    15    Package cray-xpmem was not found in the pkg-config search path.
    16    Perhaps you should add the directory containing `cray-xpmem.pc'
    17    to the PKG_CONFIG_PATH environment variable
    18    No package 'cray-xpmem' found
    19    Package dsmml was not found in the pkg-config search path.
    20    Perhaps you should add the directory containing `dsmml.pc'
    21    to the PKG_CONFIG_PATH environment variable
    22    No package 'dsmml' found
   

   I have tried to fix it in maaany ways, including hard coding the required paths in PKG_CONFIG_PATH in the compilers.yaml and even in the julia recipe itself:

       def setup_build_environment(self, env):
       env.prepend_path("PKG_CONFIG_PATH", "/opt/cray/xpmem/2.4.4-2.3_13.8__gff0e1d9.shasta/lib64/pkgconfig")
       env.prepend_path("PKG_CONFIG_PATH", "/opt/cray/pe/dsmml/0.2.2/dsmml/lib/pkgconfig")
   

   I believe these are brought in somehow from the Cray PE environment, as they appear nowhere in the julia recipe.

I don't have a solution right now, but wanted to report here on these aspects.

[dipietrantonio]

Interesting the issue with llvm, they might have introduced breaking changes in the develop branch. We can compile it with spack/0.19.0. I will try to install Julia myself and let you know if I discover something useful.

[dipietrantonio]

Hi @marcodelapierre I observed this issue on other occasions too last month. You are right, it is not a Julia-specific error, rather something related to the environment Spack was run in. I honestly can't recall what it was, but I remember to have found a solution. Too many things I worked on. I will run the Julia install again to confirm this problem is not there anymore.

[dipietrantonio]

I tried installing Julia again, this time I get the following error

==> Error: NoLibrariesError: Unable to recursively locate nghttp2 libraries in /usr

/scratch/pawsey0001/cdipietrantonio/test-spack-env/2024.05-upstreams/spack/var/spack/repos/builtin/packages/julia/package.py:263, in setup_build_environment:
        260        if self.spec.satisfies("@1.7.0:"):
        261            pkgs.append("libblastrampoline")
        262        for pkg in pkgs:
  >>    263            for dir in self.spec[pkg].libs.directories:
        264                env.prepend_path(linker_var, dir)
        265        for dir in self.spec["zlib-api"].libs.directories:
        266            env.prepend_path(linker_var, dir)