Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pumi test fails on perlmutter #51

Open
wspear opened this issue Dec 5, 2022 · 3 comments
Open

Pumi test fails on perlmutter #51

wspear opened this issue Dec 5, 2022 · 3 comments

Comments

@wspear
Copy link
Collaborator

wspear commented Dec 5, 2022

@cwsmith

The spack test for this variant of pumi installed on perlmutter by e4s 22.11:

-- linux-sles15-zen3 / [email protected] -------------------------------
[email protected]~fortran~int64~ipo~shared+simmodsuite_version_check~testing~zoltan build_system=cmake build_type=RelWithDebInfo simmodsuite=none

Fails with the following console output:

==> Error: TestFailure: 2 tests failed.


Command exited with status 127:
    './uniform' '../testdata/pipe.dmg' '../testdata/pipe.smb' 'pipe_unif.smb'
./uniform: error while loading shared libraries: libfabric.so.1: cannot open shared object file: No such file or directory



1 error found in test log:
     1    ==> Testing package pumi-2.2.7-llomhac
     2    ==> [2022-11-21-12:25:41.390040] testing pumi uniform mesh refinement
     3    ==> [2022-11-21-12:25:41.390483] './uniform' '../testdata/pipe.dmg' '../testdata/pipe.smb' 'pipe_unif.smb'
     4    ./uniform: error while loading shared libraries: libfabric.so.1: cannot open shared object file: No such file or directory
     5    FAILED: Command exited with status 127:
     6        './uniform' '../testdata/pipe.dmg' '../testdata/pipe.smb' 'pipe_unif.smb'
  >> 7    ./uniform: error while loading shared libraries: libfabric.so.1: cannot open shared object file: No such file or directory
     8    
     9      File "/global/cfs/cdirs/m3896/shared/ParaTools/E4S/22.11/PrgEnv-gnu/spack/bin/spack", line 100, in <module>



Failed to find executable '/opt/cray/pe/mpich/8.1.17/ofi/gnu/9.1/bin/mpiexec'

/global/cfs/cdirs/m3896/shared/ParaTools/E4S/22.11/PrgEnv-gnu/spack/lib/spack/spack/package_base.py:2023, in run_test:
       2020                    # We're below the package context, so get context from
       2021                    # stack instead of from traceback.
       2022                    # The traceback is truncated here, so we can't use it to
  >>   2023                    # traverse the stack.
       2024                    m = "\n".join(spack.build_environment.get_package_context(tb))
       2025
       2026                exc = e  # e is deleted after this block


/global/cfs/cdirs/m3896/shared/ParaTools/E4S/22.11/PrgEnv-gnu/spack/lib/spack/spack/build_environment.py:1086, in _setup_pkg_and_run:
       1083        tb_string = traceback.format_exc()
       1084
       1085        # build up some context from the offending package so we can
  >>   1086        # show that, too.
       1087        package_context = get_package_context(tb)
       1088
       1089        logfile = None

See test log for details:
  /pscratch/sd/w/wspear/perlmutter/spack_user_cache/test/hyjemf5zrzgrnmfmzuddmljbsxwuyat4/pumi-2.2.7-llomhac-test-out.txt

==> Error: 1 test(s) in the suite failed.

--- ==> Spack test hyjemf5zrzgrnmfmzuddmljbsxwuyat4
==> Testing package pumi-2.2.7-llomhac
======================== 1 failed, 0 passed of 1 specs ========================= ---

@cwsmith
Copy link

cwsmith commented Dec 6, 2022

Hi @wspear.

Hmmm. I'll try to reproduce this.

Do the tests run on a front end node or a compute node?

@wspear
Copy link
Collaborator Author

wspear commented Dec 6, 2022

@cwsmith I run these on the compute nodes. I'm not sure how they behave on the login.

@wspear
Copy link
Collaborator Author

wspear commented Dec 14, 2022

@cwsmith I'm seeing similar behavior on Crusher:

==> Error: TestFailure: 2 tests failed.


Command exited with status -6:
    './uniform' '../testdata/pipe.dmg' '../testdata/pipe.smb' 'pipe_unif.smb'
gmi failed: could not open model file



1 error found in test log:
  >> 5    FAILED: Command exited with status -6:
     6        './uniform' '../testdata/pipe.dmg' '../testdata/pipe.smb' 'pipe_unif.smb'
     7    gmi failed: could not open model file
     8    
     9      File "/gpfs/alpine/csc439/world-shared/E4S/ParaTools/22.11/PrgEnv-gnu/spack/bin/spack", line 100, in <module>
     10       sys.exit(spack.main.main())
     11     File "/gpfs/alpine/csc439/world-shared/E4S/ParaTools/22.11/PrgEnv-gnu/spack/lib/spack/spack/main.py", line 10
          03, in main



Failed to find executable '/opt/cray/pe/mpich/8.1.17/ofi/gnu/9.1/bin/mpiexec'

/gpfs/alpine/csc439/world-shared/E4S/ParaTools/22.11/PrgEnv-gnu/spack/lib/spack/spack/package_base.py:2023, in run_test:
       2020                    # We're below the package context, so get context from
       2021                    # stack instead of from traceback.
       2022                    # The traceback is truncated here, so we can't use it to
  >>   2023                    # traverse the stack.
       2024                    m = "\n".join(spack.build_environment.get_package_context(tb))
       2025
       2026                exc = e  # e is deleted after this block


/gpfs/alpine/csc439/world-shared/E4S/ParaTools/22.11/PrgEnv-gnu/spack/lib/spack/spack/build_environment.py:1086, in _setup_pkg_and_run:
       1083        tb_string = traceback.format_exc()
       1084
       1085        # build up some context from the offending package so we can
  >>   1086        # show that, too.
       1087        package_context = get_package_context(tb)
       1088
       1089        logfile = None

See test log for details:
  /ccs/home/wspear/.spack/test/jo3wjmyxtqz4zrufu4wv2pxbbvskugeo/pumi-2.2.7-bh3g2dq-test-out.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants