Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update programming environment: spack support, cmake preset, gpu and external libs #2285

Open
wants to merge 68 commits into
base: develop
Choose a base branch
from

Conversation

prudhomm
Copy link
Member

@prudhomm prudhomm commented Sep 15, 2024

  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes?
  • Have you successfully run the Feel++ testsuite with your changes locally?
  • Have you written Doxygen comments in your contribution ?

update cmake preset using jsonnet for systematic generation #2283
add support for spack #2016

detect spack environment in cmake
add support for spack #2016
list spack versions
add spack:opempi in feelpp-core to start with
@prudhomm prudhomm added env: cmake build environment cmake 📦 spack labels Sep 15, 2024
@prudhomm prudhomm added this to the v111 milestone Sep 15, 2024
@prudhomm prudhomm self-assigned this Sep 15, 2024
@prudhomm prudhomm linked an issue Sep 15, 2024 that may be closed by this pull request
add spack environment to the main CI #2284

skip toolboxes skip testsuite skip mor skip python
add spack environment to the main CI #2284

skip toolboxes skip mor skip python skip testsuite
/cc @thomas-saigre

update presets for install prefix

add support for spack #2016
and update glog and gflags #2294

we have now a custom mpi logger for glog

[ci skip]
…dling

print for all ranks if severity is ERROR or FATAL
use log_option to print to stdout and sterr

use glog/gflags from spack and update support #2299
update glog and gflags #2294

[ci skip]
use glog/gflags from spack and update support #2299
update glog and gflags #2294

[ci skip]
thanks to warnoff/warnon headers, code is more readable
@prudhomm prudhomm changed the title resolves 2284: add spack environment to the main ci update programming environment: spack support, cmake preset, gpu and external libs Nov 1, 2024
add support for spack #2016
spackified
- nlopt, ipopt and fix tests/codes
- tabulate and indicators
- EigenRand

add support for spack #2016

[ci skip]
@prudhomm
Copy link
Member Author

prudhomm commented Nov 13, 2024

here are the results of the testsuite 923 tests on the spack env feelpp-openmpi4:


94% tests passed, 52 tests failed out of 923

Label Time Summary:
cfpdes               =   7.09 sec*proc (2 tests)
heat                 = 107.27 sec*proc (5 tests)
testalg              =  42.91 sec*proc (26 tests)
testcore             =  78.00 sec*proc (62 tests)
testdiscr            = 1822.89 sec*proc (112 tests)
testevents           =   4.00 sec*proc (2 tests)
testfilters          =  90.46 sec*proc (20 tests)
testfit              =   8.24 sec*proc (4 tests)
testintegration      =  42.67 sec*proc (16 tests)
testinterpolation    = 372.26 sec*proc (25 tests)
testleaks            =  28.20 sec*proc (16 tests)
testls               =   5.05 sec*proc (2 tests)
testmaterial         =   0.71 sec*proc (2 tests)
testmath             =   0.68 sec*proc (2 tests)
testmesh             = 122.16 sec*proc (38 tests)
testmodels           =   4.28 sec*proc (2 tests)
testmor              = 113.20 sec*proc (25 tests)
testopt              =   4.86 sec*proc (3 tests)
testpde              =  15.76 sec*proc (6 tests)
testpoly             =  57.89 sec*proc (20 tests)
testpython           =   9.79 sec*proc (6 tests)
testts               =  12.64 sec*proc (7 tests)
testvf               = 152.90 sec*proc (44 tests)
testviewfactor       = 102.48 sec*proc (2 tests)

Total Test time (real) = 6111.75 sec

The following tests did not run:
        743 - feelpp_test_hdiv-np-1 (Skipped)
        842 - feelpp_test_bdf3-np-1 (Skipped)

The following tests FAILED:
        465 - feelpp_qs_python-alg (Failed)
        466 - feelpp_qs_python-mpi-alg (Failed)
        467 - feelpp_qs_python-core (Failed)
        468 - feelpp_qs_python-mpi-core (Failed)
        469 - feelpp_qs_python-discr (Failed)
        470 - feelpp_qs_python-mpi-discr (Failed)
        471 - feelpp_qs_python-exporter (Failed)
        472 - feelpp_qs_python-mpi-exporter (Failed)
        473 - feelpp_qs_python-integrate (Failed)
        474 - feelpp_qs_python-mpi-integrate (Failed)
        475 - feelpp_qs_python-measure (Failed)
        476 - feelpp_qs_python-mpi-measure (Failed)
        477 - feelpp_qs_python-mesh (Failed)
        478 - feelpp_qs_python-mpi-mesh (Failed)
        479 - feelpp_qs_python-meshmover (Failed)
        480 - feelpp_qs_python-mpi-meshmover (Failed)
        481 - feelpp_qs_python-operators (Failed)
        482 - feelpp_qs_python-mpi-operators (Failed)
        483 - feelpp_qs_python-remesh (Failed)
        484 - feelpp_qs_python-mpi-remesh (Failed)
        485 - feelpp_qs_python-remotedata (Failed)
        486 - feelpp_qs_python-mpi-remotedata (Failed)
        487 - feelpp_qs_python-timing (Failed)
        488 - feelpp_qs_python-mpi-timing (Failed)
        489 - feelpp_toolbox_python-tests-fluid (Failed)
        490 - feelpp_toolbox_python-tests-mpi-fluid (Failed)
        491 - feelpp_toolbox_python-tests-cfpdes (Failed)
        492 - feelpp_toolbox_python-tests-mpi-cfpdes (Failed)
        493 - feelpp_toolbox_python-tests-heat (Failed)
        494 - feelpp_toolbox_python-tests-mpi-heat (Failed)
        495 - feelpp_toolbox_python-tests-electric (Failed)
        496 - feelpp_toolbox_python-tests-mpi-electric (Failed)
        497 - feelpp_toolbox_python-tests-thermoelectric (Failed)
        498 - feelpp_toolbox_python-tests-mpi-thermoelectric (Failed)
        499 - feelpp_toolbox_python-tests-heatfluid (Failed)
        500 - feelpp_toolbox_python-tests-mpi-heatfluid (Failed)
        501 - feelpp_toolbox_python-tests-interpolation (Failed)
        502 - feelpp_toolbox_python-tests-mpi-interpolation (Failed)
        503 - feelpp_mor_python-tests-mor (Failed)
        504 - feelpp_mor_python-tests-mpi-mor (Failed)
        505 - feelpp_mor_python-tests-nirb (Failed)
        506 - feelpp_mor_python-tests-mpi-nirb (Failed)
        661 - feelpp_test_importergmsh-np-6 (Failed)
        662 - feelpp_test_importergmsh-np-1 (Failed)
        703 - feelpp_test_interpolation_RT-np-1 (Failed)
        704 - feelpp_test_interpolation_Nedelec-np-1 (Failed)
        705 - feelpp_test_interpolation_Nedelec3d-np-1 (Timeout)
        722 - feelpp_test_integration_relatedmesh-np-6 (Failed)
        744 - feelpp_test_hdiv_oneelt-np-1 (Failed)
        745 - feelpp_test_hdiv3D_oneelt-np-1 (Timeout)
        746 - feelpp_test_hcurl_oneelt-np-1 (Failed)
        752 - feelpp_test_hcurl3D_oneelt-np-1 (Timeout)
Errors while running CTest

the python failures should not be a problem and easily fixed

@prudhomm
Copy link
Member Author

for the python tests, we actually need to do a make install and then set PYTHONPATH to the install python directory. Once done it is all good :

> cmake --build --preset <preset> -t install/fast
> export PYTHONPATH=<snip>/install/<preset>/lib/python3.11/site-packages/
> ctest  --preset <preset>  -R _python-

here are the results for preset feelpp-clang-cpp20-spack-cpu-release:

95% tests passed, 2 tests failed out of 42

Total Test time (real) = 596.18 sec

The following tests FAILED:
        505 - feelpp_mor_python-tests-nirb (Failed)
        506 - feelpp_mor_python-tests-mpi-nirb (Failed)
Errors while running CTest

the last 2 failures are due to missing tqdm python package in environment

@prudhomm
Copy link
Member Author

@thomas-saigre py-tqdm is now installed in our environment for nirb

prudhomm and others added 19 commits November 13, 2024 18:05
- add distro to support eg ubuntu or homebrew
- add analysisTool eg eztrace, scorep ...
- Introduced exreader.hpp for default prototype table.
- Updated CMakeLists.txt to include excompiler files.
- Enhanced ginacmatrix.hpp with utility to convert vectors to GiNaC lists.
- Modified environment.cpp to utilize Feel's cleanup_ex function.
- Added ginac-extern-math.h for external math function declarations.
…atus messaging

don't forget to install before testing, otherwise the pythn tests and some tests in the testsuite will fail

various minor fixes for excompiler and tests
@prudhomm
Copy link
Member Author

prudhomm commented Jan 6, 2025

here is the state of the CI with spack:

https://github.com/feelpp/feelpp/actions/runs/12623421589/job/35172411071?pr=2285#step:5:2388

925 tests passed 🎆

The next steps are:

  • fix Ubuntu and Debian builds
  • add docker support
  • improve cpack support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment