Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pangea 4 docker build #276

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c9cc97a
add p4 docker files
AJ0526712 Jul 11, 2024
42b012e
fix spack
Bubusch Jul 18, 2024
ac4671e
use onetech images
Bubusch Jul 25, 2024
bb23d51
rename docker files and add doc
Bubusch Jul 25, 2024
11cc69f
rename pecan base docker file
Bubusch Jul 25, 2024
4c809ba
update doc
Bubusch Jul 25, 2024
37ebcaf
fix cray wrappers
Bubusch Jul 26, 2024
f7ce25a
fix hpcx and mkl
Bubusch Jul 26, 2024
048aa9c
remove extra hpcx env
Bubusch Jul 26, 2024
cac5385
use cray wrapper dir as ARG
Bubusch Jul 26, 2024
037a9c5
fix merge conflict
Bubusch Jul 26, 2024
a56fced
fix blt merge conflict
Bubusch Jul 26, 2024
5995cdc
fix missing runs on in workflow
Bubusch Jul 26, 2024
e684d42
add ninja install for geos
Bubusch Jul 29, 2024
c677276
Merge branch 'master' into feature/bubusch/p4-docker
AJ0526712 Nov 7, 2024
999568a
split p3 vs p4
AJ0526712 Nov 14, 2024
fd35c42
run P4 build on ubuntu latest
Bubusch Nov 14, 2024
daf5cd5
fix tpl docker file tag
Bubusch Nov 14, 2024
2e9e5c1
fix typo in p4 host config
Bubusch Nov 14, 2024
e1ae092
Merge branch 'master' into feature/bubusch/p4-docker
Bubusch Nov 15, 2024
b154d2b
set clang version to 17.0.6
Bubusch Nov 27, 2024
4046a51
no update for clang
Bubusch Nov 27, 2024
4904e2c
dnf update only once
Bubusch Nov 27, 2024
adb3585
try to fix caliper build
Bubusch Nov 27, 2024
e7cd018
copy fix from trilinos PR
Bubusch Dec 4, 2024
220a288
fix local p4 build
Bubusch Dec 6, 2024
a20f291
Merge branch 'master' into feature/bubusch/p4-docker
Bubusch Dec 17, 2024
15cc460
changed modules for pangea-3 configuration
sframba Jan 9, 2025
b5163c8
changed P3 dockerfile
sframba Jan 16, 2025
3a5c1a1
update docker pangea3 with gcc11
Bubusch Jan 22, 2025
f47b085
fix cmake version
Bubusch Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/docker_build_tpls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ jobs:

matrix:
include:
- name: Pangea 4 (redhat 8, gcc 12.1.0, hpcx 2.17.1, mkl 2023.2.0)
DOCKER_REPOSITORY: geosx/pangea4-gcc12.1-hpcxompi2.17.1-onemkl2023.2.0
DOCKERFILE: docker/TotalEnergies/tpl-pangea4.Dockerfile
DOCKER_ROOT_IMAGE: onetechssc/pangea4:gcc12.1-hpcxompi2.17.1-onemkl2023.2.0_v1.0
HOST_CONFIG: host-configs/TOTAL/pangea4-gcc12.1-hpcxompi2.17.1-onemkl2023.2.0.cmake
INSTALL_DIR_ROOT: /workrd/SCR/NUM/GEOS_environment/p4/install/tpl
RUNS_ON: ubuntu-latest
- name: Sherlock CPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, openblas 0.3.10, zlib 1.2.11)
DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-openblas0.3.10-zlib1.2.11
DOCKERFILE: docker/Stanford/Dockerfile
Expand Down
3 changes: 1 addition & 2 deletions docker/Stanford/Sherlock.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# This Dockerfile aims at reproducing (some part of) the SHERLOCK environment.
# Please see TotalEnergies/Pangea2.Dockerfile for fully commented version

#
# While loading module ompi4.1.2 on Sherlock, the following are aut-loaded:
# UCX/1.12.1
# LIBFABRIC/1.14.0
Expand Down
20 changes: 20 additions & 0 deletions docker/TotalEnergies/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# TotalEnergies Dockerfiles

## Overview

This directory contains several Dockerfiles used for building and deploying the TPLs.
Below is a table summarizing the purpose and key characteristics of each Dockerfile.

| Dockerfile | Description | Base Image | Key Components Installed |
|------------------------------------------|------------------------------------------------------------------------------|-------------------------------|------------------------------------------------------------------------------------------------------|
| `pecan-gcc-openmpi-mkl-cuda.Dockerfile` | Builds a toolchain with GCC, OpenMPI, CUDA, and Intel MKL for GEOS on Pecan. | `centos:7.7.1908` | GCC, OpenMPI, CUDA, Intel MKL, numactl-devel, ca-certificates, curl, Python3, zlib-devel |
| `tpl-pecan.Dockerfile` | Builds third-party libraries for GEOS on Pecan environment. | `pecan-gcc-openmpi-mkl-cuda` | CMake, make, bc, file, bison, flex, patch, openssh-clients, texlive, graphviz, libxml2, git, sccache |
| `pangea4-gcc.Dockerfile` | Builds a toolchain with GCC, CMake and Python on Pangea 4. | `spack/centos-stream` | GCC, CMake, Python, Wget |
| `pangea4-gcc-hpcxompi-onemkl.Dockerfile` | Builds a toolchain with GCC, HPCXOMPI and ONEMKL on Pangea 4. | `pangea4-gcc` | GCC, CMake, Python, Wget, HPCXOMPI, ONEMKL |
| `tpl-pangea4.Dockerfile` | Builds third-party libraries for GEOS on Pangea 4 environment. | `pangea4-gcc-hpcxompi-onemkl` | GCC, CMake, Python, Wget, HPCXOMPI, ONEMKL, sccache |

## DockerHub

The Docker images built from the ${cluster}.Dockerfile are available on DockerHub under for:
- pangea4 cluster : [onetechssc/pangea4](https://hub.docker.com/r/onetechssc/pangea4)
- pecan cluster : [totogaz/pecan-gcc8.2.0-openmpi4.0.1-mkl2019.5-cuda11.5.119-no-geosx](https://hub.docker.com/r/totogaz/pecan-gcc8.2.0-openmpi4.0.1-mkl2019.5-cuda11.5.119-no-geosx)
65 changes: 65 additions & 0 deletions docker/TotalEnergies/pangea4-gcc-hpcxompi-onemkl.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#######################################
# Pangea 4 image : gcc - hpcxompi - onemkl
#######################################
#
# Installs :
# - gcc = 12.1
# - hpcx = 2.17.1
# - intel-oneapi-mkl = 2023.2.0
# - cmake = 3.27.9
# - python = 3.11
#
#######################################
#
# Description :
# - the image is based on onetechssc/pangea4:gcc12.1_v1.0 built from pangea4-gcc.Dockerfile
# - this image is deployed as onetechssc/pangea4:gcc12.1-hpcx2.17.1-onemkl2023.2.0_v1.0
# - oneAPI MKL is installed via spack (as done for Pangea 4)
# - hpcx is installed by extracting the tarball (as done for Pangea 4)
# - gcc wrappers are created to mimic the Cray environment (cc, CC, ftn)
#
#######################################

# -------------------------------------
# PANGEA4 - GCC-HPCX-MKL
FROM onetechssc/pangea4:gcc12.1_v1.0 AS pangea4
# ------
# LABELS
LABEL description="Pangea 4 image : gcc - cmake - python - hpcx - mkl"
LABEL version="1.0"
LABEL maintainer="TotalEnergies HPC Team"
# ------
# ARGS
ARG ONEAPI_MKL_VERSION=2023.2.0
ARG HPCX_VERSION="hpcx-v2.17.1-gcc-mlnx_ofed-redhat8-cuda12-x86_64"
ARG HPCX_TARBALL="$HPCX_VERSION.tbz"
ARG HPCX_URL="http://www.mellanox.com/downloads/hpc/hpc-x/v2.17.1/$HPCX_TARBALL"
ARG CRAY_WRAPPERS_DIR=/sw/cray-wrappers
# ------
# INSTALL
# intel-oneapi-mkl
RUN spack install intel-oneapi-mkl@$ONEAPI_MKL_VERSION %gcc@$GCC_VERSION
# hpcx not available in spack -> download and untar in /sw directory
RUN mkdir -p /sw && \
spack load wget && \
wget $HPCX_URL -O /tmp/$HPCX_TARBALL && \
tar -xvf /tmp/$HPCX_TARBALL -C /sw
ENV HPCX_HOME=/sw/$HPCX_VERSION
# ------
# ENV
# - create wrappers for gcc
RUN mkdir -p $CRAY_WRAPPERS_DIR && \
spack load gcc@$GCC_VERSION python@$PYTHON_VERSION cmake@$CMAKE_VERSION intel-oneapi-mkl@$ONEAPI_MKL_VERSION && \
GCC_INSTALL_DIR=$(spack location -i gcc@$GCC_VERSION) && \
ln -s ${GCC_INSTALL_DIR}/bin/gcc $CRAY_WRAPPERS_DIR/cc && \
ln -s ${GCC_INSTALL_DIR}/bin/g++ $CRAY_WRAPPERS_DIR/CC && \
ln -s ${GCC_INSTALL_DIR}/bin/gfortran $CRAY_WRAPPERS_DIR/ftn
# create env script
RUN <<EOF cat > /root/set_env.sh
#!/bin/bash
spack load gcc@$GCC_VERSION python@$PYTHON_VERSION cmake@$CMAKE_VERSION intel-oneapi-mkl@$ONEAPI_MKL_VERSION && \
source ${HPCX_HOME}/hpcx-init.sh
hpcx_load
PATH=$CRAY_WRAPPERS_DIR:\$PATH
EOF
RUN chmod +x /root/set_env.sh
50 changes: 50 additions & 0 deletions docker/TotalEnergies/pangea4-gcc.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#######################################
# Pangea 4 image : gcc - cmake - python
#######################################
#
# Installs :
# - gcc = 12.1
# - cmake = 3.27.9
# - python = 3.11
#
#######################################
#
# Description :
# - the image is based on spack/centos-stream:latest (CentOS Stream 8 image with spack installed)
# - this image is deployed as onetechssc/pangea4:gcc12.1_v1.0
# - gcc, cmake, python and wget are installed via spack (as done for Pangea 4)
#
#######################################

# -------------------------------------
# PANGEA4 - BASE [GCC - CMAKE - PYTHON]
# install tools via spack for linux/x86_64 architecture and redhat8 platform
FROM --platform=linux/x86_64 spack/centos-stream:latest AS pangea4-base
# ------
# LABELS
LABEL description="Pangea 4 image : gcc - cmake - python"
LABEL version="1.0"
LABEL maintainer="TotalEnergies HPC Team"
# ------
# ARGS
ARG GCC_VERSION=12.1
ARG CMAKE_VERSION=3.27.9
ARG PYTHON_VERSION=3.11
# ------
# INSTALL
# gcc
RUN spack install gcc@$GCC_VERSION
RUN . /opt/spack/share/spack/setup-env.sh &&\
spack load gcc@$GCC_VERSION &&\
spack compiler find
# python
RUN spack install python@$PYTHON_VERSION %gcc@$GCC_VERSION
# cmake
RUN spack install cmake@$CMAKE_VERSION %gcc@$GCC_VERSION
# wget
RUN spack install wget %gcc@$GCC_VERSION
# ------
# ENV
ENV GCC_VERSION=${GCC_VERSION}
ENV CMAKE_VERSION=${CMAKE_VERSION}
ENV PYTHON_VERSION=${PYTHON_VERSION}
41 changes: 41 additions & 0 deletions docker/TotalEnergies/tpl-pangea4.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#######################################
# Pangea 4 tpl image
#
# Description :
# - generic image for building geos tpls on Pangea 4 environments
# - the docker base image can be any pangea4 docker file
# - tools of the base image are expected to be sourced in the set_env.sh script
#######################################

# -------------------------------------
# PANGEA4 - TPL TOOLCHAIN
ARG DOCKER_ROOT_IMAGE
FROM $DOCKER_ROOT_IMAGE AS pangea4_tpl_toolchain
# ------
# LABELS
LABEL description="Pangea 4 image : geos_tpl"
LABEL version="1.0"
LABEL maintainer="TotalEnergies HPC Team"
# ------
# ARGS
ARG TMP_DIR=/tmp
ARG SRC_DIR=$TMP_DIR/src
ARG BLD_DIR=$TMP_DIR/build
ARG INSTALL_DIR
ARG HOST_CONFIG
# ------
# ENV
ENV GEOSX_TPL_DIR=$INSTALL_DIR
ENV GCC_PATH=\${GCC_INSTALL_DIR}
# ------
# INSTALL
RUN --mount=src=.,dst=$SRC_DIR source /root/set_env.sh && \
$SRC_DIR/docker/configure-tpl.sh -DMKL_LIBRARIES=\${MKLROOT}/lib/intel64/libmkl_rt.so
WORKDIR $BLD_DIR
RUN --mount=src=.,dst=$SRC_DIR source /root/set_env.sh && \
make
# ------
# CACHE
# install `sccache` binaries to speed up the build of `geos`
RUN --mount=src=.,dst=$SRC_DIR $SRC_DIR/docker/install-sccache.sh
ENV SCCACHE=/opt/sccache/bin/sccache