Kokkos Hackathon

build on cori with shifter

get source code

git clone https://github.com/WireCell/wire-cell-gen-kokkos.git

run shifter:

shifter --image=hepcce2/larwire-kokkos:2.1 wire-cell-gen-kokkos/container/entrypoint.sh

run shifter with CUDA:

module load cgpu
salloc -A m2845 -C gpu -c 4 -N 1 --gres=gpu:1 --image=docker:hepcce2/larwire-kokkos:2.1 --module=gpu -t 60
srun -n 1 -c 4 --pty shifter wire-cell-gen-kokkos/container/entrypoint.sh

building script using cmake:

• use enable_kokkos_cuda or enable_kokkos_omp to use kokkos with different backend.
• use this script to build: ./build-cmake -j 4
• export LD_LIBRARY_PATH=<path-to-wire-cell-gen-kokkos>/build/kokkos/:$LD_LIBRARY_PATH
• export WIRECELL_PATH=<path-to-wire-cell-gen-kokkos>/cfg:$WIRECELL_PATH

unit test

run the kokkos unit test manually:

In the source folder (one level above build)

./build/kokkos/test/test_fft 

Example output:

Space: N6Kokkos6OpenMPE
WIRECELL_KOKKOSARRAY_FFTW
A: init 3.049067 ms.
A dims: 2 3
WIRECELL_KOKKOSARRAY_FFTW
irep: 0 0.524948 ms.
A: nrep: 1 avg: 0.52 ms.
WIRECELL_KOKKOSARRAY_FFTW
B: init 11.097288 ms.
B dims: 4 3
WIRECELL_KOKKOSARRAY_FFTW
irep: 0 2.083998 ms.
B: nrep: 1 avg: 2.08 ms.

full test running wire-cell as plugin of LArSoft

• sample input file (referred as g4.root below): https://www.phy.bnl.gov/~yuhw/wire-cell-pps/kokkos-hackathon/sample/
• in the example folder: lar -n 1 -c sim.fcl g4.root