Profiling Vlasiator with TAU

Vlasiator compiled with phiprof does not require re-compilation to work with TAU. Simply launch with:

srun tau_exec -T pthread,mpi,phiprof,papi ./vlasiator --run_config Configuration.cfg

Set different profiling options for TAU with environment variables

export TAU_CALLSITE=1
export TAU_CALLPATH=1
export TAU_CALLPATH_DEPTH=100

export TAU_PROFILE_FORMAT=merged
#export TAU_VERBOSE=1
#export TAU_VERBOSE_FILES=1

#export TAU_SAMPLING=1
#export TAU_EBS_RESOLUTION=line
#export TAU_EBS_RESOLUTION=function
#export TAU_EBS_RESOLUTION=file

Yann has built Tau on Lumi at: export PATH=$PATH:/scratch/project_465000287/kempfyan/tau/tau-2.32/craycnl/bin using ./configure -bfd=download -pthread -mpi -papi=/opt/cray/pe/papi/6.0.0.15 -dwarf=download -otf=download -iowrapper -unwind=download -useropt=-std=c++17 -phiprof

This example uses pthread instead of openmp: Add unformation

Tracing post-processing is required, output .json files can be opened with https://ui.perfetto.dev after adding a missing square bracket ] to the end of the file:

export TAU_TRACE=1; mpirun –np 64 tau_exec ./a.out; tau_treemerge.pl;
tau_trace2json tau.trc tau.edf –chrome –ignoreatomic –o app.json
echo ']' >>app.json

Event-based sampling (EBS, TAU_SAMPLING=1) should work, but at least multi-node jobs fail to progress.