how to enable OpenMPI?

[ekoontz]

Hi Marian devs, I'm trying to build with OpenMPI but haven't been successful in getting cmake to find my OpenMPI installation. I've installed OpenMPI with:

git clone https://github.com/open-mpi/ompi.git
cd ompi
./autogen.pl
./configure && make && make install

All of the above works apparently without problems, and I see mpi.h in /usr/local/include and libmpi.so in /usr/local/include, as expected.

Then I built marian with:

cmake .. -DCMAKE_BUILD_TYPE=Release -DUSE_SENTENCEPIECE=ON -DCOMPILE_CUDA=OFF -DUSE_MPI=ON && CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build .

(This is on a machine with no Nvidia GPU, thus the -DCOMPILE_CUDA=OFF, but that shouldn't matter for MPI support, right?)

When I run marian --build-info 2>&1 | grep USE_MPI, I see:

USE_MPI=ON

However, when I run marian -c transformer-model.yml (using the transformer-intro tutorial), I see:

....
[2023-06-05 23:02:00] [comm] Compiled without MPI support. Running as a single process on <hostname>

Did I miss something obvious? I will continue to dig into the code to look for the solution.