CarpetX: only multi-thread MFIter if AMReX supports it

[rhaas80]

This adds a couple of #ifdef AMP_USE_OPENMP around pragma omp parallel to avoid calling AMReX functions from within OpenMP parallel sections if AMReX itself is not compiled with OpenMP support (and thus eg fails to check for nested MFIter instantiations only on a single thread).

The code that would trigger this is in AMReX Src/Base/AMReX_MFIter.cpp:

#ifdef AMREX_USE_OMP
#pragma omp master
#endif
    {
        ++depth;
        AMREX_ALWAYS_ASSERT_WITH_MESSAGE(depth == 1 || MFIter::allow_multiple_mfiters,
            "Nested or multiple active MFIters is not supported by default.  This can be changed by calling MFIter::allowMultipleMFIters(true)".);
    }

and AMReX user code like this:

#pragma omp parallel
for(MFIter mfi(foo) ; ...)

will trigger the error since the parallel section causes a MFIter object to be created in each thread and without AMREX_USE_OMP AMReX will trigger the depth == 1 test on all threads instead of just the master thread.

[rhaas80]

@eschnett says (private conversation) to rather abort build if OpenMP capabilities between CarpetX and AMReX do not match.