v23.0 kernel compile error -Kernel not initialized or has been freed #508

dlwldud99 · 2023-07-03T08:34:19Z

dlwldud99
Jul 3, 2023

Hello,

I'm trying to run turbChannel using a constant body force which could be done in Nek5000 by setting the following in the .usr file.

      ffx = 1.0 
      ffy = 0.0
      ffz = 0.0

I followed the steps in Setting Custom Source Terms from docs, however I'm unsure how to modify the code to suit the current version as I am confused by the meaning of "Single source udf+oudf" in the release note.
I simply removed the

void UDF_LoadKernels(nrs_t * nrs)
{
  constantFillKernel = udfBuildKernel(nrs, "constantFill");
}

section but left the others parts of the code exactly the same as in the docs (only modified the gravity to the force term as required).

When I run this case, I'm getting the following error.

---[ Error ]--------------------------------------------------------------------
    File     : /home/lee16/nekRS/3rd_party/occa/src/core/kernel.cpp
    Line     : 42
    Function : assertInitialized
    Message  : Kernel not initialized or has been freed
    Stack
      10 occa::error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
       9 occa::kernel::assertInitialized() const
       8 occa::kernel::operator()(occa::kernelArg const&, occa::kernelArg const&, occa::kernelArg const&, occa::kernelArg const&) const
       7 custom_source(nrs_t*, double, occa::memory, occa::memory)
       6 std::_Function_handler<void (nrs_t*, double, occa::memory, occa::memory), void (*)(nrs_t*, double, occa::memory, occa::memory)>::_M_invoke(std::_Any_data const&, nrs_t*&&, double&&, occa::memory&&, occa::memory&&)
       5 timeStepper::makef(nrs_t*, double, int, occa::memory, occa::memory)
       4 timeStepper::initStep(nrs_t*, double, double, int)
       3 nekrs(main+0x111d)
       2 /usr/lib64/libc.so.6(__libc_start_main+0xf5)
       1 nekrs()
================================================================================

--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[warn] Epoll MOD(1) on fd 24 failed.  Old events were 6; read change was 0 (none); write change was 2 (del): Bad file descriptor
[warn] Epoll MOD(4) on fd 24 failed.  Old events were 6; read change was 2 (del); write change was 0 (none): Bad file descriptor

Can I get some help on resolving this error?
Thank you.

Regards,
Joy

Answered by stgeke

Jul 3, 2023

The kernel function name has to match the actual kernel name. So if you okl kernel is named foo, you have to call foo() in the udf.

View full answer

stgeke · 2023-07-03T08:41:25Z

stgeke
Jul 3, 2023
Maintainer

Can you please share your udf.

2 replies

dlwldud99 Jul 3, 2023
Author

Hi stgeke,
here is my udf file.

//
// nekRS User Defined File
//
#include <math.h>
#include "udf.hpp"

static dfloat ReTau;
static dfloat zLength;
static dfloat xLength;
static dfloat betaY;
static occa::kernel constantFillKernel;

#ifdef __okl__

#include "turbChan.oudf"

#endif

/* User Functions */

void custom_source(nrs_t *nrs, dfloat time, occa::memory o_U, occa::memory o_FU)
{
  mesh_t * mesh = nrs->meshV;
  // what momentum equation we want to add force to
  int component = 0;
  constantFillKernel(nrs->meshV->Nelements, 1, component * nrs->fieldOffset, o_FU);
}

void useric(nrs_t *nrs)
{
  auto mesh = nrs->meshV;

  const auto C = 5.17;
  const auto k = 0.41;
  const auto eps = 1e-2;
  const auto kx = 23.0;
  const auto kz = 13.0;
  const auto alpha = kx * 2 * M_PI / xLength;
  const auto beta = kz * 2 * M_PI / zLength;
  dfloat mue;
  platform->options.getArgs("VISCOSITY", mue);

  if (platform->options.getArgs("RESTART FILE NAME").empty()) {
    for (int i = 1; i < mesh->Nlocal; i++) {
      const auto x = mesh->x[i];
      const auto y = mesh->y[i];
      const auto z = mesh->z[i];
      const auto yp = (y < 0) ? (1 + y) * ReTau : (1 - y) * ReTau;

      dfloat ux =
          1 / k * log(1 + k * yp) + (C - (1 / k) * log(k)) * (1 - exp(-yp / 11) - yp / 11 * exp(-yp / 3));
      ux *= ReTau * mue;

      nrs->U[i + 0 * nrs->fieldOffset] = ux + eps * beta * sin(alpha * x) * cos(beta * z);
      nrs->U[i + 1 * nrs->fieldOffset] = eps * sin(alpha * x) * sin(beta * z);
      nrs->U[i + 2 * nrs->fieldOffset] = -eps * alpha * cos(alpha * x) * sin(beta * z);
    }
  }
}

/* UDF Functions */

void UDF_Setup0(MPI_Comm comm, setupAide &options)
{
  platform->par->extract("casedata", "ReTau", ReTau);
  platform->par->extract("casedata", "zLength", zLength);
  platform->par->extract("casedata", "xLength", xLength);
  platform->par->extract("casedata", "betaY", betaY);

  *((double *)nek::scPtr(1)) = xLength;
  *((double *)nek::scPtr(2)) = zLength;
  *((double *)nek::scPtr(3)) = betaY;
}

void UDF_Setup(nrs_t *nrs)
{
  if (platform->options.compareArgs("CONSTANT FLOW RATE", "FALSE")) {
    // assign function pointer to drive flow by constant mean pressure gradient
    udf.uEqnSource = &custom_source;
  }

  useric(nrs);
}

void UDF_ExecuteStep(nrs_t *nrs, dfloat time, int tstep)
{

#if 0
  if (nrs->isOutputStep) {
    nek::ocopyToNek(time, tstep);
    nek::userchk();
  }
#endif
}

dlwldud99 Jul 3, 2023
Author

edit:
I believe the compiling is done successfully, I shouldn't have titled this discussion thread as a compile error.
I only get the error when I try to run this case. Here is the top part of the compile log,

UTC time: Mon Jul  3 08:49:21 2023

                 __    ____  _____
   ____   ___   / /__ / __ \/ ___/
  / __ \ / _ \ / //_// /_/ /\__ \ 
 / / / //  __// ,<  / _, _/___/ / 
/_/ /_/ \___//_/|_|/_/ |_|/____/  v23.0.0 (4f87e0e)

COPYRIGHT (c) 2019-2023 UCHICAGO ARGONNE, LLC

MPI tasks: 1

jit-compiling for >=1 MPI tasks ...

reading par file ...
Initializing device 
active occa mode: CUDA

Loading cached [simpleAtomicAdd] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/4c0a989a9c91bcd8/string_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/4c0a989a9c91bcd8/binary]
Loading cached [simpleAtomicAdd] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/4c0a989a9c91bcd8/launcher_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/4c0a989a9c91bcd8/launcher_binary]
Loading cached [myDummyKernelName] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/3761293f41930474/string_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/3761293f41930474/binary]
Loading cached [myDummyKernelName] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/3761293f41930474/launcher_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/3761293f41930474/launcher_binary]
using NEKRS_HOME: /home/lee16/nekrs
using NEKRS_CACHE_DIR: /data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache
using OCCA_CACHE_DIR: /data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/

building nekInterface for lx1=8, lelt=3840 and lelg=3840 ... done (0.490462s)
building udf ... 
�[32mPreprocessing CXX source to CMakeFiles/OKL.dir/okl.cpp.i�[0m
�[35m�[1mConsolidate compiler generated dependencies of target UDF�[0m
[100%] Built target UDF
done (2.19536s)
loading udf kernels ... 
Loading cached [constantFill] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/udf/udf.okl] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/963f8631ee8851cc/binary]
Loading cached [constantFill] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/963f8631ee8851cc/launcher_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/963f8631ee8851cc/launcher_binary]
Loading cached [compileUDFKernelsTest] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/8a2e204014edeaef/string_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/8a2e204014edeaef/binary]
Loading cached [compileUDFKernelsTest] from [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/8a2e204014edeaef/launcher_source.cpp] in [/data/gpfs/projects/punim0428/Joy/sawtooth/minChan/.cache/occa/cache/8a2e204014edeaef/launcher_binary]
done (0.00244832s)

(please note the name change from turbChan -> minChan)

stgeke · 2023-07-03T09:03:54Z

stgeke
Jul 3, 2023
Maintainer

The kernel function name has to match the actual kernel name. So if you okl kernel is named foo, you have to call foo() in the udf.

1 reply

dlwldud99 Jul 3, 2023
Author

I added the following to load the kernel but the error still appears.

void UDF_LoadKernels(occa::properties& kernelInfo)
{
  constantFillKernel = oudfBuildKernel(occa::properties& kernelInfo, "constantFill");
}

The name of the kernel is 'constantFill' in .oudf.

I believe this is the correct way of calling the kernel in v22, should the format be modified in any way to suit v23 regarding the new feature "Single source udf+oudf"?

Regards,
Joy

stgeke · 2023-07-03T09:20:17Z

stgeke
Jul 3, 2023
Maintainer

In your case the names don't match (constantFill vs constantFillKernel). Please check the examples in v23.

1 reply

dlwldud99 Jul 3, 2023
Author

Thank you for your comments, I have now fixed the codes. I completely misunderstood the structures of the input files as I have just transferred from Nek5000.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v23.0 kernel compile error -Kernel not initialized or has been freed #508

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

v23.0 kernel compile error -Kernel not initialized or has been freed #508

dlwldud99 Jul 3, 2023

Replies: 3 comments · 4 replies

stgeke Jul 3, 2023 Maintainer

dlwldud99 Jul 3, 2023 Author

dlwldud99 Jul 3, 2023 Author

stgeke Jul 3, 2023 Maintainer

dlwldud99 Jul 3, 2023 Author

stgeke Jul 3, 2023 Maintainer

dlwldud99 Jul 3, 2023 Author

dlwldud99
Jul 3, 2023

Replies: 3 comments 4 replies

stgeke
Jul 3, 2023
Maintainer

dlwldud99 Jul 3, 2023
Author

dlwldud99 Jul 3, 2023
Author

stgeke
Jul 3, 2023
Maintainer

dlwldud99 Jul 3, 2023
Author

stgeke
Jul 3, 2023
Maintainer

dlwldud99 Jul 3, 2023
Author