Build Moonray 1.5.0.0 for Houdini with Docker

[adro79]

Hello there! I'll show the steps I made for successfully build Moonray for Houdini using Docker.

⚠️ Keep in mind this isn't an easy proccess and may differ for your Houdini version. In this guide I'll use Houdini 20.0.547 GCC11.2 and Arch Linux as my workspace, in the end we'll get a Moonray build that could run in any Linux machine that uses the same Houdini Version.

Building Moonray for Houdini requires using Houdini's libraries, for that we'll build an environment similar to the VFX Platform CY2023 Specification (as mentioned here)

0. Current Issues

• XPU Mode: I couldn't find a way to enable xpu for Solaris. Maybe it's not available yet.
• Missing features: https://docs.openmoonray.org/user-reference/tools/hydra/hdmoonray-houdini/ (bottom)

Report your issues in the comments so we can get constancy. Thanks!

1. Prepare

Create an empty directory and copy the following:

- Houdini Dir: /opt/hfs20.0
- Optix SDK 7.6: We can download it from here.
- CMake Houdini Configs: https://docs.openmoonray.org/assets/build_extras/moonray_houdini_pxr_config.tar
- Moonray Source Code: git clone --recurse-submodules https://github.com/dreamworksanimation/openmoonray.git

2. Patching

We'll need to apply some patches to make Moonray buildable for Houdini 20.0:

- pxr-houdini: This patch sets USD version to 23.08, support for python 3.10 and repaths some libraries.
Patch file

cd pxr-houdini
patch -i ../pxr-houdini.patch

- Moonray: This one fixes building for USD <23 (see here) and a fix for exr 3.0 (see here)
Patch file

cd openmoonray
git apply ../moonray.patch

- CMakeLists: Custom version of CMakeList to build the dependency versions I tested.
CMakeLists.txt

cp -f CMakeLists.txt openmoonray/building/CMakeLists.txt

HDAs not working

I encountered an error while trying to use Moonray VOPs, seems like this commit broke the HDAs. So we would need to revert the files only on the moonray/moonray_dcc_plugins/houdini/otls folder
You can use v1.0.0.0: https://github.com/dreamworksanimation/moonray_dcc_plugins/tree/v1.0.0.0

2. Building

Create an empty file in the directory named Dockerfile with this content:

FROM rockylinux:9
WORKDIR /

COPY openmoonray /source
COPY hfs20.0 /opt/hfs20.0
COPY pxr-houdini /pxr-houdini
COPY optix /opt/optix

CMD ["bash"]

RUN dnf install -y epel-release && dnf config-manager --enable crb
RUN dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
RUN dnf install -y cuda-runtime-12-3-12.3.1 cuda-toolkit-12-3-12.3.1 && dnf clean packages -y
RUN dnf install -y gcc gcc-c++ glibc bison flex wget git patch libglvnd-devel giflib-devel libmng libtiff-devel libjpeg-devel pam-devel libatomic libuuid-devel openssl-devel curl-devel freetype-devel zlib-devel which && dnf clean packages -y

RUN mkdir -p /moonray/{bin,lib,include}
RUN wget https://github.com/Kitware/CMake/releases/download/v3.27.2/cmake-3.27.2-linux-x86_64.tar.gz
RUN tar xzf cmake-3.27.2-linux-x86_64.tar.gz -C /moonray
ENV PATH=/moonray/cmake-3.27.2-linux-x86_64/bin:${PATH}
ENV LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH}

RUN mkdir -p /moonray/cmake

RUN cmake -S /source/building -B /build
RUN cmake --build /build -- -j $(nproc) && rm -rf /build

ENV PATH=/opt/hfs20.0/python/bin:$PATH
ENV CMAKE_PREFIX_PATH=/moonray:/pxr-houdini:/opt/optix
ENV PREFIX_PXR=/pxr-houdini
ENV PXR_LIB_PREFIX=/opt/hfs20.0/dsolib
ENV HOUDINI_INSTALL_DIR=/opt/hfs20.0
ENV PXR_INCLUDE_PREFIX=/opt/hfs20.0/toolkit/include
ENV PXR_DEPS_PREFIX=/opt/hfs20.0/python
ENV PXR_BOOST_PYTHON_LIB=/opt/hfs20.0/dsolib/libhboost_python310-mt-x64.so

# Fix finding libraries
RUN ln -s /usr/lib64/libSM.so.6.0.1 /usr/lib64/libSM.so
RUN ln -s /usr/lib64/libICE.so.6.3.0 /usr/lib64/libICE.so
RUN ln -s /usr/lib64/libXext.so.6.4.0 /usr/lib64/libXext.so

RUN dnf install -y qt5-qtbase-devel qt5-qtscript-devel
RUN cmake -S /source -B /build -DPython_ROOT=/opt/hfs20.0/python -DABI_VERSION=0 -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
RUN cmake --build /build -j $(nproc)
RUN cmake --install /build --prefix /moonray/openmoonray
RUN rm -rf /build

# Copy some system librarie
RUN cp -f /usr/lib64/libicu* /moonray/lib64/
RUN cp -f /usr/lib64/libtiff* /moonray/lib64/

You may want to change the cuda version to be in pair as your local machine.
https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/

Congrats! you can start building moonray with this command:

docker build -t moonray-houdini .

WARNING! This command may take a while to complete since it will download and build large packages.

3. Installing

Once the previous command finishes correctly we should have a new docker image in our local machine, run it using:

docker run -it moonray-houdini:latest

Open a new terminal and run:

docker ps

Find the "CONTAINER ID" of your docker image, in my case the ID is "5973ae7005cf":

CONTAINER ID   IMAGE                    COMMAND                  CREATED         STATUS                    PORTS                                       NAMES
5973ae7005cf   moonray-houdini:latest   "bash"                   6 seconds ago   Up 6 seconds                                                          mystifying_jemison

Using the Container ID, extract the moonray directory to your local machine:

docker cp <CONTAINERID>:/moonray <PATH/TO/EXTRACT/MOONRAY>

Go to the extracted Moonray directory. In my case I cleaned up the folder a little:
Delete cmake-3.27.2-linux-x86_64, move openmoonray contents to the parent directory and move lib64 files to lib.

Copy the python3.9libs directory to python 3.10libs since that is the python version that Houdini uses:

cp -r <PATH/TO/MOONRAY>plugin/houdini/python3.9libs <PATH/TO/MOONRAY>plugin/houdini/python3.10libs

Run this command once:

LD_LIBRARY_PATH=<PATH/TO/MOONRAY>/lib bin/rdl2_json_exporter --out shader_json/ --sparse

4. Setup Environment

For Moonray to work, you'll need to add some environment variables to your system and houdini.env.

Add this line in your houdini.env:

HOUDINI_PATH = "<PATH/TO/MOONRAY>/houdini;<PATH/TO/MOONRAY>/plugin/houdini;$HOUDINI_PATH;&"

And add those environment variables in your system (in a terminal before launching houdini or creating a sh file):

# MOONRAY #
export MOONRAY_ROOT=/mnt/proj/deps/moonrayHoudini
export PXR_PLUGINPATH_NAME=$PXR_PLUGINPATH_NAME:$MOONRAY_ROOT/plugin/usd
export RDL2_DSO_PATH=$MOONRAY_ROOT/rdl2dso
export MOONRAY_CLASS_PATH=$MOONRAY_ROOT/shader_json
export ARRAS_SESSION_PATH=$MOONRAY_ROOT/sessions

export PATH=$PATH:$MOONRAY_ROOT/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MOONRAY_ROOT/lib

5. Inside Houdini

You can now find Moonray inside Solaris! As of now, there some things that you should now.

Moonray Shaders inside Solaris (H20.0)

As of Houdini 20.0, VOP nodes inside Solaris have been splitted into different subnetworks. To use Moonray VOPs you need to add moonray to the Tab Menu Mask in any of those Builders.

Geometry Settings

We can use the LOP Wrangle Node to add our desired geometry settings as the documentation says:

usd_addprimvar(0, @primpath, "moonray:mesh_resolution", "float");
usd_setprimvar(0, @primpath, "moonray:mesh_resolution", 10);

However we can also create them using the Edit Properties node.

I hope that this guide helped you build Moonray, can't wait to use it in my personal projects soon.
Feel free to comment some feedback for this guide if you can, as I don't have any experience on Docker and learnt on the fly.

Happy creating!

[BrianHanke]

Nice guide, thanks for sharing!

[mpjohn]

Yes thanks for sharing. I just want to confirm the version of Houdini...
The closest i can find is Houdini 20.0.547 GCC11.2

[adro79]

Sorry my mistake, I'm using Houdini 20.0.547 as you're saying.
The version doesn't really matter as long as it's H20 (unless SideFx modifies dependencies versions).

[mpjohn]

Great stuff. Quick question...do you think this will work with a regular build ( non container build ? )
If not is there some advantage to the container build?
Thanks

[adro79]

If you are using Rocky Linux 9/Centos 7 as your main distro you can build it without a container. The main goal of this guide is to get Moonray working on any disto that can run Houdini. (Like if you install any other renderer)

[mzogithub]

Nice work. Did you consider using Docker via Distrobox (https://github.com/89luca89/distrobox)? It should theoretically overcome many of the issues, especially those related to GPU.

[adro79]

Hi, I don't have that much experience with containers as I mention in the end of the guide. This method is only for building Moonray, not running it inside Docker. If you manage to fix some issues with Distrobox please tell me. Thanks.

[szuhow]

I compiled Moonray against Houdini 20 as instructed in description above (with some changes, like increased version of Embree, and building in Centos 7 container, not Rocky Linux). After all operations Houdini has Moonray viewport in Solaris - and it is working. However, when I want to use hd_render/hd_usd2rdl in terminal, I have error:

Could not find platform independent libraries
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to [:<exec_prefix>]
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = '/opt/rez_packages/demeter/0.1.2'
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/bin/python3'
sys.base_prefix = '/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10'
sys.base_exec_prefix = '/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10'
sys.platlibdir = 'lib'
sys.executable = '/usr/bin/python3'
sys.prefix = '/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10'
sys.exec_prefix = '/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10'
sys.path = [
'/opt/rez_packages/demeter/0.1.2',
'/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10/lib/python310.zip',
'/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10/lib/python3.10',
'/home/prisms/builder-new/WeeklyDevToolsHEAD/dev_tools/src/python/python-3.10/prefix-py3.10/lib/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f626ef2c9c0 (most recent call first):

What could be reason for that?

[adro79]

Hi, I didn't try this feature yet. Are you using system's python? Maybe it works with hython.

[szuhow]

I am using Python that was used to compile Houdini. Anyway, I have visible Moonray (Alpha) in render delegate, but all tabs like Lights, Render settings do not have Moonray Tab (I exported envs for houdini_path). Any help on that?

[adro79]

Are you using the HDAs from v1.0.0.0 as mentioned in the patching section?
https://github.com/dreamworksanimation/moonray_dcc_plugins/tree/v1.0.0.0

[szuhow]

Yes, I did use that one, but still no visible Moonray tag:

[adro79]

Sorry I didn't read your first comment carefully.
I don't recommend using centos 7 for building Moonray since the libraries are older than the ones that Houdini 20 is using, so it can cause some incompatibilities.
I chose Rocky Linux 9 as a base because it's the system that's more in par with the versions used for Houdini https://www.sidefx.com/docs/houdini/licenses/.
Minor changes in the build process can cause Moonray not run at all on Houdini so consider sticking to the guide as closely as possible.

[mpjohn]

Am I the only one waiting on the new release to make this a bit more user friendly.
I mean it's hard enough to build successfully but getting it to work with DCC apps is another thing entirely.
Surely there's a way to make this much more user friendly and accessible?!

[adro79]

You're not the only one. Note that Moonray is still in Alpha state, so only the experienced ones can play with it.
Things like prebuilt binaries are too far away (if they're even possible for legal stuff), but I'll try to write an easier guide for the next update.

[mpjohn]

That would be fantastic!. Build instructions that work with specific DCC's are what are needed. Otherwise you end up with a build that can't be used...which as you can imagine is kind of pointless.

[mpjohn]

Yep I switched over to Rocky and the build works great but I don't have a clear route to DCC apps. ( preferably Maya!! )
I got a bum steer last time I tried to rebuild with Houdini's specific version of python.

[adro79]

Houdini is the only supported DCC atm; it doesn't even have all the functionalities like light linking or render variables.
Maybe you could be able to run it in Maya, but I think you wouldn't be able to do anything other than render.
Blender could be the second DCC to have Moonray support since it has more interest, but they still need to have proper USD support as a workflow rather than a file archive, so it depends on Blender developers.

That being said, since Houdini is the only supported DCC, the other DCCs would require you to build the necessary tools to use them.

[Helio-Alex]

Hello,

Great guide. This is the furthest I have gotten in building Moonray for Houdini.

While building, I ran into an issue with the Boost library.
CMake throws an error that python-config.jam cannot be found in moonray/building.

I manually created that file but kept it empty, and that seemed to make it pass that build step. However, I was wondering if you have that file and what the contents of the file are?

The build then fails again with:
RUN cmake -S /source -B /build -DPython_ROOT=/opt/hfs20.0/python -DABI_VERSION=0 -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc

Also, a Boost-related error:

CMake Error at /moonray/lib/cmake/Boost-1.76.0/BoostConfig.cmake:141 (find_package):
  Could not find a package configuration file provided by "boost_python"
  (requested version 1.76.0) with any of the following names:

    boost_pythonConfig.cmake
    boost_python-config.cmake

  Add the installation prefix of "boost_python" to CMAKE_PREFIX_PATH or set
  "boost_python_DIR" to a directory containing one of the above files.  If
  "boost_python" provides a separate development package or SDK, be sure it
  has been installed.

Not sure what I have done wrong?

Thanks again.
Alex.

[Helio-Alex]

I ended up doing this which seems to do the trick and the compile was successful.

import python ;
using python : 3.10 : /opt/hfs20.0/python/bin/python3 ;

Still need to test in Houdini if everything actually worked

[adro79]

I ended up doing this which seems to do the trick and the compile was successful.

import python ;
using python : 3.10 : /opt/hfs20.0/python/bin/python3 ;

Still need to test in Houdini if everything actually worked

Great to see that you managed to compile it. I'm thinking of creating an script/guide for Moonray 1.6 and H20.5 using distrobox soon, so stay tuned :)

[Helio-Alex]

Awesome news!!!

Considering that I am in the middle of doing the same process myself, I would be happy to test it and provide feedback for you.