SoOffscreenRenderer problems when running on RHEL8 and SLES12

[shanslow]

I'm not sure if this is a problem per se, or if there's advice that can be given on something we may be doing incorrectly, but we're currently seeing issues outputting image files to disk on RHEL8 and SLES12. Our software is fully built on RHEL7.9 but deployed on these sister versions. Historically that's worked fine building on version X and running on X+1 from RHEL4 onwards but seemingly no longer for RHEL7->RHEL8.

Rendering to the screen is fine, but when we instantiate and the SoOffscreenRenderer and attempt to render to file we get the following messages before the appliaction segementation faults:

Coin warning in glxglue_context_create_software(): Couldn't create GLX context.
Coin warning in SoOffscreenRenderer::renderFromBase(): Could not set up an offscreen OpenGL context.

The problem seems to manifest at line 959, with the call to this->glcanvas.deactivateGLContext(); where it attempts to deactivate the offscreen GL canvas that it's created. This ultimately ends up in glxglue_context_reinstate_previous where the glXMakeCurrent call fails. On RHEL7 it appears to be able to store a reusable context in the glxglue_contextdata but the same information is null on RHEL8.

Is this a problem or could someone point out what we may be missing to get the offscreen rendering feature to work for us? Our build flags for building Coin, and the source below, are minimal with nothing specific other than optimisation level, ensure it's 64 bit and position independent code. We initially thought it may be a VM related issue with a software video driver, but the problem is manifesting on real hardware with dedicated GPU.

The attached source below exhibits the problem, it renders (badly!) a simple cuboid scene, which it then attempts to write to disk, which works without issue On RHEL7.

Thanks

source.txt

[ytterx]

Hi,

We are encountering the same issue, it seems it is due to that on RHEL8 "Indirect GLX" is disabled.
Advisory on Xorg:
https://www.x.org/wiki/Development/Security/Advisory-2014-12-09/
It might be possible to turn it on in your case in /etc/X11/xorg.conf.

I made a code patch which seems to work for RHEL8, this has not been tested anywhere else.

[shanslow]

Thanks for the pointer. I've spent a bit of time following your advice and have managed to find a working config. Firstly, we had been using Wayland on RHEL 8, I'm not sure if that's default or choice of our IT department, but was using that none-the-less. I couldn't find a way for that to work nicely, despite it having a +iglx option. The short of it is that the following allowed me to get something working:

• create a default /etc/X11/xorg.conf via Xorg -configure
• Append the following to /etc/X11/xorg.conf:

Section "ServerFlags"
    Option "AllowIndirectGLX" "on"
    Option "IndirectGLX" "on"
EndSection

• Login via a Classic X11 Display Server session

So we at least have a short term workaround thanks to your hint about Indirect GLX

[ytterx]

@shanslow My change was merged by @VolkerEnderlein. With the latest code there is now an environment variable that can control the direct rendering option. You might want to try this as well.