Postprocessing problem: Bug or initialization?

[BStringhamVRSK]

I just updated from 1.6.0 (ancient, I know) to 1.9.25. When I run with postprocessing enabled, I get a black screen with a texture in the bottom corner, that could be a ColorGrading LUT (see attached image). If I disable postprocessing, then I get the expected scene, without ColorGrading, of course.

I'm trying to figure out if I've not initialized something, or if there is a bug.
I'm running on Windows10, OpenGL, NVidia GeForce GTX 1050

Strangely, I ran my program under RenderDoc to see if I could find the problem, and it displayed my scene correctly! This suggests to me that it could be an initialization problem (either in my code or in Filament libraries), or perhaps a timing issue, where RenderDoc created just enough delay that a possible race condition resolved in the correct way.

[romainguy]

This texture is not the color grading LUT, it's the DFG LUT used for lighting. Do you have a repro test we could use?

[BStringhamVRSK]

Another possible symptom:
When I have depth of field enabled (so things will display), if I turn on VSM shadows, I get an abort. Here is the call stack:
utils::abort() Line 29 C++ utils::panic(const char * func, const char * file, int line, const char * assertion) Line 35 C++ filament::FMaterial::getSurfaceProgramSlow(unsigned char variantKey) Line 372 C++ filament::FMaterial::getProgramSlow(unsigned char variantKey) Line 357 C++ filament::FMaterial::getProgram(unsigned char variantKey) Line 87 C++ filament::RenderPass::recordDriverCommands(filament::backend::CommandStream & driver, const filament::RenderPass::Command * first, const filament::RenderPass::Command * last) Line 232 C++ filament::RenderPass::executeCommands(const char * name) Line 188 C++ filament::FRenderer::colorPass'::2'::<lambda_2>::operator()(const filament::FrameGraphResources & resources, const filament::FRenderer::colorPass::__l2::ColorPassData & data, filament::backend::CommandStream & driver) Line 831 C++ filament::FrameGraphPass<filament::FRenderer::colorPass'::2'::ColorPassData,filament::FRenderer::colorPass'::2'::<lambda_2>>::execute(const filament::FrameGraphResources & resources, filament::backend::CommandStream & driver) Line 69 C++ filament::RenderPassNode::execute(const filament::FrameGraphResources & resources, filament::backend::CommandStream & driver) Line 74 C++ filament::FrameGraph::execute(filament::backend::CommandStream & driver) Line 211 C++ filament::FRenderer::renderJob(utils::ArenaScope<utils::Arena<utils::LinearAllocator,utils::LockingPolicy::NoLock,utils::TrackingPolicy::DebugAndHighWatermark>> & arena, filament::FView & view) Line 565 C++ filament::FRenderer::renderInternal(const filament::FView * view) Line 212 C++ filament::FRenderer::render(const filament::FView * view) Line 196 C++ filament::Renderer::render(const filament::View * view) Line 1159 C++

It's failing an assert here in Material.cpp FMaterial::getSurfaceProgramSlow(uint8_t variantKey):
assert_invariant( variantKey == Variant::filterVariant(variantKey, isVariantLit()) );
=== FAILING HERE vvv ===
assert_invariant(!Variant::isReserved(variantKey));
==================

varientKey is 65

[romainguy]

It seems like you have a material that wasn't compiled with support for VSM. Or maybe a material that was not compiled with the appropriate version of matc.

[BStringhamVRSK]

@romainguy
I tracked down this abort to an object that has a lit material, but whose mesh was set to NOT cast and NOT receive shadows. If I set it to cast and receive shadows (or receive, which seems to enable cast), then it works. Setting it to only cast, not receive also causes the abort.

[romainguy]

Do you use variant filters?

[BStringhamVRSK]

No. I can post my shader code here, if you like. It's a very simple shader with a base color, roughness, metalness, and reflectance, which are passed in as numeric parameters.

[jeanlemotan]

I get the same issue sometimes, and I managed to work around by turning on MSAA with a sample count == 8 (didn't try other sample counts). The texture displayed in the corner seems to be the last one used - either in other views or in the current view.
Using the OpenGL backend, and so far it happens on an NVidia 1650 QMax and an integrated Intel GPU (same laptop) but doesn't happen on a Ryzen with a Radeon 480 nor a Vega 56.

[BStringhamVRSK]

Thank you for your reply. My symptoms do seem very similar to yours. The corner texture does seem to be the most recent used for me, as well. I will try changing some of my other view settings to see if I can create a work-around, like you've found, until a fix is made.

[BStringhamVRSK]

Just to follow up on this: If I enable MSAA by setting View::setSampleCount(4), then it also works for me. Haven't tried other sample counts, but I assume it would work.
I also noted that enabling depth of field makes things work and seems to have a lower performance impact.