Maya crashes when toggling single view and four view with texture mode on

[csyshing]

Describe the bug

Hi,

Our artists reported they turned on texture mode on perspective view and when toggling single view and four view would cause Maya to crash.
This one is hard to reproduce, we have tried our best to simplify the repro steps with the Alab scene.

Steps to reproduce
Steps to reproduce the behavior:

1. Launch Maya and load the Alab USD scene
2. On the perspective view, turn on 'Textured'
3. Repeatedly pressing the space bar quickly to toggle single view and four view, hover the mouse cursor at the meantime (claim: this is not the usual workflow in production but we found that it's a sufficient way to reproduce the problem)
4. Maya would crash at some point

Stack trace:

  OGSMayaConsolidationInfo::UpdateTargetEffectParameterImpl(OGS::Objects::UString*)
  OGSShaderManager::EffectInstanceUpdated(OGS::Devices::EffectInstance*, OGS::Objects::UString*)
  Autodesk::Maya::OpenMaya20230000::MHWRender::MShaderInstance::setParameter(Autodesk::Maya::OpenMaya20230000::MString const&, float)
  pxrInternal_v0_23__pxrReserved__::HdVP2Material::CompiledNetwork::_UpdateShaderInstance(pxrInternal_v0_23__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_23__pxrReserved__::HdMaterialNetwork const&)
  pxrInternal_v0_23__pxrReserved__::HdVP2Material::CompiledNetwork::Sync(pxrInternal_v0_23__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_23__pxrReserved__::HdMaterialNetworkMap const&)
  pxrInternal_v0_23__pxrReserved__::HdVP2Material::Sync(pxrInternal_v0_23__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_23__pxrReserved__::HdRenderParam*, unsigned int*)
  pxrInternal_v0_23__pxrReserved__::Hd_PrimTypeIndex<pxrInternal_v0_23__pxrReserved__::HdSprim>::SyncPrims(pxrInternal_v0_23__pxrReserved__::HdChangeTracker&, pxrInternal_v0_23__pxrReserved__::HdRenderParam*)
  pxrInternal_v0_23__pxrReserved__::HdRenderIndex::SyncAll(std::vector<std::shared_ptr<pxrInternal_v0_23__pxrReserved__::HdTask>, std::allocator<std::shared_ptr<pxrInternal_v0_23__pxrReserved__::HdTask> > >*, std::unordered_map<pxrInternal_v0_23__pxrReserved__::TfToken, pxrInternal_v0_23__pxrReserved__::VtValue, pxrInternal_v0_23__pxrReserved__::TfToken::HashFunctor, std::equal_to<pxrInternal_v0_23__pxrReserved__::TfToken>, std::allocator<std::pair<pxrInternal_v0_23__pxrReserved__::TfToken const, pxrInternal_v0_23__pxrReserved__::VtValue> > >*)
  pxrInternal_v0_23__pxrReserved__::HdEngine::Execute(pxrInternal_v0_23__pxrReserved__::HdRenderIndex*, std::vector<std::shared_ptr<pxrInternal_v0_23__pxrReserved__::HdTask>, std::allocator<std::shared_ptr<pxrInternal_v0_23__pxrReserved__::HdTask> > >*)
  pxrInternal_v0_23__pxrReserved__::ProxyRenderDelegate::_Execute(Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext const&)
  pxrInternal_v0_23__pxrReserved__::ProxyRenderDelegate::update(Autodesk::Maya::OpenMaya20230000::MHWRender::MSubSceneContainer&, Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext const&)
  /LD_LIBRARY_PATH/libOpenMayaRender.so(+0xe26ef) [0x7f3891c5d6ef]
  OGSSubSceneItem::update()
  OGSMayaSubSceneManager::UpdateAllSubScenes()
  /LD_LIBRARY_PATH/libOGSMayaBridge.so(+0x1fb86d) [0x7f38e032886d]
  /LD_LIBRARY_PATH/libOGSMayaBridge.so(+0x1fae78) [0x7f38e0327e78]
  OGSApplicationBridge::UpdateScene(OGSApplicationUpdateCallback&)
  OGSMayaRenderer::updateScene(OGSApplicationUpdateCallback&, bool*)
  OGSMayaBaseRenderer::performSceneRender(OGSMayaSceneOperation*, unsigned long, bool*)
  /LD_LIBRARY_PATH/libOGSMayaBridge.so(+0x1f176b) [0x7f38e031e76b]
  /LD_LIBRARY_PATH/libOGSMayaBridge.so(+0x1f8189) [0x7f38e0325189]
  TidleRefreshCmd::refreshOGSRender(T3dView*, bool, bool, TdisplayAppearance, bool, bool, bool)
  TidleRefreshCmd::refresh(T3dView*, TdisplayAppearance, bool, bool, bool, bool, bool, bool)
  TtoolRefresh::refreshFast(T3dView*)
  T3dView::doUpdate(Tevent const&)
  T3dPort::OnPaint()
  QWidget::event(QEvent*)
  QApplicationPrivate::notify_helper(QObject*, QEvent*)
  QApplication::notify(QObject*, QEvent*)
  /LD_LIBRARY_PATH/libExtensionLayer.so(+0x33fc9a) [0x7f38d7e8fc9a]
  QCoreApplication::notifyInternal2(QObject*, QEvent*)
  QWidgetPrivate::sendPaintEvent(QRegion const&)
  QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, QFlags<QWidgetPrivate::DrawWidgetFlag>, QPainter*, QWidgetRepaintManager*)
  QWidgetPrivate::paintOnScreen(QRegion const&)
  /LD_LIBRARY_PATH/libQt5Widgets.so.5(+0x1c5d24) [0x7f38d6d89d24]
  /LD_LIBRARY_PATH/libQt5Widgets.so.5(+0x1c66d0) [0x7f38d6d8a6d0]
  QApplicationPrivate::notify_helper(QObject*, QEvent*)
  QApplication::notify(QObject*, QEvent*)
  /LD_LIBRARY_PATH/libExtensionLayer.so(+0x33fc9a) [0x7f38d7e8fc9a]
  QCoreApplication::notifyInternal2(QObject*, QEvent*)

Expected behavior
Expect no crash when toggling views.

Attachments

crash_toggling_viewports.mp4

Specs (if applicable):

• OS & version : CentOS 7.8
• Compiler & version GCC 6.3
• Maya version : 2023.3
• Maya USD commit SHA: Official v0.28.0
• Pixar USD commit SHA: Official 23.02/22.5

Additional context
I noticed that when turning on texture mode on perspective view, the top/front/side views have texture mode off, I suspect frameContext.getDisplayStyle() inside ProxyRenderDelegate::_Execute() returns different values for perspective view and other views, which is causing the material dirtiness being on and off too frequently, that triggering material resync more often than needed and Maya crashes at some point.

[wallworm]

Thanks for reporting. The team will investigate.

[santosd]

Hello @csyshing , I gave this a try on my end and spent a while testing it but I did not hit the crash. I tested in 2024.2 using 0.28.0 and our Pre Release Maya using a pre release version of Maya USD and I was not able to reproduce the crash. I will see if others can test and attempt to reproduce. I spent a long time in each version, I did notice that memory usage went up, but no crash on my end. @csyshing did you log a CER after the crash? If you get a CER window after the crash, please submit it and I can look it up on my end and see if I can find out more information.

[csyshing]

Hi @santosd , no, I don't have the CER, due to the company security policy I don't think I can submit the log, but let me check what I can do.
In the meantime, if you can think of any other info besides CER that might be useful, do let me know, I will try my best.

[csyshing]

Also, if I change the line in proxyRenderDelegate.cpp#L1143 from:

 ComputeCombinedDisplayStyles(frameContext.getDisplayStyle());

to be like this:

 ComputeCombinedDisplayStyles(frameContext.getDisplayStyleOfAllViewports());

Our users no longer encounter such crash, but I am not sure if this is the right fix (and the API getDisplayStyleOfAllViewports() is only available for Maya-2023+).