Trying to integrate this with AR and hand gesture controls

[balavenkataraman123]

I have been working on a project which finds your location in 3d space using only the laptop webcam, and uses that to render a CAD model or game from that perspective, and create an effect similar to a 3d screen (but it only uses the laptop screen and webcam). I have also built hand gesture-based computer control programs using MediaPipe, and want to build something like the thing Tony Stark uses to design his creation in the Iron Man movie (https://www.youtube.com/watch?v=DZaAFADoF1M)

While I have currently experimented with a basic OpenGL program viewing a simple obj file test to see how well the effect works, I would like to try to integrate this with a CAD software. This one is appealing because it can work with a lot of types of files, is open source, and runs on Linux.

So, to develop this, do you have any advice on what is the best way to modify the rendering (such as setting the view frustum) and UI input code to work with that program? Thanks a lot in advance :)

[HuguesDelorme]

You can change the view frustrum with OpenCascade Graphic3d_Camera. Each 3D view has an object of that type.
The 3D view is owned by gui/GuiDocument, the most straightforward in your case is to configure what you need in gui/GuiDocument's constructor(just after L102 in src/gui/gui_document.cpp):

    // 3D view - Set gradient background
    m_v3dView->SetBgGradientColors(
                GuiDocument::defaultGradientBackground().color1,
                GuiDocument::defaultGradientBackground().color2,
                GuiDocument::defaultGradientBackground().fillStyle
    );
    //m_v3dView->SetShadingModel(Graphic3d_TOSM_PBR);

    // BEGIN OF YOUR CUSTOMIZATION
    m_v3dView->Camera()->SetProjectionType(Graphic3d_Camera::Projection_Perspective);
    // ...
    // END OF YOUR CUSTOMIZATION

    m_cameraAnimation->setView(m_v3dView);

Not sure what you mean by "UI input" but I understand it as "user input to control 3D view" like eg rotation, panning, ...
This is handled by base class gui/V3dViewController and its redefinition app/WidgetOccViewController for Qt

Feel free to ask for more information if you're stuck

[balavenkataraman123]

Thanks a lot! This is the exact information I was looking for.