How to render MuJoCo over an existing OpenGL scene with transparent background?

[HongtaeLee-97]

Intro

Hi!

I am a researcher, I use MuJoCo for my research on robot motion verification.

My setup

MuJoCo 3.3.4 / Windows / (MFC, WGL/HDC/HGLRC), OpenGL / C++

My question

As soon as I call mjr_render, the background becomes black and covers the previously drawn point cloud. I would like MuJoCo to not paint/clear the background (i.e., render just geoms), or to be transparent over my already-rendered content.

Is there a supported way to make MuJoCo not clear / not paint the background in mjr_render, i.e., render only the geoms on the current framebuffer?

Is there a setting (flag/option/material) that makes the background transparent or skips skybox/clear completely?

Any example code for integrating MuJoCo rendering into an existing OpenGL pipeline (WGL, not GLFW) without letting it overwrite the color buffer background?

The first picture is a point cloud, and the second picture is a rendered mujoco model while maintaining the point cloud.

Minimal model and/or code that explain my question

used xml : https://github.com/google-deepmind/mujoco_menagerie/tree/main/universal_robots_ur5e

Code:

// MFC OnDraw
ctx->Begin();                        // wglMakeCurrent
glDrawBuffer(GL_BACK);
glClearColor(0.92f,0.92f,0.92f,1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

// 1) My scene (point cloud) -> draws fine
m_Renderer->RenderScene();

// 2) MuJoCo (same back buffer)
mjv_updateScene(model, data, &opt, &pert, &cam, mjCAT_ALL, &scn);

// disable background-like effects
scn.flags[mjRND_SKYBOX]     = 0;
scn.flags[mjRND_FOG]        = 0;
scn.flags[mjRND_HAZE]       = 0;
scn.flags[mjRND_SHADOW]     = 0;
scn.flags[mjRND_REFLECTION] = 0;
scn.framewidth = 0;

// camera: I build mjvGLCamera from my app camera (mm->m) and assign:
// scn.camera[0] = glcam; scn.camera[1] = glcam;

scn.enabletransform = 0;             // for sync btw mujoco & point cloud unit scale 
mjr_setBuffer(mjFB_WINDOW, &con);
mjr_render({0,0,vpW,vpH}, &scn, &con);   // <-- after this, background turns black

ctx->Swap();
ctx->End();

Confirmations

• I searched the latest documentation thoroughly before posting.
• I searched previous Issues and Discussions, I am certain this has not been raised before.

[atyshka]

@HongtaeLee-97 Did you ever find a solution?