Support for using stereoscopic video as textures

[nlsdvl]

Context

Our goal is to implement stereoscopic video texture for content playback in a virtual movie theater scene. MV-HEVC could be used for encoding such video content, but note that the problem applies for any stereoscopic content.

Problem description

Stereoscopy achieves the illusion of depth by providing a pair of images each tailored either for Left or Right eye. When rendering stereoscopic content in an HMD, the presentation engine uses the appropriate view from the stereoscopic video for each eye.

I couldn’t find any way, neither in the base glTF specification, nor using the MPEG-I SD extensions, to achieve signaling of stereoscopic media that would allow implementing our scenario.

The presentation engine needs to know which image/texture/buffer should be used when rendering either for Left or Right eye.

Suggested solution

Extend MPEG_video_texture to allow the presentation engine to render only the appropriate images for each eye.

One option would be to use two textures that differ only by their accessor, with an extra property signaling which eye it is intended for.

Optionally, specifying a default a.k.a. ‘hero’ eye could be useful when not rendering the scene on an HMD.

[ibouazizi]

Thanks Nils for pointing this problem out. We will address it hopefully in the next possible amendment.

[haudiobe]

An input is provided: https://git.mpeg.expert/MPEG/Systems/SceneDescription/MPEG-Contributions/-/issues/794. We will check after MPEG#149