Multiview Rendering

Multiview rendering refers to instancing draw calls into layers of a 2D texture array. In Qt, it is relevant, in particular for VR/AR applications built with Qt Quick 3D Xr. Instead of independently doing scene traversal, rendering preparations, and render pass recording for the left and right eye's content, multiview rendering allows doing it once, with the draw calls in the single render pass being instanced into layers 0 and 1 of a texture array. The vertex shader uses a special variable indicating the view index and computes per-view values based on that. Uniforms that contain view-dependent data, such as camera matrices, must be provided for both eyes. Multiview rendering is expected to decrease the renderer's load on the system, potentially leading to better performance. It comes at the expense of having to make the renderer and shaders aware of working with view-dependent data and texture array as appropriate.

Low-Level Overview

Qt application developers do not necessarily need a full understanding of how multiview rendering is enabled on the lower levels of the Qt rendering stack. The following links are provided for developers wishing to look more into the details under the hood.

Multiview support in 3D APIs

Multiview rendering is available only when the underlying 3D API supports it at run time. For details, see the appropriate specifications and documentation:

OpenGL: OVR_multiview1 and OVR_multiview2
Vulkan: VK_KHR_multiview
Direct 3D 12: ViewInstancing
Metal: rendering_to_multiple_texture_slices

Multiview support in Qt's rendering hardware interface (RHI)

In Qt, the 3D graphics APIs are abstracted by the QRhi class. Qt Quick and Qt Quick 3D uses this infrastructure for all their accelerated rendering. See the following for further low-level information on multiview rendering support:

Multiview Support in the Qt Quick - Quick 3D - Quick 3D XR Stack

Multiview support in Qt Quick

Qt Quick is multiview-aware, but is never using multiview rendering on its own. Supporting multiview rendering becomes important in combination with Qt Quick 3D when 2D elements are embedded into the 3D scene. To render the 2D content correctly when the 3D scene is output to a multiview render target, the 2D renderer and its materials (shaders) must be prepared for multiview support.

Developers of Qt-based applications do not need to take this into consideration in many cases because Qt Quick's built-in materials that items such as Rectangle, Image, or Text are built on are all multiview compatible.

However, when developing custom materials (QSGMaterial, QSGMaterialShader) or writing shaders for ShaderEffect, and the intention is to use that 2D content within a VR/AR scene in a Qt Quick 3D Xr application, the custom content must be multiview-aware. see QSGMaterial::viewCount() for details on this.

Writing multiview-aware shaders is enabled by Qt's shader conditioning pipeline. See the Multiview sections in QSB Manual and Qt Shader Tools Build System Integration for details.

Multiview support in Qt Quick 3D

Qt Quick 3D applications that do not use Qt Quick 3D Xr, meaning they are not VR/AR applications, cannot currently use multiview rendering. The 3D renderer is fully multiview-capable, however, since Qt Quick 3D Xr is built on the same infrastructure. All standard, built-in features, such as Model or PrincipledMaterial are fully multiview compatible. Some deprecated functionality, such as the old standalone effects module, may not support multiview rendering.

When custom shader snippets are involved in a CustomMaterial or Effect, the application-provided shader code needs to be written with multiview support in mind in order to function correctly in a Qt Quick 3D Xr application with multiview rendering enabled. See the documentation of these types on how to achieve this. The special keywords for which this is particularly important are VIEW_INDEX, INPUT, SCREEN_TEXTURE, DEPTH_TEXTURE, AO_TEXTURE.

For example, the following postprocessing effect is multiview compatible, because it is prepared for the case when INPUT is a sampler2DArray instead of a sampler2D:

 void MAIN()
 {
     vec4 c = texture(someTexture, TEXTURE_UV);
     // ...
 #if QSHADER_VIEW_COUNT >= 2
     FRAGCOLOR = c * texture(INPUT, vec3(INPUT_UV, VIEW_INDEX));
 #else
     FRAGCOLOR = c * texture(INPUT, INPUT_UV);
 #endif
 }

Multiview support in Qt Quick 3D Xr

Multiview rendering is disabled by default in Qt Quick 3D Xr applications. This is done to ensure the best level of compatibility. To enable multiview rendering, set multiViewRenderingEnabled on XrView to true. This will have no effect when the underlying 3D API does not support multiview rendering. To query if multiview rendering can be supported, check the multiViewRenderingSupported property.

For development and testing purposes, it can be useful to force the usage of multiview rendering via an environment variable. This is done by setting QT_QUICK3D_XR_MULTIVIEW to a non-zero value.

To query if multiview rendering is in use (regardless of how it was enabled), use the same multiViewRenderingEnabled property. The value stays false always if multiview rendering cannot be enabled.

In general, it is recommended that VR/AR applications enable multiview rendering unconditionally via the XrView property once rendering is verified to work as expected in multiview mode during the development phase. Multiview rendering is expected to improve performance, reducing the GPU and especially the CPU load.

Contents