Latency

Latency is an important topic for VR.

Latency in a typical Cave is around 50ms, but is globally in the range of 30-80ms.

The end-to-end latency is mainly impacted by the following elements:

- Hardware tracking latency (hardware + windows drivers) : with a fast tracking like ART, it is typically around 10ms. Make sure to run the tracking as fast as possible. With ART you can also enable prediction to reduce latency:

- MiddleVR kernel update: gather info from the trackers' drivers, update MiddleVR nodes (position, orientation, cameras' perspectives. Typically around 1ms.

- MiddleVR's cluster synchronization: MiddleVR synchronizes the information from all its devices / nodes / camera / time / frame / vrCommands from the server to the clients. This is mostly dependent on the latency of the network (ping from server to client).

- User code & Unity rendering: It is possible that using deferred rendering adds some latency. Try to prefer forward rendering.

- (Optional) MiddleVR's post-frame cluster synchronization: position orientation of Unity nodes that have been manually asked to be synchronized by using the VRClusterObject script and physics nodes if the "SimpleCluster" option is set to true. Again mostly dependent on the latency of the network.

- MiddleVR compositor: Flush the DirectX rendering pipeline, "grab" Unity texture (typically <0.5ms per texture, especially with the new compositor mode Direct3D11). Note that flushing the DirectX pipeline actually reduces the perceived latency: instead of starting a new Unity frame right after the end of the current frame, which would cause the sampling of the tracking information too early, we wait for the end of the cluster synchronization to start the new frame, so the tracking information is sampled later and is thus more up to date.

- Warping: Warping is done in the compositor. As this is mostly texture sampling this should take a negligable amont of time as long as there is enough GPU RAM to maintain the warping maps. Warping SDKs are often used in flight simulators which require very low latency.

- Windows 10 compositor: adds one frame of delay.

- SwapLock: Wait for all cluster nodes to finish their rendering then display: dependent on the rendering load of the cluster nodes and the network latency.

- Hardware display latency: the time it takes for the hardware (graphics card + projectors / screens) to actually display the rendered image.

• Graphics card: Make sure that the "Triple buffering" setting in the NVidia Control Panel (3D Settings > Manage 3D settings) is off.
• Projectors: Make sure your projectors / screens don't buffer any frame. Typically a projector will add 1 frame delay.
• If you have some hardware between the graphics card and the projectors, like a matrix controller, they also typically add 1 frame delay.

The MiddleVR logs should give you indication of some of those values  in the logs. Make sure to increase the log levels to at least 3, but note that if you get to higher log levels will slow down the process.