Latency is an important topic for VR.
The end-to-end latency is mainly impacted by the following elements:
- Hardware tracking latency (hardware + windows drivers)
- MiddleVR kernel update: gather info from the trackers' drivers, update MiddleVR nodes (position, orientation, cameras' perspectives. Typically around 1ms.
- MiddleVR's cluster synchronization: MiddleVR synchronizes the information from all its devices / nodes / camera / time / frame / vrCommands from the server to the clients. This is mostly dependent on the latency of the network (ping from server to client).
- User code
- (Optional) MiddleVR's post-frame cluster synchronization: position orientation of Unity nodes that have been manually asked to be synchronized by using the VRClusterObject script and physics nodes if the "SimpleCluster" option is set to true. Again mostly dependent on the latency of the network.
- Transfer texture from DirectX to OpenGL: Flush the DirectX rendering pipeline, convert the texture (typically 0.5ms per texture). Note that flushing the DirectX pipeline actually reduces the perceived latency: instead of starting a new Unity frame right after the end of the current frame, which would cause the sampling of the tracking information too early, we wait for the end of the cluster synchronization to start the new frame, so the tracking information is sampled later and is thus more up to date.
- SwapLock: Wait for all cluster nodes to finish their rendering then display: dependent on the rendering load of the cluster nodes and the network latency.
- Hardware display latency: the time it takes for the hardware (graphics card + projectors / screens) to actually display the rendered image. Make sure that the "Triple buffering" setting in the NVidia Control Panel (3D Settings > Manage 3D settings) is off. Make sure your projectors / screens don't buffer any frame.
The MiddleVR logs should give you indication of some of those values in the logs. Make sure to increase the log levels to at least 3, but note that if you get to higher log levels will slow down the process.