Controlling viewpoint from markerless head tracking in an immersive ball game using a commodity depth-based camera

We present the use of commodity depth‐based cameras to control viewpoint from passive unencumbered head tracking. This allowed participants to maintain correct perspective while playing an immersive computer ball game, without the need to wear devices, wires or markers. Viewpoint update drives parallax that supports orientation, the judging of distance and scale, moving in response to the simulation and the feeling of presence. Its incorporation increases the validity of using simulations in applications in which any of these are important. Although updating viewpoint from a depth camera has been demonstrated, it has not undergone accurate measures of latency, task performance or user experience. A randomised within subjects trial of n=12 participants was used to determine if head and hand tracking from depth‐based cameras helped the playability and experience of a multi‐player ball game. Findings suggest that experience is only improved when both ball interaction and viewpoint are controlled through natural movements and that the quality of viewpoint update from a depth‐based camera was sufficient to allow a ball game to be played without inducing disorientation or nausea. This work demonstrates both a potential benefit and risk of using depth‐based cameras to update viewpoint.