Nomadic virtual reality : overcoming challenges of mobile virtual reality head-mounted displays

Abstract

Technological advancements in the fields of optics, display technology and miniaturization have enabled high-quality virtual reality (VR) head-mounted displays (HMDs) to be used beyond research labs and become available as consumer products. In turn, this development enabled mobile VR HMDs, which are untethered and self-contained headsets, allowing users to immerse themselves wherever and whenever they wish. This creates a novel interaction scenario in which a user is immersed in a virtual environment using a mobile VR HMD inside an unknown context (e.g., watching a 360-degree video while commuting by public transport).

This thesis defines this novel interaction scenario as nomadic VR and systematically explores its upcoming challenges and opportunities. For this, the interaction scenario is embedded into a larger vision of ubiquitous mixed reality, using models and approaches from the field of context-aware computing which already explain a similar transformation and paradigm shift from stationary PCs to mobile computing (smartphones): The form factor changed dramatically, cursor-based input was replaced with multi-touch, sound and visual feedback was extended with vibration and the constant changing environment enabled a variety of location-based features and services. We argue that a similar transformation will happen from stationary VR HMDs to mobile VR HMDs: the input will be adapted, novel output modalities will be added and the context of use will be incorporated into the virtual environment.

This dissertation consists of six case studies, each addressing one aspect of these challenges (input, output and context). To enable fast and precise input we present FaceTouch, a novel interaction concept leveraging the backside of the HMD as a touch-sensitive surface. FaceTouch allows the user to select virtual content inside the nomadic VR interaction scenario without the need for additional accessories or expansive gestures. To extend the output capabilities of mobile VR HMDs, we propose GyroVR, a set of HMD-attached flywheels, leveraging the gyroscopic effect of resistance when changing the spinning axis of rotation and generating the perception of inertia. GyroVR was designed as a mobile and ungrounded feedback device fitting into the nomadic VR interaction scenario.

The context was divided into the physical environment and human factors. With CarVR, we explored how to enable the usage of VR HMDs inside of moving vehicles such as cars. The CarVR system senses and incorporates the additional motions arising inside of these dynamic physical environments, enabling an increment of enjoyment and reduction of simulator sickness compared to a stationary setup. The SwiVRChair system presents a motorized office chair, exploring how everyday objects inside a static physical environment can be incorporated into the nomadic VR interaction scenario to enhance the overall user experience. Since the nomadic VR interaction scenario often takes place inside of public environments, for the human factor context we focused on social scenarios in which people use VR HMDs when people without HMDs (non-HMD users) are in the vicinity. With the ShareVR system, we present a prototype which uses floor projection and mobile displays combined with positional tracking to visualize the virtual world to (non-HMD) users and enable an asymmetric interaction. In a followup case study, we adapted the ShareVR concept to fit into a mobile VR HMD. FaceDisplay is a modified VR HMD that consists of three touch-sensitive displays and a depth camera attached to the back of the HMD, allowing the non-HMD user to perceive and interact with the virtual world through touch or gestures.

We conclude this dissertation with three overarching findings that resulted not out of the individual research questions but emerged throughout the whole process of this thesis: (1) We argue that current HMDs are mainly optimized for the wearer and ignore the whole social context; future HMDs have to be designed to be able to include non-HMD users. (2) We show that the physical environment should not only be seen as a challenge, but can be leveraged to reduce problems such as simulator sickness and increase immersion. (3) We propose that similar to the very first smartphone, current HMDs should be seen as an unfinished device type. We argue for an engineering research approach that extends the current form factor through novel sensors and actuators.