This article explores the foundation of design and how it can be translated into a user experience (UX) for fully immersive involvement in virtual reality (VR), utilizing modern UX techniques to create interactions that are more intuitive, allowing for more immersive engagements. As VR gear becomes more accessible, sleek, and mainstream, it is imperative that developers continue learning new ways to approach UX. Beginning with the concept phase, into design, onward to user types and controls, followed by navigation and safety, and finally concerning implementation, there must be a considerate way to meet the user in their reality to have the most impact.
What is UX?
True engagement is hard to come by unless the quality of the experience is authentic, and the interactions are easy to use and understand, UX is the relationship between the user and their satisfaction with the functionality of a product. In VR, the key to immersion is engaging the mind with realism in movement, function, and quality. Design of the UX must be thought of first, taking into consideration the use case, flow, and intended users. Often confused with the elements that interact with the user interface (UI), UX must be treated as separate, and even more fundamental to a good product.
There are many tools to explore and perfect UX, below are some examples. To understand the audience, a user persona will focus on how a type of user would perceive and functionally experience a design. Flow diagrams enable a designer to map out the process that a user would take through a system and allow for testing with wireframing and prototyping to prove or falsify assumptions. Other choices for tools relevant in creating VR UX include user journey maps, interviewing and research. User-centered design creates personally impactful experiences as a result of implementing such tools in product development.
User Persona
Flow Diagram
Wireframe & Prototyping
How to Explore and Create Concepts for VR
When beginning to develop a concept for VR, many people, especially those new to VR, tend to think that VR sits on top of current experiences like movies, marketing, or videos. These old processes are often not questioned for a new medium and lead to interesting issues, like being given storyboards. If storyboarding is still used then the expectation should be a fluid idea that the content in the storyboard may not be the main point of interest in the experience, given that the user is the driving force. As far as translating a VR experience from thought to concept the foundational supporting items such as modelling and concept art help steer the vision, but often miss important features that users need. There are several other more functional options such as Unity* EditorVR which will allow for in engine experimentation while developing a fleshed out idea for experience functionality; cardboard or brown boxing in which a rapid physical prototype can be created with depth, dimension, and interaction; or recorded video using stand-ins, all of which can provide a better take on how to interact with the environment.
In this example, The MR Configurator Tool from Underminer Studios, you can see some hints toward UX with cues for the user such the push button and a familiar modern tablet interface. These are valuable parts of development, but lack the thorough understanding of the user to accurately predict and design an intuitive experience.
Concept Drawings
Models
To gain some perspective on the roots of entertainment media, let's take a look at video games and movies that take their roots from theatre, in the position of an audience member; whereas VR takes its roots from performance theatre, where each patron is a part of the play itself. Visual trickery and movie magic that would be found used in non-VR media are perceived as cheap and visually unpleasant in VR. Especially with regard to special effects, users are bothered by the sizing and placement of effects in VR more so than others. To make a visually pleasing and spatially accurate experience more akin to a simulation, developers must use a high level of realism for spacing and perception to make more realistic experiences. For example, in an experience where the user is driving a racecar, the need to interact with the steering wheel and gear shift at the same time must be met in an authentic way that makes the user believe they are sitting in a car and can actually perform the tasks necessary to drive.
At GDC 2016, Jesse Schell and Shawn Patton spoke about lessons learned in VR design in their talk, I Expect You To Die: New Puzzles, New Hands. The example below is a clear representation of how brown boxing is such an effective and efficient tool for physical prototyping for VR. Watching a user interact with an environment and props gives great insight into the 'why' of an interaction.
Typography is inevitable and still has no great solution. We currently see a lot of screen-jacking, which forces the view to be blocked by a billboard-style display covering the world and lowering immersion. The most fundamental need here is communication. All UI and UX should be from the environment itself in an ideal scenario, but providing the proper cues is very hard. Audio cues or pictogram-style instructions seem to have supplanted some of the older paradigms, and new interfaces with hand tracking make a much more intuitive interaction that allows for fewer words.
Design Fundamental
Good design involves taking into consideration the intuitive needs of many varied users. No one knows that it is good design per se; they just know that it works easily and doesn't have a steep learning curve. Take children's toys, for example. A well-designed toy will not need an explanation, but instead becomes more of an extended function of the user. Designers expect that things are used a certain way, but instead the use case is not the same and users are frustrated by the functionality. One way to avoid this issue is to playtest, a lot. The more users, the wider the swath of functionality tested and proven or disproven.
Bad design in VR has some negative physical effects such as nausea. Issues like sensory conflict are created by forced camera movements, low-quality animation, frames per second (fps), the rate at which the world is rendered—these can be reduced or removed as issues when UX design is taken into consideration early on. A real-world example here is doors. If a door is designed properly with the correct flat plate for pushing and a handle or bar for pulling, we then would not need push or pull signs or have surfaces that need constant wiping from handprints, but even this most basic daily function is overlooked in design. Can you imagine getting sick every time you encountered a poorly designed door?
In VR, a huge element of design is actually putting on a headset. The colors, scale, and space will look very different from this vantage point versus looking at a monitor. Building a room-scale space which is two meters by two meters can drastically affect the UX with regard to lighting textures and parallax, which are the key elements to creating a sense of depth. An experience built using the capabilities of room scale within the limits of the actual physical space means that the locomotion or movement within the world needs to be carefully chosen. If you build a poor experience that doesn't take space into account, you can find yourself in a metaphorical corner, where users are constantly teleporting around with no focus, never quite sure of the space around them. Environmental cues like directional audio, lighting, clear paths, or glowing objects (also known as inductive design), where the environment itself tells you information, drawing your attention to the important parts of the scene. Other opportunities to make this this more clear are using level of detail (LOD) for contrast in distant objects including changing the appearance to be more faded. Users need feedback from the environment to feel immersed; this can include shadows cast from hands and sound that is responsive to movement. These considerations are important for all experiences, but taking into account the users and their norms is even more crucial.
Designing for Users
Tutorials are very important for every experience to teach users the unique paradigms they need to understand, in order to function in the VR environment, especially in these early days when there are inconsistent norms. The ideal would be task-based practice without the ability to skip, including six degrees of movement—yaw, pitch, roll, and x, y, and z axis—for any input including controllers, hands, visuals, and other elements when creating a standard for each experience. There are three traditional types of users that can be grouped together, and to make a universally enjoyable engagement, the designer should know how to overcome the obstacles for each.
- One user type is the T-Rex; these people put their arms at a 45-degree angle and don't tend to turn or look around. The key is to loosen them up with directional audio and visual cues, and very obvious cues like arrows with timers.
- The second user type is the Ultra-Enthusiastic; they do not read any text and just want to get to game play. They are typically used to playing video games and may be experienced in other VR, but they have the expectation that they will be able to pick it up along the way. The best way to help this user is by forced fun gameplay like tutorials.
- The third user type is the Precise, who reads absolutely everything and does all of it properly. The negative of this user is that they are not having an intuitive interaction, and likely will get lost if the instructions are not exact. In some cases, this user will be able to use critical thinking and get over certain shortcomings, and in other cases this user will get stuck without further explanation. Using very clear cues and the same paradigms is the best way to account for this user.
Universal design is inclusivity for both mental and physical differences and likenesses. Taking into account all learning and bodily abilities and planning for a solution is imperative when creating long-lasting paradigms. Experiences that can be had by all the above user types and used from a seated position with controls that can be adapted to any set of physical limitations will be accessible by a wider audience. As hardware becomes more low profile, intuitive, and available we can really start to break down these barriers for all kinds of users.
User Input Controls
Functional controls can be a hand or any one of the various controllers. Each type of control has distinct functionality, and since they are all less than two years old, implementation does not have an established norm, and users will likely not understand the basics including how to hold, button uses, UI access, or interaction among others. Using pictograms, infographics, text, or hands-on demonstration are necessary until these become more mainstream.
There are several types of controllers specific to individual systems each constructed to be ergonomic, durable, and as familiar as possible as well as custom controllers that are more specialized, like a gun, sword, bicycle, or other props for more realistic interactions. There are suits that allow for a built-in motion capture experience by tracking all movements. Each has its own set of capabilities and negatives. These tend to increase cost and lower accessibility and add another barrier to adoption. Below are some examples of variations. The other consideration with tracking controllers is that the experience must be optimized to track them well or immersion is hindered.
Hand tracking has far fewer constraints and is a way to interact with the system that is the most intuitive and natural. Gestural control is a much faster and more realistic input control. User interfaces that are embedded in the hand functions, like flipping over to the palm, create an easier way for users to get over the learning curve. The engagement increases, and the number of errors decreases. The hardware itself is newer and improving vastly with each iteration.
In the above image you can see the Underminer Studios VR data visualization tool, ManuVRing Data, using Leap Motion controls. We developed this project on a Vive and had the option to choose between the controllers or adding hand tracking capabilities. Given that this was a rapid prototype during a hackathon, we wanted the users to have more intuitive interactions within the system. The judges for the competition had little VR experience and were a perfect example of how to use design thinking for all skill levels. In order to combat a sense of unknown using controllers that are unfamiliar, using one's own hands to pinch, zoom, rotate, select, and manipulate the virtual world allowed for instinctive motions and little explanation needed for the paradigms present in the experience.
Navigation
Navigation is imperative to a realistic experience. Camera angles, frame rates, pace, and locomotion are key factors while acceptance can be intentional or accidental; a good designer plans for both directional and time-based acceptance criteria. Taking into account the user types mentioned above can help a designer prepare for typical issues. Both the T-Rex and Precise users are aware of the environment and of the negatives of the physical experience when they don't follow the paradigms. While the Ultra-Enthusiastic user tends to be far less cautious searching for the more visceral experience, they often do not make the connection that there could be physical harm in real life. Taking this into account is important for navigation and safety.
Gaze-based navigation is when your line of sight is your selection tool and there is no other input needed to change content. In fast-paced environments this can be an issue because the user is limited to one input at a time. This is a great tool for informational, educational, or visual experiences that are turn based or strategy focused.
Teleporting or blink is when a user jumps from one location to another with their vision being obscured by a brief darkening or lightening of the screen; if white is chosen it can increase nausea. Timing between the blink, location, and mental catch-up is also a huge factor here. A slight timing delay between the locations and a very light animation that freezes time for three to five frames, so the user can mentally process the change helps to reduce negative effects. A newer technique is blinking without removing the visual; a slight motion blur and a darkening of the visual with a vignette-style view allows the individual to have situation awareness.
Turn-based navigation is when you take an experience and you augment the amount of forward movement and turning relation that a user believes they are performing versus their visual perception. Essentially the user is walking 50 paces forward and turning visually 180 degrees, but the actual movement is only 90 degrees. This design can maximize playable space. Negatives to this approach are having to be very knowledgeable about three-dimensional math and the limitations of each headset type, including updates per system for tracking. This is a tricky one to get right and must be executed precisely to limit adverse effects.
Auditory cues are core to making a proper VR experience. A user cannot look everywhere, and the expectation is that people are not looking in the right direction at all times. So, in order to attract their attention and create more opportunity and an inductive experience, leveraging audio design and 360-degree directional sound will add cues that a user can follow naturally. Another option is to add psychoacoustic audio, which registers above and below a human perceptible hearing range and creates an involuntary emotional response; this can lead a user to pursue or even avoid specific interactions.
User Safety and Comfort
This is, unfortunately, typically an afterthought. A good designer will consider the long-term ramifications of a user's actions in VR and plan to avoid or reduce them where possible. For instance, people do not naturally know how to look at their feet by bending at the waist in VR. An intended path that is not followed can cause nausea; using visual cues to stop a user at a visual barrier instead of a physical barrier like clipping, when someone bounces into a wall and off of it, is not a good mechanism for VR. The most important safety mechanism is the chaperone bounds, a grid that warns of physical bounds of the play space, which is there for safety. It is important to add these basics into a tutorial as well, so that the user knows how to react to these elements during an experience.
Implementation
Virtual reality is an exciting new frontier that needs a lot more exploration. When designing for 3D interactive experiences there are a lot of influences from psychology, to architecture, sound, physics, lighting, and so on. The more comprehensive look at the design influences, the better. To be successful at creating impactful VR experiences, a designer must have an awareness of the hardware differentiations and limitations that will directly affect the optimization needs and shape the overall experience. Since we are so early in the lifecycle of VR this generation, there is still a lot of experimentation needed to continue creating a comprehensive best practices guide. For now, adhering to user-centered design, beautiful and realistic experiences, and optimizing to maximize an experience will be the fundamentals.
About the Author
Alexandria Porter is the is CEO of Underminer Studios. She brings a grounded perspective to a fast-paced business on the leading-edge of technology with small business experience, design prowess, and management skills. As a solution-focused company, we serve clients that seek to leverage technology to push boundaries and change the perspective of how to solve real problems. Utilizing more than a decade of experience, strong industry connections, and out of the box thinking to create unique products, Underminer Studios is driven by a passion for impactful uses of technology that will shape the future.