Virtual supermarket in Oculus Rift

New media, new technology and new visual languages have the ability to create compelling and immersive new worlds. We can view these worlds on computers, devices and in the (home) cinema. With new Virtual Reality goggles reaching consumers in 2014, we will see a big leap forward in immersiveness and easy of use of these virtual worlds. Businesses increasingly use virtual worlds to test products and services that do not yet exist. However, to maximize the business potential, we need to know where the user's attention is and what their emotional reactions are to the virtual world. The unique value of this proposal is that we will be the first to combine psychophysiology (the measurement of emotional reactions) and eye tracking with virtual reality goggles. As such we will be able to observe behaviour in the virtual world to predict behaviour in the real world

With 3D technologies, we can see depth and sense that virtual worlds are not just flat pictures. But 3D cinema and TV are mediated experiences, not user directed. To look around in 3D game worlds, a controller is necessary and this decreases the immersive experience. Virtual Reality goggles solve this: The glasses contain screens in front of the eyes that create a 3D effect. The hardware enables the user to move his or her head exactly as if in reality. This creates a compelling sense of being inside the virtual world.

Research versions of VR goggles have existed for some time, but these were too costly. A new company Oculus VR (oculusvr.com) is set to launch consumer-grade goggles in 2014, at around 300 USD (Figure 1). They already sold 20,000 developer units; the NHTV has acquired one. VR goggles create an enormous potential for businesses: New games have been written for it (riftenabled.com) and new media will undoubtedly appear for it. New visual metaphors are formed and needed to help users navigate this new domain. The importance of visual style and visual design is increased further. In the words of one developer: "with VR, I feel that the atmosphere of the location is at least as important as the game idea itself."

In the last 4 years, once costly 3D simulations are becoming affordable due to developments in gaming. Physical products, from package designs to building layouts, can be created virtually for examination on a computer screen. NHTV is at the forefront of this: We have four years of experience in designing simulations and creating virtual worlds, such as our flagship, a complete supermarket simulation (Figure 2). Here, shelves can move and be restocked at the click of a button and new products, concepts and services are designed and tested before they reach consumers.

Many companies have used our Virtual Supermarket for research or product development. Until recently, they had to use our CAVE (a large room with four walls of video projection). The major critical reflections we hear from industry is that a large CAVE like the NHTV has is not efficient for them because of its size and the costs involved. In-company departments and participants would spend too much time traveling to a central CAVE. To really make use of virtual environments, they need a more flexible solution that reach professionals (such as visual artists) in the company and consumers at home. VR goggles solve this problem.

When using VR goggles for development, showing a virtual world is just step one. Companies want to obtain measurements of in-world behaviour, of consumer interaction with objects and their emotional responses. To gauge object interaction, eye tracking is the leading solution (without attention no effect). To test a new visual design, an eye tracking heat map is an invaluable tool (Figure 3). Regarding gauging emotional responses, NHTV has 4 years experience with psychophysiology, the measurement of the (emotional) reactions of the body. Together, the user experience is charted from information uptake (eye tracking), via emotional responses (psychophysiology), to subjective evaluation (questionnaires). Technicians and designers use these insights to build new worlds; marketing and development use them to create new products.

The challenge posed by this proposal is to combine these methods into one affordable and user friendly package. Our main objective is to provide companies with an enhanced version of VR goggles that add eye tracking and psychophysiology. Secondly, we will provide virtual worlds for companies to embed their virtual products in. Finally, analysis software will present the results in an accessible format.

The technological challenge we take up is to combine eye tracking with VR goggles: Because the goggles are very close to the eye, the techniques used in commercial eye trackers cannot be used. Our knowledge of eye tracking, psychophysiology, and the needs of companies puts us in a unique place to further develop the solution we have for this problem.

The introduction of VR goggles, combined with the goals of this project, raised the interest of many potential international partners from the retail environment and research companies. We have had visits and requests from retail, research and design organizations, ranging from large international companies to local SMEs (a list can be obtained from NHTV).

The proposed research will create new jobs and methods for research, art, and design. In the 2012 global market, research turnover was estimated to rise, reaching US $33.5 billion (+3.8%). Europe has the largest research (42%). The project will create new jobs once we take into account the trend of selling products via virtual stores and virtual environments (eg. Tesco, Figure 4). Finally, the technology will be used by schools and knowledge institutions to prepare students for jobs that may not currently exist, such as virtual store management and virtual-reality concept design artists.

In sum, our project will have six deliverables:

  • D1: Add psychophysiology and eye tracking to VR goggles;
  • D2: Create visual training and calibration routines;
  • D3: Automatically present results in a user friendly manner;
  • D4: Run a validation study in the virtual supermarket;
  • D5: Distribute knowledge, vr worlds and methods to market;
  • D6: Embed the findings in teaching at NHTV and elsewhere.