[This is a re-post of the original Gamasutra article which was published on 12 Jan 2014]

Introduction: presence

Through my career I have tested many different VR systems, from entry level to high-end, with systems costing several million euros. I have developed a feeling of what VR represents for me. This feeling of being present in the virtual world is very strong. I have already defined “presence” at length in this article. To summarize, you have cognitive immersion (like in video games, stories, books..) and perceptive immersion (fooling the senses), which is the part that makes VR distinctive.

For me, there seem to be two options:
– your brain thinks the simulation is a regular video game with just a fancy display (3D and/or 360). You will not feel very involved with the simulation. This is not VR, even if you have a very expensive system.
– your brain accepts this simulation as reality and will react as such: natural reactions, natural interactions, natural emotions. You will have fear of heights, you will try to catch an object thrown at you, you will fear for your life! You feel present. This is VR, even if the system is cheap.

A perfect VR experience should make you feel present for the whole time. Recreating a reality that can fool your brain is hard to achieve, so VR game designers should at least try to minimise the breaks in presence.
What makes you feel present and what breaks it is still a big research topic. As we are working with the human brain this feeling of presence is also highly personal. People who have never tried VR can be overwhelmed quite easily. The same happened a century ago: one of the first movies was showing a train entering a train station and people in the cinema ran away thinking this was really happening.

[Although this may be a myth].

[Update 17/01: Oculus has just released a “Best practices” which is very interesting! It is quite low-level though, this article is about higher-level interactions.]

Current VR games

The Oculus Rift is the first low-cost VR system to provide this presence feeling “easily”. I have been lucky to test it before the famous Kickstarter, back when it was all duct taped and presented in a shoe box. I tested Doom 3: the immersion was great, but I got sick really quickly. Even before that, I didn’t really feel present in the game. Neither did I feel present in Team Fortress 2 or Half-Life 2. My brain did not accept the simulation as reality. I felt like having a 360° view, but not that I was actually there. In particular the interactions were quite poor and non “natural”: sitting in a chair, interacting with a keyboard/mouse/gamepad, moving at high speed does not feel natural. Those games are perfect for desktop PC (I spend a lot of time in those), they are not perfect for VR in their current form. The content was not designed to maximise presence, which should be the first goal of any VR application. Without presence, I wouldn’t call it VR.

I have tested several of the new VR games and experiences recently released and haven’t felt present in much of them. My “presence barrier” has unfortunately been pushed far.

[Update 17/01: I do not mean this as condescending. I’m extremely happy that so many people are joing the VR cause, and this article is all about providing constructive feedback.]

The notable exceptions are the ones where you are in a vehicle: Titans of Space, the UDK Roller Coaster, etc. The reason they work well is because there are no interactions: you are seated in the simulation as you are seated in real life, and the interaction is limited to (automatic) navigation. It is easier to deliver what your brain expects. Hawken and EVE: Valkyrie will probably also work because you are in vehicles, whether it is a robot or a spaceship.

The case of “The Gallery: Six Elements”

There are very few experiences where you are interacting naturally with your environment. The Gallery: Six Elements / exploration school preview is a very encouraging and ambitious experiment: I felt present at several points, but unfortunately not for the whole experience. The Gallery was successfully funded by Kickstarter. “Influenced by the Myst series, The Gallery: Six Elements is a transcendental adventure with a heavy emphasis on environmental immersion, exploration & path-finding, physical challenges, puzzle solving, and emotional depth. (…) [it] is being designed from the ground up to immerse players as deeply into the experience as possible. No cut scenes, no 3rd person, no unnatural interruptions to the game experience.”

I would like to take some time to discuss some interactions in the game, because it is exactly the kind of game I want to see in VR. There are a lot of things to learn from it already. You can purchase the pre-order directly here (which gives you access to Alpha, Beta and Release content). I also encourage you to greenlight the project on Steam here.
I have been lucky to talk to Denny Unger about some of the points I want to highlight. Denny is the president and creative director from Cloudhead Games, editors of The Gallery.  I have included some of his answers.

Denny Unger, creative director at Cloudhead Games

The good

• Great graphics! Really great to see that pushed forward even though it is not necessary for a good VR experience. Denny’s answer: “We just brought on talent from Blizzard to further polish our assets!”
• Seeing your body and hands is totally awesome. Seeing the shadows is definitely great and important for consistency.
• Help and menu on a tablet: really great. It gives a real-life analogy rather than floating GUIs.
• Objects behave in a coherent physical way: they fall, they break. This is how you expect physics to happen in real life. Even if it’s not perfect, it is very acceptable.
• The sound is also great and really important: it really adds to the immersion and coherence between what you see and what your brain expects. Denny’s answer: “Sound design is hugely important to a solid VR experience. We were lucky enough to recently bring on Joel Green from Bioware (Mass Effect, Dragon Age) to really push the limits there. Should be interesting!”
• All small experiments are really cool: paintball, graffiti. Especially the climbing part is awesome and feels very natural (even for me practicing real life mountain climbing). The only issue is that you quickly reach the tracking limits of the Hydra.

The less good

• Rotations are too fast. I was reall really close to throw up (I’m very sensitive). Denny’s answer: “Rotation speed is something we played with quite a bit. If rotational translation is too slow it makes people sick, too fast it makes people sick. So you have to find this balance in the middle and it doesn’t work for everyone. Its really something you have to acclimate to.” I still think having an option to change the sensitivity would help.
• Lots of objects that are in the game are supposed to be interactive in real life: drawers, switches, … but you can’t interact with them in the game. This is very frustrating and definitely breaks presence. Those objects should either be removed or there should be an an excuse so that it is obvious they can’t be interactive, or there could simply be dummy interactions (like flipping the switch with no effect). Denny’s answer: “Absolutely. It was more of an issue of running out of time before launching the Alpha. We plan on being more diligent with interactions.”
• I found it strange that the hand collision makes the body move back. This is particularly problematic when you are trying to reach an object on a table for example. If you are not careful it is impossible to reach it. It would be better to constrain the movement of the hand. This would also provide some “pseudo haptics”, touch feedback through vision. Denny’s answer: “We actually tried it both ways. This is a tricky one to explain but it has to do with the interactions between IK, Mechanim, Unity physics and our unique implementations for climbing and other behaviors. When we tried hand constraints Unity gave us massive issues that we could not overcome at the time. That being said, I’ve never been a fan of the current implementation. We’ll be digging deeper into that one for sure.“

• Because of this it is extremely hard and frustrating to grab objects. Most of the time you touch the surface beneath before, which makes your body go back. Also you tend to make objects fall. This also definitely breaks presence. Denny’s answer: “Positional tracking is going to help us immensely on that front. Once that is in play, leaning in for object grabs suddenly becomes a non-issue.”
• Highlight of objects: I’m not sure if this is a good thing or a bad thing. This is not realistic, and in light of previous points I would say everything that seem interactive should be, so no highlight should be needed. But this does not seem critical. Denny’s answer: “We plan on making this an optional feature for beginners. We haven’t yet implemented our complete menu settings. I agree though, its not an option I would personally select for my particular play style. I want full immersion wherever possible!”
• I would like to see the use of buttons limited to its absolute minimum. Denny’s answer: “Hopefully we can limit the number of button interactions once we have STEM to work with. That being said because the game is also playable without an HMD and with gamepad/Keyboard compatibility, you have to build in conventional button options.” Maybe using the menu from the virtual tablet could help reduce that, or using gestures ?
• The reflection in the mirror is missing. This is a minor issue but it still breaks presence. Denny’s answer: “Agreed. Again, Unity gave us some issues with occlusion culling on the mirror that we couldn’t fix prior to the Alpha deadline. You can however see your reflection in the water in the sewer.”

• It is hard to read text on the tablet with a non-HD Rift. Denny’s answer: “We made the fonts as large as possible for readability on DevKit 1 while still giving us a pleasing interface. We brought it as close to the player as we could to help with reading. We are limited by DK1’s low resolution but, at the end of the day, this is a commercial game being designed for the commercial Oculus with a much higher resolution.”

This is still the most promising experience I’ve seen so far, and a several of those points can be easily addressed. A big thanks to Denny for taking the time to answer!

Conclusion: presence is in the details

Adding interactions in VR is hard because it is very easy to do something “wrong”, resulting in breaks in presence. Creating a realistic environment is even harder because your brain knows exactly how physics, sounds, shadows, interactions etc. are supposed to behave. If they don’t behave in a coherent way, you will be reminded that this is “just” a simulation, and not a possible reality.

As we are still in the prehistory of VR, I would advise starting with simpler environments. Recreating a whole new reality is a long task, and presence is in the details. If you create a complex and rich environment, you will have more chances of introducing inconsistencies.

Finally keep in mind that the simulation does not have to be realistic as long as it is consistent. A cartoon has its own unnatural rules, but you still accept them because they are consistent.

It is probably time to watch “Who framed Roger Rabbit” again!