Now everyone is starting to realize that stereoscopy (S3D) is cool. Of course big companies are hoping to set a new trend to get more of our money, and the film industry looks at it as if it was the new messiah.

And of course I’m really happy to see this trend rising, this helps me explain my job to my family (“See Avatar? I’m doing that but you can also interact.”).

What’s even more cool is that you can now play S3D games. That’s really interesting, but for me it’s only the first step towards real VR games, and maybe not even a necessary one! We’re not there yet, even with Natal, the PS3 Move or the Wiimote, but we’re on the Starway to VR heaven (c).

But what is really stereoscopy useful for ?

“To perceive depth, dumbass!”

Yeah true, but a one-eyed person can still perceive depth, so stereoscopy is not the only cue to achieve this. And in reality, stereoscopy is not always the best cue; for example it is only relevant for distances up to a few meters.

– Motion parallax, shadows, occlusion, perspective … –

Parallax, the apparent displacement of objects when you move, is another very important cue. Objects close to your eye move faster than objects that are far. And according to Laure Leroy’s PhD thesis (Mines ParisTech, “Interfaçage visuel stéréoscopique : diminution de la fatigue visuelle et caractérisation de la perception des formes“), motion parallax (with head tracking) is at least, if not more, important than stereoscopy ! (Thank you Alexis!)

Monoscopic cues (perspective, atmospheric artifacts (fog), relative sizes, occlusion, parallax, shadows), and stereoscopic cues (eye accomodation and convergence angle, disparity between left and right images) are all very important depth cues. ( “Le traité de la réalité virtuelle” has a very good chapter on depth perception in its first volume, chapter 3.2.3 [3rd edition])

(It’s interesting to note that monoscopic cues are learnt, it’s a cognitive activity, whereas stereoscopy is also a neurological activity.)

Look at these videos, you perceive depth but your screen doesn’t show stereoscopy ! Parallax in action :

The most striking example I have is from my own experience. One day I made a demo in our Cave, where you seat inside a car. While quickly setuping the demo, I didn’t realize that stereoscopy wasn’t enabled. Motion parallax induced by headtracking only was enough to give me a very good sense of depth. Then the whole group of people from the demo tried the car interior and nobody told anything about stereoscopy being disabled. I guess they probably didn’t notice.

Only after David asked me : “Did you realize stereoscopy was off?”. No, I hadn’t ! I then returned to the Cave to understand how I could get fooled. So I switched stereoscopy on and off continuously for 5 minutes. Of course when you know it you see the difference. The distances and objects size seem weird. But I felt very present anyway ! ( Last week I had my stereoscopic capabilities check and their perfect, so it’s not just my eyes!). The fact that is was a car interior, with all objects pretty close is also probably a major factor. We tried on outdoor scenes and the result was much less convincing.

– Made in Sri Lanka with the Fuji Finepix Real 3D W1 and StereoPhoto maker –

Also I think people are not yet used to see stereoscopy on a screen. One day at Laval Virtual a stereoscopic movie was shown every 15 mins and the whole first day the left and right eyes were reversed. Nobody noticed or dared to say a word.

The same happens when you have somebody wear a HMD: they won’t know if stereoscopy is on or off. Only if you turn it on they will notice the difference.

What does this all mean ? Stereoscopy can be expensive to get. If your application doesn’t require precise size or distance evaluation, you can try to just use head tracking without stereoscopy, it might be more than enough !

If your application requires distance or size evaluation, stereoscopy is probably required. Also know that it’s very hard to evaluate an absolute depth, but stereoscopy is very useful to evaluate a very small difference of depth. When seeing an object from 1m, we can perceive a 1mm depth difference. At 10m, we can only perceive a 10cm difference !

On a cinema or movie screen you cannot have head tracking for a lot of people so stereoscopy is a very good addition to perceive depth and improve immersion.

For VR games I’d first go with head tracking, but of course if you can get both .. 🙂