22:04 
Virtual Reality
15:29 Virtual Reality
Back at Laval Virtual, David did a lot of videos for internal DS communication of applications made with Virtools VR Pack. They have now released the videos on Youtube here!
Warning, it’s a bit corporate 
I’ve also added some of those to the State of VR .
Here are some :
9:24 VR Applications
Lutz Jancke, from the Neuropsychology lab of Univeristy of Zurich, made an amazing presentation about the reality of VR to our brain. The short answer is for the brain, VR is just another reality; the brain only interprets inputs from its senses and experience, and VR is providing inputs that can be realistic enough to fool the brain. Moreover, VR experiences can shape your brain!
The brain evolves during all your life
The human brain is highly constructive, and constructs reality with the input it gets from the different senses. Perception of our world is a matter of interpretation by your brain of these different inputs.
Studies have been conducted on twins that were separately fostered to know what is the influence of the genes on intelligence. It turns out that only 50% max of your intelligence comes from your genes; this means that 50% of intelligence comes from experience!!
It also appears that much of the grey matter that makes us humans (visual sense, language etc..) is not determined by genes. In fact the brain is largely prepared to learn, it’s a giant learning machine that is able to learn during the whole lifetime of a human. During all your life your brain is restructured based on what you do, what you train at, for example music, juggling etc. But your capacities also decrease when you stop practicing.
Grey matter density can increase with elder people too. Aging doesn’t prevent learning; the cognitive aspects of learning of an elder people is comparable to youths.
Emotions
There’s a zone in your brain that is a strong indicator of presence in a VE : the right-sided dorsal prefrontal cortex (DLPFC). The stronger this zone is activated, the less you feel present. This zone is in fact controlling your lymbic system which is responsible for your emotions. So if you feel emotions, the DLPFC will try to control it. If you feel emotions, it means you feel present in the world.
But the DLPFC of a human matures very late, so children are not able to control their emotions. Kids show strong emotions, can’t control their pleasure and are in danger of getting addicted to anything, games or VR for example !
That’s why parents have to play the role of the DLPFC by training the kids to restrain themselves; we have to replace the DLPFC by authority.
It also seems that the pre-frontal cortex, responsible for the control of behaviour, self discipline and motivation only matures at 20 years old, so that would explain behavioral problems of teenagers. (wow I should tell that to my mum!)
It’s also one of the first brain areas to degenerate with age if it’s not properly activated with specific tasks.
Presence
Studies have proved what we intuitively already knew; even on a simple screen, 1st person view is more immersive than 3rd person.
Another experiment was conducted with a driving simulator. During the ride, a deer or a child would jump on the street in front of the car. It turns out that people didn’t get used to the kid jumping on the street, proving that the brain is working as if the situation was real!
VR training and rehabilitation
VR has a huge potential for training and rehabilitation even if you only take the motivational part; as VR is more interesting/fun, people will want to take the training or rehabilitation sessions whereas they get bored and don’t show up at traditional sessions.
Motivation is the most important factor in learning, and VR is very motivationnal!
Conclusion
The brain constructs reality. It is remarkably plastic and matures late, and for him, VR can be real.
The brain can also be shaped by VR experience.
M. Janke states that The Matrix is completely possible; reality is already a virtual world. We interpret reality through the lens of our experience, and if properly created, a virtual environment can seem very real to the brain, with all the positive and negative aspects this can give to its creator.
So let’s use that great possibility for the better!
“I know kung-fu!”
18:01 VR Applications, Virtual Reality
From Monday 10th to Wednesday 12th March 2008 was the actual IEEE VR conference. See the complete program.
Training
Again, for me one of the most interesting applications of AR/VR is training. See also Virtual Humans below.
In the paper by John Quarles, Samsun Lampotang, Ira Fischler, Paul Fishwick and Benjamin Lok, from University of Florida, “Mixed Reality Merges Abstract and Concrete Knowledge” we have seen that Augmented Reality helps .. merge abstract and concrete world. That means that it’s perfect for training on real and complex machines by adding abstract information that helps understand the underlying mechanics. When the user operates the physical machine, the virtual model is also updated, an you can see the updated augmented information to understand the results of your actions.
Virtual Humans
I really enjoyed the Virtual Humans sessions, and particularly the “Virtual Human + Tangible Interface = Mixed Reality Human” paper by Aaron Kotranza and Benjamin Lok from University of Florida. Here’s the abstract of the paper :
Virtual human (VH) experiences are receiving increased attention for training real-world interpersonal scenarios. Communication in interpersonal scenarios consists of not onlyspeech and gestures, but also relies heavily on haptic interaction –interpersonal touch. By adding haptic interaction to VH experiences, the bandwidth of human-VH communication can be increased to approach that of human-human communication.
To afford haptic interaction, a new species of embodied agent isproposed – mixed reality humans (MRHs). A MRH is a virtual human embodied by a tangible interface that shares the same registered space. The tangible interface affords the haptic interaction that is critical to effective simulation of interpersonal scenarios. We applied MRHs to simulate a virtual patient requiring a breast cancer screening (medical interview and physical exam). The design of the MRH patient is presented. This paper also presents the results of a pilot study in which eight (n =physician-assistant students performed a clinical breast exam on the MRH patient. Results show that when afforded haptic interaction with a MRH patient, users demonstrated interpersonal touch and social engagement similarly to interacting with a human patient.
What this all means is that interaction with virtual humans help change real life behavior in the real world. Self perception is changed. ixed reality humans, by adding the dimension of touch give the trainee more empathy, social engagement, through interpersonnal touch. Touch drives the interaction. Users interacted with the MVH as if they were real humans, so what they learn in this training is directly transferable to the real world.
Honors
As for the honors, Bernd Fröhlich got the VR Technical Achievement Award, and Bowen Loftin got the VR Career Award.
12:08 Augmented Reality, VR Applications, Virtual Reality
On March 9th I attended the Searis 2008 Workshop: Software Engineering and Architectures for Realtime Interactive Systems, which was the perfect workshop for me
The workshop was about presenting existing architectures and regroup people working on that particular topic, learn from existing projects so that people don’t reinvent the wheel everytime, learn from others and try for find some formalization.
Anthony Steed pointed out that having a platform running for many years is very hard and that it has to be rewritten once in a while. Architecture gets cluttered and needs a fresh restart. Nonetheless, the DIVE platform still allows him to run demos that are several years old, which is probably not the case on many systems. Many platforms break compatibility when upgrading (or license expire for commercial products).
We had the presentation of a lot of middleware architectures (InTml, Lightning, FlowVR, OpenMask, ViSTA VR, Morgan) with lots of common ground such as data flow, abstraction of devices/interaction techniques/renderer, easy cluster distribution, portability.
And there is a lot of wheel reinvention on all this common ground, but at least people are not reinventing scenegraphs (well not everyone but for good reasons, see below). Scenegraph libraries are widely used, with a lot of OpenSG, OpenSceneGraph and Ogre3D. It seems it might be a good idea to create a meta scenegraph library (if it was at all possible ;), as every engine is switching from one scenegraph lib to another at some point.
There is a new trend towards interaction techniques abstraction, following the devices abstraction.
There is also a trend of using handheld devices with limited rendering capabilities, like smartphones, PDA, mainly for Augmented Reality. Not so many platforms/toolkits support those devices (which is why Morgan created their own SceneGraph that supports this feature).
An interesting approach for manipulating heavy graphics data is to use multi-frame rate rendering. The problem is that if your data can only be displayed at very low frame-rates, your interaction will also be that slow. But you can have one fast rendering loop for user interaction, and one slow rendering loop to display the graphics data. The two images would be created on two different graphics card or PCs, then digitally composed to create the final image.
Research in the academics are being conducted to get some higher level frameworks. For example, engines that use functional reactive programming, or the actor model on abstract semantic level. I’m sorry but I didn’t quite understand what it was all about, a bit too academic programming for me.
Some other high level interesting additions are semantic description of a world/application/interaction and the use of dual state machine/data flow approach. Semantic description using web semantic tools were discussed several times.
If you’re interested in joining the group, Raimund Dachselt will give you the details of the next steps.
18:25 VR Applications, VR Devices, VR Displays, Virtual Reality
On saturday March 8th I attended the 3DUI Conference which was really interesting.
It seems the trend is to simplify 3D interactions by adding some real life constraints.
A lot of talk was about using 2D or constrained interactions inside a 3D world, because real life interactions are mainly 2D and so, our brain is mainly wired for 2D interactions. Only aircraft and helicopter pilots are really doing 3D interactions.
M. Stuerzlinger talked about his idea of Smart 3D; there should be no floating objects (there aren’t in the real world), no object interpenetration. You should only be able to select and interact with objects that are visible or in front. People in real life would move to interact with hidden objects. He also argues that perspective and occlusion are the strongest depth cues so you wouldn’t need stereo if you assume non floating objects. He also notes that interactions are mainly 2D. Moreover, 2D input devices (mouse) have much higher precision than any 3/6 Dof input device (10 to 100 times!), and this seems to be a major point in achieving precise interactions. An interesting result was also that having a surface support (like a table when using a mouse) didn’t improve interaction precision as much as device resolution does.
Doug Bowman also states that we don’t see much complex 3d interactions in VR applications. Apparently stereo rendering doesn’t improve efficiency, but the choice of the interaction technique does.
So the question raised was, do we have to mimic the real world? Can’t we just break the rules?
It depends on the task, but if you want your knowledge to be transferable to the real world, having real world rules helps.
Following that trend, Withindows “is a theoretical framework for developing user interfaces that can operate in both dekstop and full 3D immersion without redevelopment.” It will constrain your interaction to a 2D plane but should improve the interaction efficiency.
I also like the 3d drawing technique presented by M. Keefe, called “Dynamic draggin for input of 3D Trajectories”.
There were a lot of very interesting posters. Like a technique to augment the size of a viewport inside a HMD simply by sliding the front plane of the camera frustum to follow the dominant hand, by Andrei Sherstyuk, Dale Vincent from University of Hawaii and Caroline Jay from University of Manchester.
My favorite poster was a technique by Franck Steinicke from the Muenster University where you wouldn’t map exactly the real walking of someone to the real world. For example, you would have to walk in circle in the real life to walk in straight line in the virtual world. You could also have the user thinks he walked 20m when is just walked 10m. Why would you want to do such a strange mapping? This translation and rotation compression would be a really good way to use a small physical space and have it seem like a huge virtual space.
EDIT: Tabitha Peck along with Mary Whitton and Henry Fuchs presented in the full VR session a very interesting paper (”Evaluation of Reorientation techniques for Walking in Large Environment”) that uses a reorientation technique (ROTs) based on distractors. From the abstract : [we use] distractors - objects in the VR for the user to focus on [like a butterfly] while the VE rotates. The reorientation technique is also “used to lift the size constraint, (…) when a user is close to walking out of the tracked space.”
There was also a very interesting new display; a box 3d display by Roberto Lopez-Gulliver and Shunsuke Yoshida from the NICT :
And an impressive freeform projection display from Daisuki Kondo and Ryuogo Kijima from the Virtual System Lab of the Gifu University. This technique allows you to project a correct image on any surface (whose 3d mesh you have to know in advance).
This technique allows you to project a correct image on any surface (whose 3d mesh you have to know in advance).
I’d love to provide links and more information to the papers/devices I mentionned here, so if you’re one of the authors and want to share it, feel free to tell me!
11:24 Virtual Reality
I’m leaving tomorrow to attend IEEE VR 2008 and 3DUI 2008.
If you’re there too, let me know so that we can meet there!
Virtools will have a booth, or I’ll be asking questions at the conferences =)
Thanks to David for enabling the trip
17:42 Virtual Reality
The french association for virtual reality (AFRV [fr], english page here) was holding its annual meeting in Marseille. Nearly all the major actors of VR in France were there.
The complete photo gallery is here.
- The 3×3x4m Cave of the CRVM -
Bruno Arnaldi has been elected as the new AFRV president, replacing Philippe Fuchs.
Here are some interesting things that were said during those days, in a simplified and out-of-context form (don’t hesitate to correct me!) :
- VR is very interesting to study humans: it allows for a good experimental control and ecological validity: you can manipulate a reality and conduct experiments that for practical or ethical reasons couldn’t be conducted in the real world.
- In a lot of fields, studies must be done to see if the actions, behavior and perception in virtual environments and reality match. This ecological validity is crucial for the validity of studies conducted in VR only.
- A lot of work has to be done to improve the communication skills of an avatar: non-verbal cues must be transmitted for better communication, and the way we interact with them must be simplified, more natural.
- A lot of people don’t want to use intrusive devices (HMDs, glasses etc)
- 3D interactions are still a big research area.
- Users are not necessarily ready to use haptics (force-feedback), even a 6DOF device is sometimes difficult!
- Rehabilitation through VR is better: closer to reality, better transfer from virtual to real, better motivation thanks to the fun aspect of VR.
- Stereo viewing is not the natural way of seeing. Sometimes we are faster at evaluating depth through parallax. Studies are being conducted to study the perception of shapes with stereoscopy compared to the real world.
17:34 VR Devices
I just returned from Weimar, Germany, attending the Immersive Projection Technologies/Eurographics Symposium on Virtual Environments.
This has been a really interesting event. I’ve met some very nice people and learned a lot of things!
I’ll try to speak about the conferences later. In the meantime here are some photos.
Here are three pieces of hardware that were presented here that I particularly enjoyed :
Intersense IS-1200 VisTracker
Which I already presented here.
A.R.T. FingerTracking with Tactile Feedback
A.R.T. was presenting a prototype of its FingerTracking with tactile feedback. Three little wires enclose your fingers and vibrate can vibrate with differente intensities. This allows to “feel” when you touch an object, and have more precision when manipulating objects, especially when you don’t have a stereo display.
Haption Virtuose 6D Desktop
A small professional and precise haptics device with 6DOF (position+orientation), already presented at Laval Virtual 2007 :
11:17 Game, VR Applications, VR Devices, VR Displays, Virtual Reality, Virtual Reality DIY
These three great days have been very intense; holding the booth, meeting people, attending conferences, trying hardware and applications, being a jury member for the student competition. First things first, here are some pictures I took there. I should have taken more of the show..
There was not much new hardware, especially hardly any new expensive hardware. The novelty came from the use of existing hardware and better software. It seems that VR is at last being democratized; people create customized and cheap input devices, use webcams, recycle hardware not meant for VR etc.
Read on for more..
This site is better viewed with :
