If you’re a VR Geek, you’ve probably already bumped into Cave Unreal Tournament (CaveUT), an adaptation of the Unreal engine to run in CAVEs and create VR applications. I’ve had the chance to meet its creator Jeffrey Jacobson at IEEE VR 2010 where he co-organized the “What is VR ?” panel. He’s also the director of PublicVR, “a Non-Profit organization dedicated to free software and methods for using Virtual Reality in education and research.”

He was in Paris for the VAST International Symposium on Virtual Reality, Archaeology and Cultural Heritage, so I invited him home for some VR chat and to show off my home-VR system.

He also took the time to answer some questions about the definition of VR, CaveUT and of course a message to VR Geeks 🙂

– Could you introduce yoursef and talk a bit about your VR carrer ?

Hi, my name is Jeffrey Jacobson, I’m now the director of PublicVR (Boston), a non-profit corporation dedicated to VR for education. Most of our applications are in the area of cultural heritage and history but we’re not limited to that necessarily.

I originally created PublicVR to hold the intellectual property of CaveUT , VRGL and other open source projects that I was doing.
I recently organized a panel at VR 2010 with Chad Windgrave. PublicVR also published seven papers this year. We won a grant from the National Endowment for the Humanities to stage an Egyptian public ritual, called the egyptian oracle. This will be a mixed reality performance where an avatar interacts with the audience and a live actor in front of the virtual temple.
The temple and the avatar are projected life size on the screen. In the theater in the Carnegie Museum of Natural History, the screen will be a digital dome. We’re also going to have performances in Boston at Puppet Showplace Theater and Axiom Gallery.
The audience will play a role of the Egyptian populace, asking questions from the divine image of the god Horus, and the priest will interpret the answers.
This is  exploration of puppeteering, which is our new research direction, especially embodiment of the human in the figure, in the representation.
My dissertation was in learning in immersive spaces. We made a game based on the virtual egyptian temple and kids played the game. Some played it on a laptop, some played it on an immersive display. The kids that played the game in the immersive display learned more.

– How do you measure that ?

We took several measurements, but the most important one was that we asked the kids to give us a tour of the temple and the one that played in an immersive environment had a better understanding of space, even though they used the same navigation scheme and software as the others. That’s almost certainly because the information about the temple is spatially organized, and with the panoramic view you have a much better memory of the geometric layout. You now have more and more people doing research about learning in VR!

The game, called The gates of Horus, has now been ported to Unity3D. We also continue to use CaveUT, and it will continue to be useful since it has some unique features.

– You organized the panel “What is VR?” with Chad Windgrave. So I would like to know your definition and the conclusions of the panel.

The panel reached no conclusion because the term VR is used in so many ways.

For me, I don’t know what VR is !!
What I do know, all humans beings live in mixed reality. This has always been true, from the beginning of being a human. We don’t live in physical reality, we live in psychological reality. We live in a world of myths and symbols and illusions. We create these perceptual, psychological illusions that are very important to us. They are socially real and psychologically real. We do this with artwork, with story telling, we do it with architecture, masks, puppeteering, theater.
The technologies and techniques and  ideas behind VR is just an extension of that. We are creating virtual spaces. AR is where I think the most exciting applications are.
So, what I want to do is mixed reality theater where you have virtual actors, physical actors, the audience.
(…)
When I did Second Life for the first time, I was really impressed with the emotional impact of seeing my avatar, dressing him. I felt a very strong emotional connection.

But Fred Brooks’ definition of VR and yours make a lot of sense to me, I agree completely. But we still have this terrible naming problem, and I don’t know what to call “Virtual Reality”.

– Can you tell us a bit more about CaveUT, how it started and how it evolved ?

In 2000, I was looking for software to do my VR research, and there was really nothing good available. Either it was very expensive systems, or you had to program all your artwork in OpenGL, which is difficult and limiting.

At that time that Quake and Unreal were the two leaders and their graphics were considerably better than any of the VR systems’. So, I decided to adapt one of them to CAVEs. I carefully analyzed both. At the time, Epic Games were new and a few more researchers were using Quake than Unreal. But Unreal was easier to use, more object oriented. I studied the system for a year. I was a graduate student and I only had limited time. I started in the spring semester, and was helped by Zimmy Hwang. We released the first version in summer 2001. We took some pictures and announced it, notably in Slashdot. That was fun!

I only inserted 6 lines of code between the Unreal engine and OpenGL. All I did was distort the view frustum. Everything else was adjusting the game parameters, where you use a multiplayer game and then you have to calculate the view frustum you want.

It was kind of a pain and complex to use.
The first system I used CaveUT in was at the school of Otolaryngology. These were doctors that were studying balance disorders. I led the team that built the BNAVE, a CAVE-like system. It was a fairly complex shape to use for our first display.
Then I created the website, wrote papers about it, went to conferences, showed it off.

Willem de Jonge wrote the matrix maths code so that we turn the camera in any direction. He also ported to CaveUT 2003. Then he wrote VRGL, a companion to CaveUT, which is a hacked OpenGL library that intercepts calls from the Unreal engine and modifies them so that you can have your perspective correction.

Epic Games gave me a source license so that Willem could know what the calls are because it’s fairly complex, and it worked !

Over the years I continued to work on CaveUT, on and off. I got some money and I paid Demiurge Studios to program the Gates of Horus game and Blue Newt Software made spherical dome extensions to VRGL. That enabled CaveUT to show in a multi-projector dome or on a 120° dish.
The only place that ever used it was Carnegie Museum of Natural History, but we got a lot of use out of that installation. It’s been a very long and productive collaboration.

Around the time we got CaveUT 2003, Marc Cavazza and his group got funding and they took CaveUT 2003 and added a lot of functions to it. They gave us the code back but by that time we were on to CaveUT 2004. So we began a separate development. That was a mistake, we should have got back to their version and build on that.

Gerke “Max” Preussner at Virtual Heroes rewrote the code for CaveUT,  improved it greatly and that’s the current public version. It’s an old game engine, but it’s still a useful tool. Right now there’s a high school in Colorado installing it.

CaveUT is now a recognized VR software and has been used in a few dozen systems, but it never really got big.
I think a reason might be that usually the people that do VR either program their own stuff, or they get big budgets and buy commercial software. The people that don’t get budgets find CaveUT difficult. The scripting part requires programming language.
The most often CaveUT was used is for students projects. It’s been a very good learning tool for students to get an introduction to VR.
I also got inquiries from big, serious projects with big budgets, and some of them survived a bit thanks to CaveUT.

The next step is to make CaveUDK!  If anyone is interested, let me know !

– And my last question : Do you have a message for VR Geeks around the world ?

Get involved with artists, especially theater ! Because really, VR is just high-tech Theater.

The other thing is: start doing more with sound. Most VR applications have very little sound, and it’s really important.