Virtual Reality with Kinect – House Browsing & Painting

September 24, 2012 in Guney Ozsan, Installation, Interactive, News

This is our first interactive installation with our audio visual group Post Illusions. Here, a user roams in a house and paints objects with body movement. This time, I participated as the programmer of this project. Although I didn’t do anything about audio or sound, the background and know-how I developed for this project is very essential for my future projects. I am working on an environment without light, in which you can walk around in a 3D space made of sound. I am expecting to create a feeling that you swim in sounds with volumes.

Virtual Reality with Kinect – House Browsing & Painting from Post Illusions on Vimeo.

Our aim and challenge in this project was to create a first person interactive experience using Kinect and Unity 3D. The user is able to walk around a house, garden and paint objects from a color catalogue interface. We considered if the current motion tracking technology can provide a swift working end user experience for daily usage outside of gaming realm.

The hardware we used are basically a Kinect, a Macbook and a videowall.  OpenNI drivers, Primesense NITE, ZigFu, Unity 3d and Cinema 4d for texture baking and modeling are on the software end. Smoothing provided by OpenNI drivers was not sufficient forfluid camera movement. Motion tracking noise is reduced using smooth follow function of Unity 3D tracking the torso of the skeleton from zero distance instead.

Since the user is physical in a virtual environment it was not possible to represent virtual object collisions in the physical space. However most users intuitively avoided the boundaries of 3d environment (walls) and used doorways to roam around.

Most of the objects (including walls) in the house are selectable so object and environment separation was difficult. In order to overcome this, we decided to assign double gesture combinations. For example left hand pointer as a bucket tool combines with right hand pointer as a brush tool to open color palette interface. Especially users roughly below 30 years old were comfortable with intuitively discovering the gestures. However habitual likeness made numerous users try -air clicking- with minimum arm motion, thus it might be necessary to track wrist motion with more accuracy in future motion sense devices.

We aimed to create visual immersion with minute details in the environment like windy outdoors, advertorial displayed on tv, ceiling fan in the kitchen, active fireplace…etc and a few dynamic lights positioned for emotional impact.

I hope this project also inspires you for further interesting projects.

Flattr this!

Leave a Reply