FlatWorld: The Mixed Reality Simulation Space (2001-2005)

Starting in 2001, I oversaw the design and development of the FlatWorld project at the Institute for Creative Technologies.

FlatWorld is a mixed reality simulation environment merging cinematic stagecraft techniques with immersive media technology.

In many virtual reality systems, users are required to wear head mounted displays that restrict a person’s freedom to physically move as they would in the real world. Furthermore, a person cannot touch or feel objects in the virtual world.

This project addresses these issues by developing an approach to virtual reality simulation which allows individuals to walk and run freely among simulated rooms, buildings, and streets.

In film and theatrical productions, sets are constructed as modular components called “flats”. Using flats, set designers produce physical, tangible structures that convey a sense of place or activity. FlatWorld utilizes a “digital flat” system. A digital flat is effectively a large screen display. A single digital flat has the ability to appear as an interior room wall or an exterior building face. When physical props are used in conjunction with digital flats, doors and windows can be simulated. Users can touch and open these portals to view an exterior virtual world. Because of their modular characteristics, several digital flats can be rapidly assembled in any open space to simulate multiple situations in a variety of geographic locations. The immersive experience is heightened through the use of immersive audio, tracked stereoscopic graphics, and “4D” sensory effects such as blowing wind and vibrating floors. This approach of using physical props within a virtual environment can create a “mixed reality” world where the physical and the virtual seamlessly coexist.

The project was documented in this IEEE Computer Graphics and Applications article:

Pair, Jarrell, Ulrich Neumann, Diane Piepol, and Bill Swartout. “FlatWorld: Combining Hollywood Set Design Techniques with VR.” IEEE Computer Graphics and Applications. January/February 2003, pp. 12-15. Download PDF

The photos and video clips below describe the one room prototype built by the November 2001 team.

Click images for full size view.

Two digital flats simulating plaster walls used in conjunction with table, window, and door props create a virtual room.	A user views the external virtual world through a physical door.
Stereoscopic graphics are used to provide a compelling sense of depth. Objects can even appear to pop out of the screen and float in the middle of a room. (Note: Stereo 3D effect is simulated in this image)	Broken concrete props and digital flats simulate crumbling room walls.
FlatWorld Image Featured on “Communications of the ACM” Magazine July 2002 Cover