The i-Cone/spl trade/ - a panoramic display system for virtual environments

The i-ConeTM – A Panoramic Display System for Virtual Environments Andreas Simon, Martin G¨obel Fraunhofer IMK Virtual Environments Schloss Birlinghoven, Sankt Augustin, Germany {andreas.simon,martin.goebel}@imk.fraunhofer.de

Abstract We present the i-ConeTM : a new projection-based panoramic display system for virtual environments. The i-ConeTM uses conical screen geometry, resulting in optimized projector placement to create an extended workspace for standing participants in a front projection curved screen display with a very large continuous field of view. Improved acoustical properties of the conical screen geometry enable the use of immersive spatial audio. Combined with the excellent visual quality and homogeneity of a curved screen display, these properties make the i-ConeTM an attractive display system for immersive virtual environments, suitable for larger audiences. Interaction paradigms and application examples for this new immersive display system are presented.

Figure 1. Four-channel 240◦ i-ConeTM display system. The conical shape has a slant of 5◦ .

1. Introduction Since Sutherland’s ”ultimate display” [10], a number of design goals are driving the development of new stereo capable display systems for immersive virtual environments. We can distinguish between the pure visual quality of a display, its ergonomics and its interaction properties. Important factors for visual quality are acuity, contrast, brightness, as well as the lack of distortion and homogeneity. Better ergonomics have been seen as the dominating driving force behind the overwhelming success of projection-based display systems for virtual environments, most notably the CAVETM [2] and the Responsive WorkbenchTM [7]. These ergonomic issues are unencumbered easy viewing, horizontal and vertical field of view, as well as limited sensitivity to head motion induced lag. One of the most exciting examples for the influence of a display for virtual environments on interaction properties is its capability to support multiple participants and collaboration. Head coupled displays natu-

rally support multiple viewpoints but strongly separate participants. While projection-based display systems typically allow for more than one participant, only very few of these display systems are true multi-view capable systems [1, 6]. Front projection curved screen displays deliver excellent image quality over a continuous field of view and are suitable for larger audiences. They show acceptable, homogeneous geometric distortion for untracked viewers. Curved screen systems have not commonly been used as a display for immersive virtual environments because of their lack of workspace for standing participants and because of the bad acoustics, preventing the use of immersive spatial audio. Compared to the large 270◦ or even 360◦ horizontal field of view offered by the CAVETM , a typical curved screen installation uses a three-channel setup for a limited horizontal field of view of only 160◦ and without floor projection. We have designed a new projection-based display sys-

tem with conical screen geometry, the i-ConeTM (immersive continuous display environment, see Figure 1). Design goal for the i-ConeTM was to create a panoramic display system for virtual environments with a very good and seamless image, suitable for larger groups of participants. The i-ConeTM overcomes a number of problems that have limited the usefulness of curved screen displays for virtual environments. By optimizing screen geometry and projector placement, we have created a front projection curved screen system that delivers extended workspace for standing participants, is capable of a very large (even full 360◦ ) horizontal field of view and allows for a projected floor. It provides good acoustics, enabling the use of immersive spatial audio. So far, two installations have been built: One permanent installation, using four channels and 240◦ field of view with a resolution of 6200x1460 pixel at our institute in Sankt Augustin, Germany. A temporary three-channel installation was presented at CeBIT 2002. This installation held up to 25 participants and drew 700 visitors per day.

2. Panorama Seamless panoramic images have a long and remarkable history that actually predates photography and other imaging techniques. The word panorama is created from the greek words pan (all) and h´orama (see) describing a landscape painting with a 360◦ surround view. Panoramas are an extension of the work of 15th century artists on central perspective, motivated by the artists’ wish to ”push the horizon beyond the picture frame”. A panorama strings a number of images to a cylinder, each with their own central perspective, effectively developing a poly-perspective image. In 1787, Irish painter Robert Baker is granted a patent for his invention of a surround image. He completes a first exhibition building at Leicester Square, London in 1794. The panorama at Leicester Square has a radius of 13m and a height of 11m with a capacity of 150 visitors. For half a century, it shows about two new giant paintings each year. Early in the 19th century, the panorama becomes a great success all over the world. Panoramas are built not only in England, but also in Holland, Denmark, the USA, France, in Switzerland and eventually even in Germany. The panorama establishes itself as the unique visual mass medium of the 19th century, open to the general paying public. It is important to realize that this is a public not exposed to images, as paintings are almost exclusively held in private or feudal collections and low cost reproduction techniques do not yet exist. Panoramas disappear almost instantly with the rise of a new and more dynamic visual mass medium, the cinema. The panorama represents a paradigm shift in painting. Artistically it leads towards a new realism: ”The basic idea of the panorama is to deliver an artfully artificial image that

makes the viewer believe to see not the painted but true nature” [8]. Subjects are grandiose landscapes and battle scenes, which at that time have to be considered current political events. Technically, the panorama extends the unique viewpoint of the central perspective to a continuum, allowing a large number of viewers to experience the picture together. To achieve strong illusion and suspension of disbelief, the panorama has to surround the viewer completely, removing any comparison between the image and reality. A platform in the center of the panorama, capable of holding up to 200 people, is entered from below, opening the view to the visitor in dramatic fashion. For full effect, good lighting of the image is important: indirect natural light from the top is used, with a blind covering the surrounding opening in the ceiling. As a historic media phenomenon, artistically and technologically the 19th century panorama is an amazing invention. It delivers suspension of disbelief to an audience that is not even accustomed to seeing images. The sheer size and capacity of these systems, the use of a poly-perspective and the high level of visual quality achieved are still remarkable.

3. Display systems for virtual environments Before the revolution of projection-based display systems was started by the CAVETM , head coupled display systems were almost exclusively used as display systems for virtual environments. Head mounted displays, as well as the BOOMTM [3], are essentially single user display systems that cut off the user from the outside (real) world. Although this isolation can lead to very strong immersion, it is significantly reducing the usability of these systems. Today, projection-based systems are the dominating display technology for virtual environments. Common projection-based stereoscopic display systems for virtual environments use one or more planar screens with back projection. A single planar screen delivers only a limited field of view, typically less that 90◦ . Systems that use multiple planar screens, like the CAVETM or the twosided Responsive WorkbenchTM , concentrate geometrical distortion and self-illumination in the edges. Head tracking is needed to eliminate or reduce geometric distortion in the edges by matching the rendered images to the tracked participants viewpoint. Head tracking also provides lookaround capability to these displays, allowing small scale maneuvering through head motion. Although most of these displays are capable to hold more than one viewer, correct and optimal viewing is only delivered to the single head tracked participant. The movement in the image induced by the head motion, stabilizing the image and correcting geometric distortion only for the head tracked participant, is quite disconcerting for the other participants. For larger audiences it

is therefore advised to switch off head tracking, leading to locally concentrated geometric distortion in the edges. Only a very limited number of displays for multiple tracked viewers have been developed. Systems like the multi-user Responsive WorkbenchTM or the Illusion Hole allow only two or three participants. These systems generate multiple stereoscopic images, one for each of the head tracked participants. Curved screen display systems smoothly distribute geometrical error from untracked viewing positions over the display area, making curved screen systems inherently suitable for large audiences. These systems are rarely used as display systems for virtual environments however, because they offer no or very limited workspace for standing participants and typically deliver a field of view smaller than 170◦ , significantly reducing visual immersion.

about 1.83m tall. At the center of the i-ConeTM , the height below the light path of the projectors is about 1.95m from the ground. This is illustrated in Figure 2. In this case, slanting the screen prevents a shadow of a standing participant from falling on the lower third of the screen by raising the light path in the center by about 0.3m.

4. The i-ConeTM The i-ConeTM is a new projection-based panoramic display system for virtual environments. We have developed the i-ConeTM to optimize image quality and to overcome problems that prevent the use of other curved screen display systems for virtual environments. One permanent fourchannel installation with 240◦ field of view is operating at our institute in Sankt Augustin, Germany. A second temporary three-channel installation was presented at CeBIT 2002.

Figure 2. Optimized projector placement: Extended workspace and prevention of shadows from crossfire. The light path is visualized for two of the four projectors.

4.1. Basic Construction 4.2. Distortion Correction The screen of the i-ConeTM is a conical section with an opening angle of 5◦ . The height of the screen is 2.80m; the radius of the system is 3.30m at the top and 3.05m at the bottom. Using four Barco 909 projectors fitted with curved screen optics, the total field of view is 240◦ using 8% blending. Because the conical section is a single curved surface, the screen can be cut from a piece of plastic material, which is rolled into the final cone shape. Use of a single, flat piece of material for the screen allows for very high geometric precision as well as simple manufacturing and mounting. Projectors are mounted at a height of 3.47m, ensuring that the lowest part of the projector is higher than the 2,80m screen and well above the light path of the other projectors. A 5◦ slant of the optical path in the i-Cone allows a projector placement that is about 0.5m higher than it is possible with a cylindrical screen. This optimized projector placement prevents a shadow of the projector from falling on the screen in a crossfire situation with the opposing projector. Typically, a crossfire situation occurs in systems delivering a field of view greater than 170◦ . The design eye point of the i-ConeTM is set in the center of the system at 1.75m height, corresponding to a viewer

With the i-ConeTM , we generally use static distortion correction for the curved screen that is computed for the design eye point in the center of the screen at 1.75m height. The distortion correction is set by the geometric adjustment of the CRT projectors and is calculated for off axis projection on a slanted virtual image plane. Virtual image planes are placed into the conical screen in such a way that they intersect in a line lying exactly on the screen’s surface. This minimizes residual distortion for stereo projection (which uses two eye points that are slightly out of the position of the design eye point). It is worth noting that in this configuration, because of the 60◦ field of view of a single image, the virtual image planes are slanted less than the screen. The slant of the virtual image plane is 4.4◦ compared to the 5◦ slant of the screen. In addition, the resulting shape of the usable section of the image between intersections is not rectangular but a trapezoid. The screen of the i-ConeTM is physically marked with 721 reference points at regular horizontal and vertical intervals. These points are applied with very high precision using a laser theodolite standing in the design point of the

system. For each projector a corresponding reference grid is generated, allowing correct manual geometric alignment by matching lines of the reference grid with reference points on the screen. The reference grid is computed by projecting lines from the screen back onto the virtual image plane.

4.3. Viewing and Interaction Our four-channel 240◦ i-ConeTM with 2.80m height, 3.30m radius and 5◦ slant is constructed in a 3.7m high 6.2m by 7.0m space, making efficient use of the available room. The same space that is used for the i-ConeTM is typically needed for a cubical CAVETM with 2.70m at the sides. The display system feels spacious and can be used comfortably by a group of 15 or more participants. Positional head tracking can be quite disorienting and disconcerting for larger audiences, introducing unwanted and unexpected image motion and creating additional distortion for all viewers except for the tracked participant. Therefore, and because real time distortion correction for curved screens would require a slow second rendering pass, we are currently using the i-ConeTM without positional head tracking. If needed, we use tracking of the orientation of the lead participants head. This is required in some applications to maintain correct stereo viewing over the wide horizontal field of view. In virtual environments, small head motions are typically used for maneuvering. This works well for small objects that are within reach of the participant. A large display system like the i-ConeTM , where the average distance of the viewer to the screen is about 3m, is more suitable for viewing larger objects at distances greater than 1.5m. Under these circumstances the use of head motion for maneuvering is less effective because head motions are small with respect to the objects in view. For object manipulation we have seen that it is very successful to use prop based techniques [5] like the Cubic MouseTM [4]. Controlling the orientation and position of the tracked prop allows precise and intuitive control of the position and orientation of a large virtual object. For large-scale travel we use a tracked flying joystick that interfaces to different motion techniques like driving and flying. The large horizontal field of view and the total lack of stable visual features in view – the image is very homogeneous and shows no residual artifacts like corners – result in to excellent visual immersion. Because of this, the effect of visual motion for a standing participant can be very strong, leading to a disturbing motion aftereffect if the flying motion is not appropriate. Self-motion of the participant in the virtual environment is much more likely to give these strong unwanted effects than object motion.

4.4. Acoustics and Sound Immersive spatial audio is a very valuable display channel for panoramic virtual environments. Audio extends the awareness of the participant beyond his immediate field of view. While the immediate field of view with stereo shutter glasses is limited to about 120◦ , audio covers a full 360◦ . For audio rendering, multiple reflections of sound sources destroy good sound source localization. Every smooth, large wall – such as a display screen – effectively acts as an acoustic mirror. In this respect, conical screen geometry is very efficient for improving the acoustic properties. In a cylindrical screen, opposing sides of the screen are parallel, leading to strong multiple reflections. The slanted walls of the i-ConeTM will generally move the virtual image of the secondary reflection of a sound source below the lower corner of the screen. This is of particular importance in systems with a very large field of view where the screen may completely surround the participant. Resonance frequency of the i-ConeTM is relatively high at 800Hz, but multiple reflections die down quickly. Currently we are using a 13-channel audio surround display with loudspeakers placed at regular intervals on the floor and on top of the screen. The audio display is driven by a sound server performing the spatial audio rendering using vector base amplitude panning [9].

4.5. Image Generation We use two different hardware platforms to drive the iConeTM : a 12 processor, four-pipe SGI Onyx2 system and a Linux PC cluster. The SGI Onyx2 is using one pipe per channel to deliver a resolution of four times 1440 by 1320 pixel at 94Hz for a total resolution of 5600 by 1320 antialiased pixel. The Linux PC cluster is a master-and-slave configuration, connected via gigabit Ethernet. Scene graph distribution is used, broadcasting incremental updates of the scene graph in the application master to the slaves for image generation. The application master and the slaves are synchronized and frame locked. ATI FireGLTM 4 graphics cards in the four slave PCs are generating genlocked stereo video output for the projectors of the display system. The resolution of the PC signal is 1600 by 1460 pixel at 94Hz for a total resolution of 6200 by 1460 pixel without antialiasing. As common software platform we use AvangoTM [11], our OpenPerformerTM based distributed virtual environment application toolkit.

4.6. Extensions and improvements At CeBIT 2002 we have presented a temporary threechannel 175◦ i-ConeTM display system. This system was

using a conical screen with 7◦ slant, further enlarging the workspace and enabling us to use a raised platform in the center to give 25 to 30 visitors unobstructed viewing conditions. The raised platform was equipped with a vibrating sound floor, which increases the acoustical and kinesthetic immersion. So far, we have not had the opportunity to add a projected floor to our system. This is a very attractive modification of the system, substantially increasing the limited vertical field of view. The experience with CAVETM systems has shown that the addition of a floor can dramatically improve immersion. The multi-pass rendering of dynamic non-linear distortion correction was so far out of reach for the available graphics hardware. We hope that with the addition of a Linux PC Cluster, we will be able to experiment with this option. We are developing a rendering technique that is using the 19th century poly-perspective to deliver correct stereo viewing to a large audience without orientation head tracking.

5. Conclusion The 19th century panorama is an amazing invention, delivering suspension of disbelief to an audience that is not even accustomed to seeing images. The capacity and the high level of visual quality of these painted panoramic displays are still remarkable. In this paper, we have presented a new projection-based panoramic display system for virtual environments: the i-ConeTM . By using a screen with a conical shape, we optimize screen geometry and projector placement. The i-ConeTM display system combines the excellent visual quality and homogeneity of a curved screen display with an extended workspace for standing participants, a very large continuous field of view, and good acoustical properties. This makes the i-ConeTM ideally suitable to enable group experiences for large audiences in a virtual environment. To date, we have built two systems which confirm the concept and show excellent visual and audio quality. Our permanent installation of the i-ConeTM has for many applications replaced the use of our CAVETM -like system.

6. Acknowledgements The authors would like to thank Barco for their continued effort and support to design and build the i-ConeTM display system.

References [1] M. Agrawala, A. C. Beers, I. McDowall, B. Fr¨ohlich, M. Bolas, and P. Hanrahan. The two-user responsive workbench: Support for collaboration through independent views of a shared space. Computer Graphics, 31(Annual Conference Series):327–332, 1997. [2] C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti. Surroundscreen projection-based virtual reality: The design and implementation of the cave. Proceedings of SIGGRAPH 93, pages 135–142, August 1993. [3] I. Dowall, M. Bolas, S. Pieper, S. S. Fisher, and J. Humphries. Implementation and integration of a counterbalanced crt-based stereoscopic display for interactive viewpoint control in virtual environment applications. In Proceedings of SPIE 1256, 1990. [4] B. Fr¨ohlich and J. Plate. The Cubic Mouse: A new device for three-dimensional input. In Proceedings of ACM CHI, pages 526–531, April 2000. [5] K. Hinckley, R. Pausch, J. C. Goble, and N. F. Kassell. A three-dimensional user interface for neurosurgical visualization. In Proc. of the SPIE Conference on Medical Imaging, pages 126–136, 1994. [6] Y. Kitamura, T. Konishi, S. Yamamoto, and F. Kishino. Interactive stereoscopic display for three or more users. In Proceedings of SIGGRAPH ’01, pages 231–239, August 2001. [7] W. Kr¨uger and B. Fr¨ohlich. The responsive workbench. IEEE Computer Graphics & Applications, 14(3):12–15, May 1994. [8] S. Ottermann. Das Panorama - Die Geschichte eines Massenmediums. Syndikat,, 1980. [9] V. Pulkki. Virtual sound source positioning using vector base amplitude panning. Journal of the Audio Engineering Society, 45(6):456–466, June 1997. [10] I. E. Sutherland. The ultimate display. In Proceedings of IFIPS Congress ’65, pages 506–508, May 1965. [11] H. Tramberend. Avocado: A distributed virtual reality framework. In L. Rosenblum, P. Astheimer, and D. Teichmann, editors, Proceedings of IEEE Virtual Reality, pages 14–21, Los Alamitos, California, 1999.