Bauhaus Scenography for Virtual Environments

Bauhaus Scenography for Virtual Environments Joshua A. Fisher Amit Garg School of Literature, Media, and Communication Georgia Institute of Technology Atlanta, USA [email protected] School of Interactive Computing Georgia Institute of Technology Atlanta, USA [email protected] Wesley Wang Karan Pratap Singh School of Interactive Computing Georgia Institute of Technology Atlanta, USA [email protected] School of Interactive Computing Georgia Institute of Technology Atlanta, USA [email protected] Abstract— To achieve Sense of Presence (SOP), 1920’s Bauhaus scenography is presented as inspiration for a design language for immersion in virtual environments (VE). The discipline’s tradition of immersion, raumempfindung, is the construction of space to facilitate a tangible experience. It is mapped into five categories for VR: movement, externals, tone, time, and topology. This language informed the design of Ares, a room-scale VR experience. A user study assessed the approach and results indicated that the Bauhaus methodology benefits immersion to support SOP. Keywords—- Sense of presence, immersion, virtual environment, design, performance studies I. INTRODUCTION In VR, considerable research is devoted to creating VEs that engender a SOP [2,3,13]. This research has not resulted in a universally applicable and transparent design language. Often conflated, immersion, which is separate from but correlated to presence, aids SOP [10]. Immersion is the affordance of the technology to provide a vivid, extensive, and inclusive VE with embodied interaction [10]. A user’s SOP is encouraged by the intentional design of that afforded illusion. To that end, a persistent and germane design language for SOP can be established by situating the contemporary practice within an existing tradition. The scenography of the Bauhaus, nearly a century old, was implemented in the case study, Ares. It helped design a VE with high immersion that contributed to SOP. II. PRESENCE THROUGH MOVEMENT IN A VE The embodied presence framework is utilized in this research [6]. Mental representations of a space are determined by the possible actions that the environment affords. Users mesh these affordances with their memorized patterns of interaction. This cognitive work results in a mental representation of the relationship between the body and the environment for the user [6]. To understand a new space, a user then moves through it, and that afforded movement is a key aspect of immersion. The extent to which this movement influences presence is dependent upon the navigation technique employed in the VE [6, 8]. It is hypothesized that the more involved the body is during navigation and interaction within a VE, the more likely a user will experience SOP. For example, engaging a user’s natural mode of locomotion to explore a VE could increase presence as 978-1-5386-4494-2/17/$31.00 ©2017 IEEE it involves the entire body [4]. These same observations being made by contemporary Human-Computer Interaction (HCI) researchers were also recognized by Bauhaus educators such as Oskar Schlemmer and Lothar Schreyer. III. BAUHAUS SCENOGRAPHY Scenography’s design language has been used for over a century for the construction of physical sets for performances. Scenography is the integration of “space, text, research, art, actors, directors and spectators [which] contributes to an original creation” [11]. Schreyer, in his short tenure as the Bauhaus theatre director, recognized that basic shapes and “pure colors” afford immersive illusions. Through the staged composition of spatial and aesthetic relations, scenographers facilitate an audience’s perception of an experience’s space. A. Raumempfindung: Feeling Space The Bauhaus sought raumempfindung, immersive space that could be felt by both the audience and performer(s) [9, 12]. Schlemmer’s system was based on the belief that space is made up of malleable material. As a performer moves through this material, it hardens around them into “felt space”. In 1927’s Dance in Space, a performer moves from one partitioned square to another. The space around them becomes hardened as a negative form, encasing them within a newly created room [9]. This is the effect of raumempfindung. The composition of malleable space, which hardens into felt performance space, may be translated into VR where space, both physical and digital, meld into new structures open for interaction. In Fig 1, Schlemmer’s line geometry demonstrates how a tight-cubic space might be partitioned into various spatial planes through the movement of the performer. In VR, these planes may be activated by a user’s movement into different scenes within an expansive VE. Figure 1. Oskar Schlemmer’s, “Figure in Space with Plane Geometry and Spatial Delineations” on the left. Shading added in center and right side to illustrate how a user’s movement spurs the creation of new spaces through the solidifying of negative forms. Transmogrifying Bauhaus scenography for VR means taking their all-encompassing approach. As such, the following inclusive design aspects are put forward. • Movement governs energy on the stage and creates space. The movement of objects may direct user attention [12], enable change blindness [8], and occur at various perceptual depths-of-field. Further, movement enables impossible spaces and antechambers, design tactics that power the VE’s immersion. • External scenography is what the audience knows outside of the theater [6], the physical characteristics of the performance space, and the broader culture [11]. • Tone, includes color, sound, light and the composition of objects [11]. The afforded interplay between these facets creates a kinetic symphony [11]. • The perception of time in VR is influenced by spatial primes [8] to achieve an interplay between Ergodic-time and Negotiation-time [5]. These elements can dissociate the user from atomic-time and increase SOP. • Topology, the aspects of scale and depth can be manipulated to encourage user curiosity in the VE and encourage SOPinducing exploration. The use of “peepholes” or “vistas” can give the illusion of larger VEs by alluding to far off spaces. IV. SCENOGRAPHY IN ARES The expression of scenographic considerations is reviewed below. Following, the potential effects of these immersive choices upon SOP is evaluated. A. Movement In Bauhaus scenography, movement of the body to achieve raumempfindung creates new immersive space for exploration. [3]. Beginning in the antechamber, Ares affords a user’s natural mode of locomotion as part of its immersion. The antechamber, uncovered by Oculus Story Studio, solves the issue of orienting the user in a VE so that they start where a scenographer intends [4]. It is the initial orientating aspect of the VR experience. The antechamber has a tantalizing object incites the user’s curiosity. In Ares, this object is a glistening space pod. The object is positioned in a location that initially affords the user with the most freedom of movement, away from physical boundaries. Commonly, antechambers are presented as dream sequences or spaces imbued with diegetic information to prime the user for the narrative. TABLE I. BAUHAUS INFLUENCED VR SCENOGRAPHIC TERMS Element of Scenography Design Considerations Movement User movement, antechamber, impossible spaces External Broader culture, immediate architecture Tone Color, sound, light, object placement Time Topology Atomic, ergodic, negotiation Scale, depth 978-1-5386-4494-2/17/$31.00 ©2017 IEEE ████████████████████████████ The antechamber in Ares consists of two levels connected by an elevator. On the top floor, diegetic audio was used to introduce the narrative to the user. On the second floor, the space pod’s position was intentionally placed to maximize a natural mode of locomotion after a crash landing. Impossible spaces are structural illusions that are built to provide the user with the largest feasible amount of navigable space through their natural mode of locomotion. The key to an effective impossible space architecture is to maintain the user’s sense of spatial perception while the structures shift around them unnoticed. This occurs through change blindness, when users fail to recognize a change in the environment outside of their visual field [3]. A hallway can be moved when a user looks away to subtly reorient them to walk down a new path. Through the constant shifting of virtual architecture, it is possible to maintain the user within the immediate physical space while driving them deeper into an ever-expanding VE. As long as the structures overlap by at least 50%, the user will not perceive a difference [4]. Impossible space structures operate along the Aristotelean understanding of physical space as a container full of malleable space. The structures of the performance space— in this instance the VE—can be reconfigured in ways as finite as the immediate physical space will allow. The container itself doesn’t change, only the perception of the space, the raumempfindung, experienced by the user. Once the user is past the antechamber and in the cavern, they are positioned to allow the most movement according to their immediate physical architecture. From there, three structural shifts occur in which overlapping architectures move like tectonic plates into position as the user makes their way through the experience. B. External Scenography External scenography encourages the use of cultural mores, contemporary news, and mainstream ideologies to ground users in an experience. The culture in which Ares was produced sought to exploit late-capitalism, post-modern society’s obsession with celebrity, and the 2016 US presidential election. As such, a story world in which users have to compete to win a reality TV show in space because the president had defunded NASA is presented. This is a speculative, possible reality that the researchers hoped users may have a connection to. We hypothesize that this situates them within the VR world. Initially, ARES was demoed in a busy lab at the Georgia Institute of Technology on the HTC Vive. This very chaotic space detracted from the user’s sense of presence. In fact, the move into a quieter lab with a more organized space informed the importance of appreciating the immediate physical architecture for immersion in a VE. Both the labs accommodated a play-space with a tracking area of 4m x 3m. C. Tone The tone shifts twice in the ARES experience. In the antechamber, constructed as the International Space Station, the tone is sterile. The base colors are white, black, and a comforting light-blue hue. The light is bright and implies the safety and structural integrity of the VE. The first tone shift occurs when the user awakens inside the underground cavern. The sounds of tumbling rocks echo in the distance, dirt and sand shift overhead, and a deep rumbling comes from below. The soundscape is meant to increase anxiety by sonically positioning the potential of danger from all sides. The dark reds of the cave walls and floor, along with narrow passageways, form a claustrophobic underworld. In ARES, tunnels cue the viewers to duck and crawl; the ledges and narrow spaces cue the viewers to walk sideways or shimmy; the discoloration of rocks beckons the user in a particular direction. The dark brown lights in the underground cave reflect off of the particle system of dust to cloud the user’s vision and impede their progress. The second tone shift is when the user makes their way to the surface of Mars. The cave rumbles below them but the Martian sky is full of interstellar commuters. Billboards satirizing the external culture of Ares provide aesthetic enjoyment in Fig 2. The sense of familiarity is meant to achieve comedic relief. D. Time Ares is meant to take about five minutes to complete. The diegetic-time is in the not-too-distant future and spans events across a 12-hour period. Ergodic-time and negotiation-time are governed by the amount of oxygen the user has available upon awaking in the crash-landed spaceship. If the oxygen runs out, which takes seven minutes, the user dies and they restart at the beginning of the cave. If the user does not trigger the right impossible space shifts, runs out of oxygen, fails to break through rocks, or falls while climbing, ergodic-time might extend diegetic-time and consume more atomic-time. E. Topology Ares' scale is constructed through the use of impossible spaces. Each new cave structure adds to the scale of the experience. The diegetic scale of the VR experience spans from the Earth to Mars. However, the scale of playable VE is causally related to its number of impossible spaces. Depth is used in three ways. First, within the antechamber, the user is able to see Earth in one direction and the vastness of space in the other. The design signals to the user that their adventure is somewhere in the infinite expanse. Second, within the cave, vistas are used to cue the user to navigate a certain Figure 2. Satirizing contemporary culture as part of the external scenography to enhance immersion in Ares. 978-1-5386-4494-2/17/$31.00 ©2017 IEEE ████████████████████████████ way. When looking through a peep-hole, a small hole in the wall, a user can see an expansive space. This extends the perceptual stage of the VE. Third, depth is used to indicate where action is about to take place. In the final room of the cave, the user looks up a great distance to see a friendly astronaut calling out to help. V. ARES USER STUDY The purpose of this study was to understand the extent to which Bauhaus-inspired scenography was successful in communicating diegetic ideas and enhancing immersion for SOP. The study is not comparative between different VEs, but rather an evaluation of the Bauhaus design approach for VEs. Since SOP is correlated with the afforded immersive illusion, presence can be used as a baseline measure for evaluation [1]. The study was divided into three parts: First, participants completed an informed consent and demographic survey before spending two minutes in a tutorial VR scene unrelated to the narrative; second, they completed and evaluated the Ares experience, on an HTC Vive and afterwards were asked to sketch their perceived layout of the VE with pen and paper; third, participants were asked to complete the core items from the Presence Questionnaire [PQ] v3.0 [1] and answer structured interview questions posed by a member of the team to gather qualitative data regarding the design of the narrative. Audio from the interviews was recorded using QuickTime on a MacBook Air. After the study, researchers transcribed the audio and coded the transcriptions using grounded theory. TABLE II. Element of Presence PRESENCE SCORE Max Pt. Possible Avg. Std. Deviation Involvement 63 49.9 5.7 Interface Quality 14 8.2 2.3 Adaption/ Immersion Visual Fidelity 42 14 34.2 10.4 5.7 2 Total 133 103 13.4 A total of 14 respondents participated in the study, 10 males and four females, between the ages of 22 and 60 (M = 32.5, SD = 10.9). They were recruited locally through a Facebook event, a department-wide email, and flyers. One participant’s data was eliminated due to being primed about the study’s goals and design. All analyses were completed with the remaining 13 participants. Eleven participants had never used a consumer VR HMD, 2 participants had worn one within the month prior to the study. Two participants were currently involved in other projects related to VR research and design, and 6 participants considered themselves “video gamers”. All participants expressed a PQ value higher than the expected value (76, a neutral rating of 4 on all 19 items). The overall minimum was 78, and overall maximum was 122. Users felt a high sense of involvement and control; they were able to adjust to the VE quickly and the perceived sensory fidelity of the VE was high. Interface Quality lagged behind the other three factors due to ambient light polluting the field of view and the HMD cords tangling, which reminded users of being tethered to the physical world. Although this was not a comparative study, it is useful to reference to two other studies that used a version of the PQ. For example, the results of [7] show a similar breakdown of PQ factors when they compare three different platforms, one of which was a VR tool. Because they do not mention which version of PQ they used (likely the 29 or 32 item version of PQ 3.0, and not the 19 core items), an empirical comparison cannot be made. However, their results demonstrate a VE that engenders high SOP, to which Ares’ results closely matched. Additionally, [13] present data on a 20item PQ, which has the 19 core PQ 3.0 items, in an analysis of a VR aircraft inspection training simulator. The overall PQ average of the 19 core items was 92.15, with a breakdown of 23.71 (Involvement), 9.57 (Interface Quality), 30.15 (Adapation/Immersion), and 8.79 (Visual Fidelity). Moreover, the authors claimed that their results indicated a high level of presence using a domain specific VE for education. Our results suggest an even higher level of presence in Ares, using the Bauhaus approach. VI. APPLYING BAUHAUS SCENOGRAPHY IN ARES For external scenography, Ares commented on society’s obsession with celebrity and corporations. Most participants recognized the intention, and it contributed to immersion. In response to tonal choices, participants felt claustrophobic in the caverns. They contorted to fit through digital cracks and tunnels. Further, the escape route provided a sense of tonal movement. For time, participants estimated that between seven and 10 minutes of atomic-time had passed from the beginning to end of Ares, which is true. However, many thought that the story of the experience ranged anywhere from 15 minutes to one hour. This embodied temporal dissociation afforded by immersion signals SOP. Concerning topology, more than half of the participants thought that the VE felt about 6 x 6 meters and 12 meters underground. However, three participants estimated that the VE could be the size of a planet. The play area was 1.5 x 1.5 meters. Regarding movement, almost all participants perceived the cave as a ‘maze'. These sentiments were supported by both interviews and sketches of sprawling caverns. Interviews highlighted the need to appreciate different modes of human locomotion and variety of body types, some users perceived their physical bodies were too large to proceed through tight sections in the VE. Study results suggest a higher level of presence in Ares, using the Bauhaus scenographic approach for the afforded VE, as compared to similar studies [7]. The parallel between raumempfindung and the impossible space design tactic for immersion underscores these results. VII. DESIGN IMPLICATIONS AND CONCLUSION Achieving SOP through increased immersion in a VE may be enhanced through the Bauhaus approach. Study results suggest that users experienced a higher SOP in Ares than in comparable studies. The interview results indicated that the approach was particularly effective in external scenography, time, and movement. The presented language provides a clarifying framework to guide design deliberations for immersive settings that contribute to presence. A VE designer does not need to be an expert in any one of the Bauhaus scenographic elements, but they must strive to be a generalist capable of weighing and understanding the interplay of a variety of aesthetic dimensions in relation to user movement. The Bauhaus’ raumempfindung clarifies this reciprocal relationship between a user’s movement in the VE and SOP. Each new environment blossoms around the user’s movement as felt space, affording opportunities for dramatic agency in a VR experience. As sensing technologies become more robust, this research informs design considerations for VEs instantiated in unique architectures and for a diverse range of bodies. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Figure 3. Maps of the VE drawn by users as part of the study. 978-1-5386-4494-2/17/$31.00 ©2017 IEEE ████████████████████████████ B. G. Witmer and M. J. Singer, “Measuring Presence in Virtual Environments: A Presence Questionnaire,” Presence: Teleoper. Virtual Environ., vol. 7, no. 3, pp. 225–240, 1998. D. A. Bowman, “Formalizing the Design, Evaluation, and Application of Interaction Techniques for Immersive Virtual Environments,” J. Vis. Lang. Comput., vol. 10, no. 1, pp. 37–53, 1999. E. A. Suma, G. Bruder, F. Steinicke, D. M. Krum, and M. Bolas, “A taxonomy for deploying redirection techniques in immersive virtual environments,” Proc. - IEEE Virtual Real., pp. 43–46, 2012. E. A. Suma, Z. Lipps, S. Finkelstein, D. M. Krum, and M. Bolas, “Impossible spaces: Maximizing natural walking in virtual environments with self-overlapping architecture,” IEEE Trans. Vis. Comput. Graph., vol. 18, no. 4, pp. 555–564, 2012. E. J. Aarseth, “Aporia and Epiphany in Doom and The Speaking Clock,” in “Computer Technology and Literary Theory”. Bloomington: Indiana University Press, pp. 31-41, 1999. H. Freshwater. Theatre & audience. Basingstoke, Hampshire: Palgrave Macmillan, 2009. K. Chalil Madathil and J. S. Greenstein, “Synchronous Remote Usability Testing: A New Approach Facilitated by Virtual Worlds,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011, pp. 2225–2234. L. Boroditsky, “Metaphoric structuring: Understanding time through spatial metaphors,” Cognition, vol. 75, no. 1, pp. 1–28, 2000. M. Piedade Ferreira, D. de Mello, and J. P. Duarte, “The Grammar of Movement: A Step Towards a Corporeal Architecture,” Nexus Netw. J., vol. 13, no. 1, pp. 131–149, Apr. 2011. M. Slater and S. Wilbur, “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments,” Presence Teleoperators Virtual Environ., vol. 6, no. 6, pp. 603–616, 1997. P. Howard, “What Is Scenography?. 1st ed. London: Routledge, 2009. Print. R. Goldberg, Performance Art. 1st ed. London: Thames and Hudson, 1993. Print. T. A. Mikropoulos and A. Natsis, “Educational virtual environments: A ten-year review of empirical research (1999–2009),” Comput. Educ., vol. 56, no. 3, pp. 769–780, 201.