research-article

Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience

Authors:

Nimesha Ranasinghe,

Nguyen Thi Ngoc Tram,

Koon Chuan Raymond Koh,

Shienny Karwita,

Chow Eason Wai Tung,

Ching Chiuan Yen,

Ellen Yi-Luen DoAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 577, Pages 1 - 13

https://doi.org/10.1145/3173574.3174151

Published: 21 April 2018 Publication History

Abstract

In the same way that we experience the real-world through a range of senses, experiencing a virtual environment through multiple sensory modalities may augment both our presence within a scenario and our reaction to it. In this paper, we present Season Traveller, a multisensory virtual reality (VR) narration of a journey through four seasons within a mystical realm. By adding olfactory and haptic (thermal and wind) stimuli, we extend traditional audio-visual VR technologies to achieve enhanced sensory engagement within interactive experiences. Using both subjective measures of presence and elicited physiological responses, we evaluated the impact of different modalities on the virtual experience. Our results indicate that 1) the addition of any singular modality improves sense of presence with respect to traditional audio-visual experiences and 2) providing a combination of these modalities produces a further significant enhancement over the aforementioned improvements. Furthermore, insights into participants' psychophysiology were extrapolated from electrodermal activity (EDA) and heart rate (HR) measurements during each of the VR experiences.

Supplementary Material

suppl.mov (pn4524-file5.mp4)

Supplemental video

Download
12.47 MB

suppl.mov (pn4524.mp4)

Supplemental video

Download
201.20 MB

References

[1]

Carmen Adams and Lieve Doucé. 2017. What's in a scent? Meaning, shape, and sensorial concepts elicited by scents. Journal of Sensory Studies 32, 2 (2017).

[2]

Bettina A Babbitt and Charles O Nystrom. 1989. Questionnaire construction manual. Technical Report. ESSEX CORP WESTLAKE VILLAGE CA HUMAN FACTORS AND TRAINING SYSTEMS GROUP.

[3]

Sylvain Cardin, Daniel Thalmann, and Frederic Vexo. 2007. Head mounted wind. In proceeding of the 20th annual conference on Computer Animation and Social Agents (CASA2007). 101--108.

[4]

John P Cater. 1994. Smell/taste: odors in reality. In Systems, Man, and Cybernetics, 1994. Humans, Information and Technology., 1994 IEEE International Conference on, Vol. 2. IEEE, 1781--vol.

[5]

Hugo Critchley and Yoko Nagai. 2013. Electrodermal Activity (EDA). In Encyclopedia of Behavioral Medicine. Springer, 666--669.

[6]

Leonidas Deligiannidis and Robert JK Jacob. 2006. The vr scooter: Wind and tactile feedback improve user performance. In 3D User Interfaces, 2006. 3DUI 2006. IEEE Symposium on. IEEE, 143--150.

Digital Library

[7]

Huong Q Dinh, Neff Walker, Larry F Hodges, Chang Song, and Akira Kobayashi. 1999. Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In Virtual Reality, 1999. Proceedings., IEEE. IEEE, 222--228.

Digital Library

[8]

José Dionisio. 1997. Virtual hell: A trip through the flames. IEEE Computer Graphics and Applications 17, 3 (1997), 11--14.

Digital Library

[9]

Anders Drachen, Lennart E Nacke, Georgios Yannakakis, and Anja Lee Pedersen. 2010. Correlation between heart rate, electrodermal activity and player experience in first-person shooter games. In Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games. ACM, 49--54.

Digital Library

[10]

Gary Fontaine. 1992. The experience of a sense of presence in intercultural and international encounters. Presence: Teleoperators&Virtual Environments 1, 4 (1992), 482--490.

Digital Library

[11]

Barry G Green, Pamela Dalton, Beverly Cowart, Greg Shaffer, Krystyna Rankin, and Jennifer Higgins. 1996. Evaluating the 'Labeled Magnitude Scale' for measuring sensations of taste and smell. Chemical senses 21, 3 (1996), 323--334.

[12]

Barry G Green and Barbara Gelhard. 1987. Perception of temperature on oral and facial skin. Somatosensory research 4, 3 (1987), 191--200.

[13]

Ping-Hsuan Han, Chiao-En Hsieh, Yang-Sheng Chen, Jui-Chun Hsiao, Kong-Chang Lee, Sheng-Fu Ko, Kuan-Wen Chen, Chien-Hsing Chou, and Yi-Ping Hung. 2017. AoEs: enhancing teleportation experience in immersive environment with mid-air haptics. In ACM SIGGRAPH 2017 Emerging Technologies. ACM, 3.

Digital Library

[14]

Ping-Hsuan Han, Da-Yuan Huang, Hsin-Ruey Tsai, Po-Chang Chen, Chen-Hsin Hsieh, Kuan-Ying Lu, De-Nian Yang, and Yi-Ping Hung. 2015. Moving around in virtual space with spider silk. In ACM SIGGRAPH 2015 Emerging Technologies. ACM, 19.

Digital Library

[15]

Carrie Heeter. 1992. Being there: The subjective experience of presence. Presence: Teleoperators&Virtual Environments 1, 2 (1992), 262--271.

Digital Library

[16]

ML Heilig. 1962. Sensorama simulator, us patent no. 3050870. (1962).

[17]

Richard M Held and Nathaniel I Durlach. 1992. Telepresence. Presence: Teleoperators&Virtual Environments 1, 1 (1992), 109--112.

Digital Library

[18]

Felix Hülsmann, Julia Fröhlich, Nikita Mattar, and Ipke Wachsmuth. 2014. Wind and warmth in virtual reality: implementation and evaluation. In Proceedings of the 2014 Virtual Reality International Conference. ACM, 24.

Digital Library

[19]

WA IJsselsteijn, YAW De Kort, and Karolien Poels. 2008. The game experience questionnaire. Manuscript in preparation (2008).

[20]

Dhruv Jain, Misha Sra, Jingru Guo, Rodrigo Marques, Raymond Wu, Justin Chiu, and Chris Schmandt. 2016. Immersive terrestrial scuba diving using virtual reality. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1563--1569.

Digital Library

[21]

Dong P Jang, In Y Kim, Sang W Nam, Brenda K Wiederhold, Mark D Wiederhold, and Sun I Kim. 2002. Analysis of physiological response to two virtual environments: driving and flying simulation. CyberPsychology&Behavior 5, 1 (2002), 11--18.

[22]

James R Lackner and Paul DiZio. 1998. Spatial orientation as a component of presence: Insights gained from nonterrestrial environments. Presence 7, 2 (1998), 108--115.

Digital Library

[23]

Matthew Lombard and Theresa Ditton. 1997. At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication 3, 2 (1997), 0--0.

[24]

Regan L Mandryk. 2008. Physiological measures for game evaluation. Game usability: Advice from the experts for advancing the player experience (2008), 207--235.

[25]

Regan L Mandryk and M Stella Atkins. 2007. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International journal of human-computer studies 65, 4 (2007), 329--347.

Digital Library

[26]

Regan L Mandryk, M Stella Atkins, and Kori M Inkpen. 2006a. A continuous and objective evaluation of emotional experience with interactive play environments. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 1027--1036.

Digital Library

[27]

Regan L Mandryk, Kori M Inkpen, and Thomas W Calvert. 2006b. Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour&information technology 25, 2 (2006), 141--158.

[28]

Michael W McGreevy. 1992. The presence of field geologists in Mars-like terrain. Presence: Teleoperators&Virtual Environments 1, 4 (1992), 375--403.

Digital Library

[29]

Michael Meehan, Brent Insko, Mary Whitton, and Frederick P Brooks Jr. 2002. Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics (TOG) 21, 3 (2002), 645--652.

Digital Library

[30]

Michael Meehan, Sharif Razzaque, Brent Insko, Mary Whitton, and Frederick P Brooks. 2005. Review of four studies on the use of physiological reaction as a measure of presence in stressful virtual environments. Applied psychophysiology and biofeedback 30, 3 (2005), 239--258.

[31]

Mitsuru Minakuchi and Satoshi Nakamura. 2007. Collaborative ambient systems by blow displays. In Proceedings of the 1st international conference on Tangible and embedded interaction. ACM, 105--108.

Digital Library

[32]

Taeyong Moon and Gerard J Kim. 2004. Design and evaluation of a wind display for virtual reality. In Proceedings of the ACM symposium on Virtual reality software and technology. ACM, 122--128.

Digital Library

[33]

Takamichi Nakamoto and Hai Pham Dinh Minh. 2007. Improvement of olfactory display using solenoid valves. In Virtual Reality Conference, 2007. VR'07. IEEE. IEEE, 179--186.

[34]

Takuji Narumi, Takashi Kajinami, Tomohiro Tanikawa, and Michitaka Hirose. 2010. Meta cookie. In ACM SIGGRAPH 2010 Posters. ACM, 143.

Digital Library

[35]

Jonas K Olofsson. 2014. Time to smell: a cascade model of human olfactory perception based on response-time (RT) measurement. Frontiers in psychology 5 (2014).

[36]

Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. ThermoVR: Exploring Integrated Thermal Haptic Feedback with Head Mounted Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5452--5456.

Digital Library

[37]

Jerrold D Prothero, Donald E Parker, T Furness, and M Wells. 1995. Towards a robust, quantitative measure for presence. In Proceedings of the conference on Experimental Analysis and Measurement of Situation Awareness. 359--366.

[38]

Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen Yi-Luen Do. 2017. Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1731--1742.

Digital Library

[39]

Max Rheiner. 2014. Birdly an attempt to fly. In ACM SIGGRAPH 2014 Emerging Technologies. ACM, 3.

Digital Library

[40]

Michael Rietzler, Katrin Plaumann, Taras Kränzle, Marcel Erath, Alexander Stahl, and Enrico Rukzio. 2017. VaiR: Simulating 3D Airflows in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5669--5677.

Digital Library

[41]

David W Schloerb. 1995. A quantitative measure of telepresence. Presence: Teleoperators&Virtual Environments 4, 1 (1995), 64--80.

Digital Library

[42]

Martijn J Schuemie, Peter Van Der Straaten, Merel Krijn, and Charles APG Van Der Mast. 2001. Research on presence in virtual reality: A survey. CyberPsychology&Behavior 4, 2 (2001), 183--201.

[43]

Thomas B Sheridan. 1992. Musings on telepresence and virtual presence. Presence: Teleoperators&Virtual Environments 1, 1 (1992), 120--126.

Digital Library

[44]

Mel Slater and Sylvia Wilbur. 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and virtual environments 6, 6 (1997), 603--616.

Digital Library

[45]

Jonathan Steuer. 1992. Defining virtual reality: Dimensions determining telepresence. Journal of communication 42, 4 (1992), 73--93.

[46]

Martin Usoh, Ernest Catena, Sima Arman, and Mel Slater. 2000. Using presence questionnaires in reality. Presence: Teleoperators and Virtual Environments 9, 5 (2000), 497--503.

Digital Library

[47]

Joy Van Baren and Wijnand IJsselsteijn. 2004. Deliverable 5 Measuring Presence: A Guide to Current Measurement Approaches. (2004).

[48]

Jouke C Verlinden, Fabian A Mulder, Joris S Vergeest, Anna de Jonge, Darina Krutiy, Zsuzsa Nagy, Bob J Logeman, and Paul Schouten. 2013. Enhancement of presence in a virtual sailing environment through localized wind simulation. Procedia Engineering 60 (2013), 435--441.

[49]

Brenda K Wiederhold, DP Jang, Mayumi Kaneda, Irene Cabral, Yair Lurie, T May, IY Kim, Mark D Wiederhold, and SI Kim. 2001. An investigation into physiological responses in virtual environments: an objective measurement of presence. Towards cyberpsychology: Mind, cognitions and society in the internet age (2001).

[50]

Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments 7, 3 (1998), 225--240.

Digital Library

[51]

Tomoya Yamada, Satoshi Yokoyama, Tomohiro Tanikawa, Koichi Hirota, and Michitaka Hirose. 2006. Wearable olfactory display: Using odor in outdoor environment. In Virtual Reality Conference, 2006. IEEE, 199--206.

Digital Library

[52]

Yasuyuki Yanagida, Shinjiro Kawato, Haruo Noma, Akira Tomono, and N Tesutani. 2004. Projection based olfactory display with nose tracking. In Virtual Reality, 2004. Proceedings. IEEE. IEEE, 43--50.

Digital Library

[53]

David Zeltzer. 1992. Autonomy, interaction, and presence. Presence: Teleoperators&Virtual Environments 1, 1 (1992), 127--132.

Digital Library

Cited By

Da Silva MSanches IBorba JBarros AFeitosa FDe Carvalho RFilho AAndrade C(2024)Elevating Virtual Reality Experiences with Olfactory Integration: A Preliminary ReviewJournal of the Brazilian Computer Society10.5753/jbcs.2024.463230:1(639-652)Online publication date: 9-Dec-2024
https://doi.org/10.5753/jbcs.2024.4632
Rantala JSalminen KIsokoski PNieminen VKarjalainen MVäliaho JMüller PKontunen AKallio PSurakka V(2024)Recall of Odorous Objects in Virtual RealityMultimodal Technologies and Interaction10.3390/mti80600428:6(42)Online publication date: 21-May-2024
https://doi.org/10.3390/mti8060042
Kim SMyung JPark YKim S(2024)Implementing a Virtual Reality Forest Healing System Using Multisensory ModulesJournal of the Korea Computer Graphics Society10.15701/kcgs.2024.30.4.3130:4(31-39)Online publication date: 1-Sep-2024
https://doi.org/10.15701/kcgs.2024.30.4.31
Show More Cited By

Index Terms

Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience
1. Human-centered computing

Recommendations

A Demonstration of Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

This demonstration presents Season Traveller, a multisensory virtual reality (VR) narration of a journey through four seasons within a mystical realm. By adding olfactory and haptic (thermal and wind) stimuli, we extend traditional audio-visual VR ...
A Review of Olfactory Display Designs for Virtual Reality Environments
The field of Virtual Reality continues to evolve to provide an ever-greater sense of immersion to the user. However, VR experiences are still primarily constrained through the human senses of vision and audition, with some interest in haptic (mainly ...
Physically touching and tasting virtual objects enhances the realism of virtual experiences

Experiment 1 explored the impact of physically touching a virtual object on how realistic the virtual environment (VE) seemed to the user. Subjects in a `no touch' group picked up a 3D virtual image of a kitchen plate in a VE, using a traditional 3D ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

135
Total Citations
View Citations
2,062
Total Downloads

Downloads (Last 12 months)280
Downloads (Last 6 weeks)33

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Da Silva MSanches IBorba JBarros AFeitosa FDe Carvalho RFilho AAndrade C(2024)Elevating Virtual Reality Experiences with Olfactory Integration: A Preliminary ReviewJournal of the Brazilian Computer Society10.5753/jbcs.2024.463230:1(639-652)Online publication date: 9-Dec-2024
https://doi.org/10.5753/jbcs.2024.4632
Rantala JSalminen KIsokoski PNieminen VKarjalainen MVäliaho JMüller PKontunen AKallio PSurakka V(2024)Recall of Odorous Objects in Virtual RealityMultimodal Technologies and Interaction10.3390/mti80600428:6(42)Online publication date: 21-May-2024
https://doi.org/10.3390/mti8060042
Kim SMyung JPark YKim S(2024)Implementing a Virtual Reality Forest Healing System Using Multisensory ModulesJournal of the Korea Computer Graphics Society10.15701/kcgs.2024.30.4.3130:4(31-39)Online publication date: 1-Sep-2024
https://doi.org/10.15701/kcgs.2024.30.4.31
Spence CGao Y(2024)Augmenting home entertainment with digitally delivered touchi-Perception10.1177/2041669524128147415:5Online publication date: 17-Oct-2024
https://doi.org/10.1177/20416695241281474
Spence CDi Stefano NReinoso-Carvalho FVelasco C(2024)Marketing sonified fragrance: Designing soundscapes for scenti-Perception10.1177/2041669524125971415:4Online publication date: 2-Aug-2024
https://doi.org/10.1177/20416695241259714
Dawes CXue JBrianza GCornelio PMontano Murillo RMaggioni EObrist M(2024)ScentHaptics: Augmenting the Haptic Experiences of Digital Mid-Air Textiles with ScentProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685715(47-56)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685715
Tewell JRanasinghe N(2024)A Review of Olfactory Display Designs for Virtual Reality EnvironmentsACM Computing Surveys10.1145/366524356:11(1-35)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3665243
Artizzu VLuyten KRuiz GSpano L(2024)ViRgilites: Multilevel Feedforward for Multimodal Interaction in VRProceedings of the ACM on Human-Computer Interaction10.1145/36586458:EICS(1-24)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3658645
Wang HSinghal YGil HKim J(2024)Fiery Hands: Designing Thermal Glove through Thermal and Tactile Integration for Virtual Object ManipulationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676457(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676457
Gao PLiu FWen DGao YZhang LWang CZhang QZhang YMa SLu QMi HXu Y(2024)Mul-O: Encouraging Olfactory Innovation in Various Scenarios Through a Task-Oriented Development PlatformProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676387(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676387
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents