skip to main content
10.1145/3543758.3543768acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmundcConference Proceedingsconference-collections
research-article

A Touch of Realities: Car-Interior-Based Haptic Interaction Supports In-Car VR Recovery from Interruptions

Published: 15 September 2022 Publication History

Abstract

Real-world interruptions will challenge virtual reality (VR) users in future everyday transport. For example, while passengers are immersed at a virtual beach, an incoming phone call might interrupt their presence and relaxation. We investigated how to help users recover from such interruptions by exploring haptic and visual cues that help them recall their prior presence in VR. We approached this by developing a passive haptic display for rear-seat passengers using an interactive armrest. In a lab study (N=30), participants played with virtual sand to relax, feeling the changes in the real armrest and seeing them on the virtual beach. We compared this multi-sensory experience to the single modalities (just visuals or just haptics). The results showed that the multi-modal experience lowered awareness of the armrest more and fostered a feeling of connectedness to the virtual world after real-world interruptions. We propose using car-interior-based haptic displays to support in-car VR recovery from interruptions.

References

[1]
Mohammed Al-Sada, Keren Jiang, Shubhankar Ranade, Xinlei Piao, Thomas Höglund, and Tatsuo Nakajima. 2018. HapticSerpent: A Wearable Haptic Feedback Robot for VR. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188518
[2]
Laura Bajorunaite, Stephen Brewster, and Julie R. Williamson. 2022. “Reality Anchors”: Bringing Cues from Reality into VR on Public Transport to Alleviate Safety and Comfort Concerns. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 383, 6 pages. https://doi.org/10.1145/3491101.3519696
[3]
Klaus Bengler, Michael Rettenmaier, Nicole Fritz, and Alexander Feierle. 2020. From HMI to HMIs: Towards an HMI Framework for Automated Driving. Information 11, 2 (Jan. 2020), 61. https://doi.org/10.3390/info11020061
[4]
Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 717–728. https://doi.org/10.1145/2984511.2984526
[5]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 2 (Jan. 2006), 77–101. https://doi.org/10.1191/1478088706qp063oa
[6]
Hyung-Jun Cho and Gerard J Kim. 2020. RoadVR: Mitigating the Effect of Vection and Sickness by Distortion of Pathways for In-Car Virtual Reality. In 26th ACM Symposium on Virtual Reality Software and Technology (Virtual Event, Canada) (VRST ’20, Article 70). Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/3385956.3422115
[7]
Jaeyong Chung and Henry J Gardner. 2012. Temporal Presence Variation in Immersive Computer Games. International Journal of Human–Computer Interaction 28, 8 (Aug. 2012), 511–529. https://doi.org/10.1080/10447318.2011.627298
[8]
Florian Daiber, Donald Degraen, André Zenner, Frank Steinicke, Oscar Javier Ariza Núñez, and Adalberto L Simeone. 2020. Everyday Proxy Objects for Virtual Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375165
[9]
Cathy Fang, Yang Zhang, Matthew Dworman, and Chris Harrison. 2020. Wireality: Enabling Complex Tangible Geometries in Virtual Reality with Worn Multi-String Haptics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3313831.3376470
[10]
Tom Alexander Garner, Wendy Powell, and Vaughan Powell. 2018. Everyday Virtual Reality. In Encyclopedia of Computer Graphics and Games, Newton Lee (Ed.). Springer International Publishing, Cham, 1–9. https://doi.org/10.1007/978-3-319-08234-9_259-1
[11]
C George, A N Tien, and H Hussmann. 2020. Seamless, Bi-directional Transitions along the Reality-Virtuality Continuum: A Conceptualization and Prototype Exploration. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). ieeexplore.ieee.org, Porto de Galinhas, Brazil, 412–424. https://doi.org/10.1109/ISMAR50242.2020.00067
[12]
Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 4021–4033. https://doi.org/10.1145/3025453.3025683
[13]
Ping-Hsuan Han, Yang-Sheng Chen, Kong-Chang Lee, Hao-Cheng Wang, Chiao-En Hsieh, Jui-Chun Hsiao, Chien-Hsing Chou, and Yi-Ping Hung. 2018. Haptic around: Multiple Tactile Sensations for Immersive Environment and Interaction in Virtual Reality. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (Tokyo, Japan) (VRST ’18). Association for Computing Machinery, New York, NY, USA, Article 35, 10 pages. https://doi.org/10.1145/3281505.3281507
[14]
Daniel Harley, Alexander Verni, Mackenzie Willis, Ashley Ng, Lucas Bozzo, and Ali Mazalek. 2018. Sensory VR: Smelling, Touching, and Eating Virtual Reality. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction(Stockholm, Sweden) (TEI ’18). Association for Computing Machinery, New York, NY, USA, 386–397. https://doi.org/10.1145/3173225.3173241
[15]
Anuruddha Hettiarachchi and Daniel Wigdor. 2016. Annexing Reality: Enabling Opportunistic Use of Everyday Objects as Tangible Proxies in Augmented Reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 1957–1967. https://doi.org/10.1145/2858036.2858134
[16]
Philipp Hock, Sebastian Benedikter, Jan Gugenheimer, and Enrico Rukzio. 2017. CarVR: Enabling In-Car Virtual Reality Entertainment. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 4034–4044. https://doi.org/10.1145/3025453.3025665
[17]
Holoride. 2019. holoride: Virtual Reality meets the real world. https://www.audi.com/en/experience-audi/mobility-and-trends/digitalization/holoride-virtual-reality-meets-the-real-world.html. Accessed: 2021-9-10.
[18]
Hsin-Yu Huang, Po-Yao (Cosmos) Wang, Jen-Hao Cheng, Chih-Wei Ning, Ping-Yi Wang, and Lung-Pan Cheng. 2020. Haptic-go-round: A Surrounding Platform for Encounter-type Haptic in Virtual Reality Experiences. In ACM SIGGRAPH 2020 Immersive Pavilion (Virtual Event, USA) (SIGGRAPH ’20, Article 8). Association for Computing Machinery, New York, NY, USA, 1–2. https://doi.org/10.1145/3388536.3407886
[19]
Matthew Lakier, Lennart E Nacke, Takeo Igarashi, and Daniel Vogel. 2019. Cross-Car, Multiplayer Games for Semi-Autonomous Driving. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY ’19). Association for Computing Machinery, New York, NY, USA, 467–480. https://doi.org/10.1145/3311350.3347166
[20]
Jingyi Li, Filippe Frulli, Stella Clarke, and Andreas Butz. 2022. Towards Balancing Real-World Awareness and VR Immersion in Mobile VR. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 436, 6 pages. https://doi.org/10.1145/3491101.3519824
[21]
Jingyi Li, Ceenu George, Andrea Ngao, Kai Holländer, Stefan Mayer, and Andreas Butz. 2020. An Exploration of Users’ Thoughts on Rear-Seat Productivity in Virtual Reality. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 92–95. https://doi.org/10.1145/3409251.3411732
[22]
Jingyi Li, Ceenu George, Andrea Ngao, Kai Holländer, Stefan Mayer, and Andreas Butz. 2021. Rear-Seat Productivity in Virtual Reality: Investigating VR Interaction in the Confined Space of a Car. Multimodal Technologies and Interaction 5, 4 (March 2021), 15. https://doi.org/10.3390/mti5040015
[23]
Jingyi Li and Linda Hirsch. 2021. Multi-Modal Transition and Traces in Everyday Mobile Virtual Reality. In ISS’21 Workshop Proceedings: ”Transitional Interfaces in Mixed and Cross-Reality: A new frontier?”, Hans-Christian Jetter, Jan-Henrik Schröder, Jan Gugenheimer, Mark Billinghurst, Christoph Anthes, Mohamed Khamis, and Tiare Feuchtner (Eds.). https://doi.org/10.18148/kops/352-2-1dhqpjucpo75v9
[24]
Jingyi Li, Yong Ma, Puzhen Li, and Andreas Butz. 2021. A Journey Through Nature: Exploring Virtual Restorative Environments as a Means to Relax in Confined Spaces. In Creativity and Cognition (Virtual Event, Italy) (C&C ’21, Article 22). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3450741.3465248
[25]
Jingyi Li, Alexandra Mayer, and Andreas Butz. 2021. Towards a Design Space of Haptics in Everyday Virtual Reality across Different Spatial Scales. Multimodal Technologies and Interaction 5, 7 (July 2021), 36. https://doi.org/10.3390/mti5070036
[26]
Jingyi Li, Agnes Reda, and Andreas Butz. 2021. Queasy Rider: How Head Movements Influence Motion Sickness in Passenger Use of Head-Mounted Displays. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United Kingdom) (AutomotiveUI ’21). Association for Computing Machinery, New York, NY, USA, 28–38. https://doi.org/10.1145/3409118.3475137
[27]
Christian Mai, Niklas Thiem, and Heinrich Hussmann. 2019. DrawingPresence: A Method for Assessing Temporal Fluctuations of Presence Status in a VR Experience. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 4 (Dec. 2019), 1–21. https://doi.org/10.1145/3369827
[28]
Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2143–2152. https://doi.org/10.1145/2702123.2702382
[29]
Mark McGill, Stephen Brewster, David McGookin, and Graham Wilson. 2020. Acoustic Transparency and the Changing Soundscape of Auditory Mixed Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3313831.3376702
[30]
Mark McGill, Stephen Brewster, David McGookin, and Graham Wilson. 2020. Acoustic Transparency and the Changing Soundscape of Auditory Mixed Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI’20). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3313831.3376702
[31]
Mark McGill, Alexander Ng, and Stephen Brewster. 2017. I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5655–5668. https://doi.org/10.1145/3025453.3026046
[32]
Mark McGill, Julie Williamson, Alexander Ng, Frank Pollick, and Stephen Brewster. 2019. Challenges in passenger use of mixed reality headsets in cars and other transportation. Virtual Real. (Dec. 2019), 583––603. https://doi.org/10.1007/s10055-019-00420-x
[33]
Yoichi Ochiai, Kota Kumagai, Takayuki Hoshi, Satoshi Hasegawa, and Yoshio Hayasaki. 2016. Cross-Field Aerial Haptics: Rendering Haptic Feedback in Air with Light and Acoustic Fields. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3238–3247. https://doi.org/10.1145/2858036.2858489
[34]
Joseph O’Hagan and Julie R Williamson. 2020. Reality aware VR headsets. In Proceedings of the 9TH ACM International Symposium on Pervasive Displays (Manchester, United Kingdom) (PerDis ’20). Association for Computing Machinery, New York, NY, USA, 9–17. https://doi.org/10.1145/3393712.3395334
[35]
Joseph O’Hagan and Julie R. Williamson. 2020. Reality Aware VR Headsets. In Proceedings of the 9TH ACM International Symposium on Pervasive Displays (Manchester, United Kingdom) (PerDis ’20). Association for Computing Machinery, New York, NY, USA, 9–17. https://doi.org/10.1145/3393712.3395334
[36]
Pablo E Paredes, Stephanie Balters, Kyle Qian, Elizabeth L Murnane, Francisco Ordóñez, Wendy Ju, and James A Landay. 2018. Driving with the Fishes: Towards Calming and Mindful Virtual Reality Experiences for the Car. (Dec. 2018). https://doi.org/10.1145/3287062
[37]
Holly Robbins, Elisa Giaccardi, and Elvin Karana. 2016. Traces as an Approach to Design for Focal Things and Practices. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (Gothenburg, Sweden) (NordiCHI ’16). Association for Computing Machinery, New York, NY, USA, Article 19, 10 pages. https://doi.org/10.1145/2971485.2971538
[38]
Rufat Rzayev, Susanne Korbely, Milena Maul, Alina Schark, Valentin Schwind, and Niels Henze. 2020. Effects of Position and Alignment of Notifications on AR Glasses during Social Interaction. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society(Tallinn, Estonia) (NordiCHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3419249.3420095
[39]
Rufat Rzayev, Sven Mayer, Christian Krauter, and Niels Henze. 2019. Notification in VR: The Effect of Notification Placement, Task and Environment. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY ’19). Association for Computing Machinery, New York, NY, USA, 199–211. https://doi.org/10.1145/3311350.3347190
[40]
Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The experience of presence: Factor analytic insights. Presence 10, 3 (June 2001), 266–281. https://doi.org/10.1162/105474601300343603
[41]
Mel Slater. 2009. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3549–3557. https://doi.org/10.1098/rstb.2009.0138
[42]
Shan-Yuan Teng, Tzu-Sheng Kuo, Chi Wang, Chi-Huan Chiang, Da-Yuan Huang, Liwei Chan, and Bing-Yu Chen. 2018. PuPoP: Pop-up Prop on Palm for Virtual Reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York, NY, USA, 5–17. https://doi.org/10.1145/3242587.3242628
[43]
Shan-Yuan Teng, Cheng-Lung Lin, Chi-Huan Chiang, Tzu-Sheng Kuo, Liwei Chan, Da-Yuan Huang, and Bing-Yu Chen. 2019. TilePoP: Tile-type Pop-up Prop for Virtual Reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 639–649. https://doi.org/10.1145/3332165.3347958
[44]
Wenn-Chieh Tsai and Elise van den Hoven. 2018. Memory Probes: Exploring Retrospective User Experience Through Traces of Use on Cherished Objects. International Journal of Design 12, 3 (2018), 57–72. http://www.ijdesign.org/index.php/IJDesign/article/view/2900/835
[45]
Dangxiao Wang, Yuan Guo, Shiyi Liu, Yuru Zhang, Weiliang Xu, and Jing Xiao. 2019. Haptic display for virtual reality: progress and challenges. Virtual Reality & Intelligent Hardware 1, 2 (April 2019), 136–162. https://doi.org/10.3724/SP.J.2096-5796.2019.0008
[46]
Yuntao Wang, Zichao (tyson) Chen, Hanchuan Li, Zhengyi Cao, Huiyi Luo, Tengxiang Zhang, Ke Ou, John Raiti, Chun Yu, Shwetak Patel, and Yuanchun Shi. 2020. MoveVR: Enabling Multiform Force Feedback in Virtual Reality using Household Cleaning Robot. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376286
[47]
Julie R Williamson, Mark McGill, and Khari Outram. 2019. PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19, Paper 80). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300310
[48]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963

Cited By

View all

Index Terms

  1. A Touch of Realities: Car-Interior-Based Haptic Interaction Supports In-Car VR Recovery from Interruptions

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      MuC '22: Proceedings of Mensch und Computer 2022
      September 2022
      624 pages
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 September 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. HMD
      2. haptic display
      3. in-car VR
      4. interruption recovery

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      Conference

      MuC '22
      MuC '22: Mensch und Computer 2022
      September 4 - 7, 2022
      Darmstadt, Germany

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)105
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 05 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media