skip to main content
10.1145/3173574.3173690acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation

Published: 19 April 2018 Publication History

Abstract

Traditional virtual reality (VR) mainly focuses on visual feedback, which is not accessible for people with visual impairments. We created Canetroller, a haptic cane controller that simulates white cane interactions, enabling people with visual impairments to navigate a virtual environment by transferring their cane skills into the virtual world. Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller. Our study showed that Canetroller was a promising tool that enabled visually impaired participants to navigate different virtual spaces. We discuss potential applications supported by Canetroller ranging from entertainment to mobility training.

Supplementary Material

ZIP File (pn1758.sbv.zip)
The auxiliary material includes the captions for the video preview.
suppl.mov (pn1758-file5.mp4)
Supplemental video
suppl.mov (pn1758.mp4)
Supplemental video

References

[1]
Rebecca Adams, Paul Finn, Elisabeth Moes, Kathleen Flannery, and Albert "Skip" Rizzo. 2009. Distractibility in Attention/Deficit/ Hyperactivity Disorder (ADHD): The Virtual Reality Classroom. Child Neuropsychology 15, 2: 120--135.
[2]
B. Ando, S. Baglio, V. Marletta, and A. Valastro. 2015. A Haptic Solution to Assist Visually Impaired in Mobility Tasks. IEEE Transactions on HumanMachine Systems 45, 5: 641--646.
[3]
Bruno Andò, Salvatore Baglio, Cristian Orazio Lombardo, Vincenzo Marletta, E. A. Pergolizzi, Antonio Pistorio, and Angelo Emanuele Valastro. 2015. An Electronic Cane with a Haptic Interface for Mobility Tasks. Springer, Cham, 189--200.
[4]
Bruno Andò, Salvatore Baglio, Vincenzo Marletta, and Angelo Valastro. 2017. A Tilt Compensated Haptic Cane for Obstacle Detection. Springer, Cham, 141-- 151.
[5]
Bruno Andò, Salvatore Baglio, and Nicola Pitrone. 2008. A Contactless Haptic Cane for Blind People. In 12th IMEKO TC1 & TC7 Joint Symposium on Man Science & Measurement, 147--152.
[6]
Joseph Bates. 1992. Virtual Reality, Art, and Entertainment. Presence: Teleoperators and Virtual Environments 1, 1: 133--138.
[7]
J Bell, S Bolanowski, and M H Holmes. 1994. The structure and function of Pacinian corpuscles: a review. Progress in neurobiology 42, 1: 79--128. Retrieved September 5, 2017 from http://www.ncbi.nlm.nih.gov/ /7480788
[8]
J. Borenstein and I. Ulrich. 1997. The GuideCane-a computerized travel aid for the active guidance of blind pedestrians. In Proceedings of International Conference on Robotics and Automation, 1283--1288.
[9]
Chetz Colwell, Helen Petrie, Andrew Hardwick, and Martlesham Heath. 1998. Use of a haptic device by blind and sighted people?: perception of virtual textures and objects. Improving the Quality of Life for the European Citizen?: Technology for Inclusive Design and Equality, JULY: 1--8.
[10]
Chetz Colwell, Helen Petrie, Diana Kornbrot, Andrew Hardwick, and Stephen Furner. 1998. Haptic virtual reality for blind computer users. In Proceedings of the third international ACM conference on Assistive technologies - Assets '98, 92--99.
[11]
J. L. González-Mora, A. Rodríguez-Hernández, L. F. Rodríguez-Ramos, L. Díaz-Saco, and N. Sosa. 1999. Development of a new space perception system for blind people, based on the creation of a virtual acoustic space. Springer, Berlin, Heidelberg, 321--330.
[12]
José Luis González-Mora, Antonio Francisco Rodríguez-Hernández, Enrique Burunat, F. Martin, and M A Castellano. 2006. Seeing the world by hearing: Virtual Acoustic Space (VAS) a new space perception system for blind people. In 2nd International Conference on Information & Communication Technologies, 837--842.
[13]
E W Hill, J. J. Rieser, M. Hill, J Halpin, and R Halpin. 1993. How persons with visual impairments explore novel spaces? A study of strategies used by exceptionally good and exceptionally poor performers. J Vis Impair Blind 87: 295--301. Retrieved August 31, 2017 from http://psycnet.apa.org/record/1994--18302001
[14]
D. Jack, R. Boian, A.S. Merians, M. Tremaine, G.C. Burdea, S.V. Adamovich, M. Recce, and H. Poizner. 2001. Virtual reality-enhanced stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 9, 3: 308--318.
[15]
W. H. Jacobson. 1993. The Art and Science of Teaching Orientation and Mobility to Persons with Visual Impairments. AFB Press.
[16]
G Jansson, H Petrie, C Colwell, and D Kornbrot. 1999. Haptic virtual environments for blind people: Exploratory experiments with two devices. The International Journal of Virtual Reality 3, 4: 8--17. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/348e/45107167a03250 51e60c883c153572a127e4.pdf
[17]
Katerina Kalyvioti and Tassos A. Mikropoulos. 2014. Virtual Environments and Dyslexia: A Literature Review. Procedia Computer Science 27: 138--147.
[18]
Yoshihiro Kawai and Fumiaki Tomita. Evaluation of Interactive Tactile Display System. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/c8da/db1e1caa0b4e31 7db3bb8f22c6a45a52f8c6.pdf
[19]
Dae Shik Kim, Robert Wall Emerson, and Amy Curtis. 2009. Drop-off Detection with the Long Cane: Effects of Different Cane Techniques on Performance. Journal of visual impairment & blindness 103, 9: 519--530. Retrieved September 2, 2017 from http://www.ncbi.nlm.nih.gov/ /21209791
[20]
Henry König, Jochen Schneider, and Thomas Strothotte. Haptic Exploration of Virtual Buildings Using Non-Realistic Haptic Rendering. Retrieved August 31, 2017 from https://pdfs.semanticscholar.org/5f54/198fc98e01be1b 73eb1d31b4ef6b95070792.pdf
[21]
Michael Kotlyar, Christopher Donahue, Paul Thuras, Matt G. Kushner, Natalie O'Gorman, Erin A Smith, and David E. Adson. 2008. Physiological response to a speech stressor presented in a virtual reality environment. Psychophysiology 45, 6: 1034--1037.
[22]
O Lahav and D Mioduser. Multisensory virtual environment for supporting blind persons' acquisition of spatial cognitive mapping, orientation, and mobility skills. Retrieved August 31, 2017 from http://playpen.icomtek.csir.co.za/~acdc/assistive devices/Artabilitation2008/archive/2002/papers/2002_28.pdf
[23]
Orly Lahav and David Mioduser. 2004. Exploration of Unknown Spaces by People Who Are Blind Using a Multi-sensory Virtual Environment. Journal of Special Education Technology 19, 3: 15--23. Retrieved August 31, 2017 from
[24]
Orly Lahav and David Mioduser. 2008. Construction of cognitive maps of unknown spaces using a multisensory virtual environment for people who are blind. Computers in Human Behavior 24, 3: 1139--1155.
[25]
Mark L. Latash, EA Atree, BM Brooks, DA Johnson, EA Attree, A Bellner, M Rydmark, H Poizner, L Hughey, CL Richards, Y Bisson, A Rovetta, S Rushton, C Selis, and J Wann. 1998. Virtual reality: A fascinating tool for motor rehabilitation (to be used with caution). Disability and Rehabilitation 20, 3: 104-- 105.
[26]
A. Lecuyer, P. Mobuchon, C. Megard, J. Perret, C. Andriot, and J.-P. Colinot. HOMERE: a multimodal system for visually impaired people to explore virtual environments. In IEEE Virtual Reality, 2003. Proceedings., 251--258.
[27]
Fabrizio Leo, Elena Cocchi, and Luca Brayda. 2017. The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25, 7: 861--872.
[28]
Shachar Maidenbaum, Shelly Levy-Tzedek, Daniel Robert Chebat, and Amir Amedi. 2013. Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the "EyeCane": Feasibility study. PLoS ONE 8, 8: e72555.
[29]
Willie Martin, Kory Dancer, Kevin Rock, Christopher Zeleney, and Kumar Yelamarthi. 2009. The Smart Cane?: An Electrical Engineering Design Project. 1--9. Retrieved August 31, 2017 from http://people.cst.cmich.edu/yelam1k/CASE/Publication s_files/Yelamarthi_ASEE_NCS_2009.pdf
[30]
M. P. Menikdiwela, K.M.I.S. Dharmasena, and A.M. Harsha S. Abeykoon. 2013. Haptic based walking stick for visually impaired people. In 2013 International conference on Circuits, Controls and Communications (CCUBE), 1--6.
[31]
Lotfi B Merabet and Jaime Sanchez. 2009. AudioBased Navigation Using Virtual Environments: Combining Technology and Neuroscience. AER Journal: Research and Practice in Visual Impairment and Blindness 2: 128--137.
[32]
Zahira Merchant, Ernest T. Goetz, Lauren Cifuentes, Wendy Keeney-Kennicutt, and Trina J. Davis. 2014. Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education 70: 29--40.
[33]
Alma S Merians, David Jack, Rares Boian, Marilyn Tremaine, Grigore C Burdea, Sergei V Adamovich, Michael Recce, and Howard Poizner. 2002. Virtual Reality--Augmented Rehabilitation for Patients Following Stroke. Physical Therapy 37, 1: 975--987.
[34]
Thomas D. Parsons, Todd Bowerly, J. Galen Buckwalter, and Albert A. Rizzo. 2007. A Controlled Clinical Comparison of Attention Performance in Children with ADHD in a Virtual Reality Classroom Compared to Standard Neuropsychological Methods. Child Neuropsychology 13, 4: 363--381.
[35]
Maurizio de Pascale, Sara Mulatto, and Domenico Prattichizzo. 2008. Bringing Haptics to Second Life for Visually Impaired People. Springer, Berlin, Heidelberg, 896--905.
[36]
Lorenzo Picinali, Amandine Afonso, Michel Denis, and Brian F.G. Katz. 2014. Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge. International Journal of Human Computer Studies 72, 4: 393--407.
[37]
S. Reardon. 2011. Playing by Ear. Science 333, 6051: 1816--1818.
[38]
Irvin Rock and Jack Victor. 1964. Vision and Touch: An Experimentally Created Conflict between the Two Senses. Science 143, 3606. Retrieved August 28, 2017 from http://science.sciencemag.org/content/143/3606/594
[39]
Joseph M. Romano and Katherine J. Kuchenbecker. 2012. Creating Realistic Virtual Textures from Contact Acceleration Data. IEEE Transactions on Haptics 5, 2: 109--119.
[40]
J H Sánchez and M A Sáenz. 2006. Assisting the mobilization through subway networks by users with visual disabilities. Virtual Reality & Assoc. Tech. Retrieved August 30, 2017 from www.dcc.uchile.cl/~jsanchez
[41]
JAIME SÁNCHEZ and MAURICIO LUMBRERAS. 1999. Virtual Environment Interaction Through 3D Audio by Blind Children. CyberPsychology & Behavior 2, 2: 101--111.
[42]
David W. Schloerb, Orly Lahav, Joseph G. Desloge, and Mandayam A. Srinivasan. 2010. BlindAid: Virtual environment system for self-reliant trip planning and orientation and mobility training. In 2010 IEEE Haptics Symposium, 363--370.
[43]
Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10, 3: 266--281.
[44]
Yoshikazu Seki and Tetsuji Sato. 2011. A training system of orientation and mobility for blind people using acoustic virtual reality. IEEE Transactions on Neural Systems and Rehabilitation Engineering 19, 1: 95--104.
[45]
SK Semwal. 2001. MoVE: Mobiltiy training in haptic virtual environment. piglet. uccs. edu/~ semwal/NSF2001PS. pdf: 1--18.
[46]
Vaibhav Singh, Rohan Paul, Dheeraj Mehra, Anuraag Gupta, Vasu Dev Sharma, Saumya Jain, Chinmay Agarwal, Ankush Garg, Sandeep Singh Gujral, M Balakrishnan, and K Paul. 2010. "Smart" Cane for the Visually Impaired: Design and Controlled Field Testing of an Affordable Obstacle Detection System. TRANSED 2010: 12th International Conference on Mobility and Transport for Elderly and Disabled Persons 53, 9: 1689--1699.
[47]
Lore Thaler, Galen M. Reich, Xinyu Zhang, Dinghe Wang, Graeme E. Smith, Zeng Tao, Raja Syamsul Azmir Bin. Raja Abdullah, Mikhail Cherniakov, Christopher J. Baker, Daniel Kish, and Michail Antoniou. 2017. Mouth-clicks used by blind expert human echolocators -- signal description and model based signal synthesis. PLOS Computational Biology 13, 8: e1005670.
[48]
Virgil Tiponut, Zoltan Haraszy, Daniel Ianchis, and Ioan Lie. 2008. Acoustic virtual reality performing man-machine interfacing of the blind. In Proceedings of the 12th WSEAS international conference on Systems, 345--349.
[49]
M A Torres-Gil, O Casanova-Gonzalez, and José Luis González-Mora. 2010. Applications of virtual reality for visually impaired people. WSEAS Transactions on Computers 9, 2: 184--193. Retrieved from http://dl.acm.org/citation.cfm?id=1852403.1852412
[50]
Shari Trewin, Mark Laff, Vicki Hanson, and Anna Cavender. 2009. Exploring Visual and Motor Accessibility in Navigating a Virtual World. ACM Transactions on Accessible Computing 2, 2: 1--35.
[51]
D. Tzovaras, G. Nikolakis, G. Fergadis, S. Malasiotis, and M. Stavrakis. 2002. Design and implementation of virtual environments training of the visually impaire. Proceedings of the fifth international ACM conference on Assistive technologies - Assets '02 12, 2: 41.
[52]
Dimitrios Tzovaras, Konstantinos Moustakas, Georgios Nikolakis, and Michael G. Strintzis. 2009. Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Personal and Ubiquitous Computing 13, 1: 51--58.
[53]
Iwan Ulrich and Johann Borenstein. 2001. The GuideCane - Applying Mobile Robot Technologies to Assist the Visually Impaired. IEEE Transactions on Systems, Man, and Cybernetics, -Part A: Systems and Humans 31, 2: 131--136. Retrieved August 31, 2017 from http://www.cs.unc.edu/~welch/class/mobility/papers/G uideCane.pdf
[54]
Maria Virvou and George Katsionis. 2008. On the usability and likeability of virtual reality games for education: The case of VR-ENGAGE. Computers and Education 50, 1: 154--178.
[55]
M Helmy Abd Wahab, Aa Talib, and Ha Kadir. 2011. Smart Cane: Assistive Cane for Visually-impaired People. IJCSI International Journal of Computer Science Issues 8, 4: 21--27. https://doi.org/1694-0814
[56]
Steven Wall and Stephen Brewster. 2006. Feeling What You Hear?: Tactile Feedback for Navigation of Audio Graphs. Conference on Human Factors in Computing Systems, April: 1123--1132.
[57]
Gareth R. White, Geraldine Fitzpatrick, and Graham McAllister. 2008. Toward accessible 3D virtual environments for the blind and visually impaired. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts DIMEA '08, 134.
[58]
Michele A Williams, Caroline Galbraith, Shaun K Kane, and Amy Hurst. "Just Let the Cane Hit It": How the Blind and Sighted See Navigation Differently. In Proceedings of the 16th interal ACM SIGACCESS conference on Computers & Accessibility - ASSETS'14. 271--224.
[59]
Michele a Williams, Amy Hurst, and Shaun K Kane. 2013. "Pray Before You Step out": Describing Personal and Situational Blind Navigation Behaviors. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility: 28:1---28:8.
[60]
Limin Zeng, Mei Miao, and Gerhard Weber. 2012. Interactive Audio-haptic Map Explorer on a Tactile Display. Interacting with Computers 27, 4: 413--429.
[61]
M. Zyda. 2005. From Visual Simulation to Virtual Reality to Games. Computer 38, 9: 25--32.
[62]
How Virtual Reality Facilitates Social Connection | Facebook IQ. Retrieved August 28, 2017 from https://www.facebook.com/iq/articles/how-virtualreality-facilitates-social-connection
[63]
Phantom Force Feedback. Retrieved August 31, 2017 from http://www.cgl.ucsf.edu/chimera/1.2470/docs/Contribu tedSoftware/phantom/phantom.html
[64]
Novint - Products. Retrieved August 31, 2017 from http://www.novint.com/index.php/products
[65]
Geomagic Touch (formerly Geomagic Phantom Omni) Overview. Retrieved August 31, 2017 from http://www.geomagic.com/en/products/phantomomni/overview
[66]
Long Cane Techniques, Study Guide: APH Step-byStep. Retrieved September 2, 2017 from http://tech.aph.org/sbs/04_sbs_lc_study.html
[67]
igroup presence questionnaire (IPQ) Database | igroup.org -- project consortium. Retrieved September 11, 2017 from http://www.igroup.org/pq/ipq/data.php

Cited By

View all

Index Terms

  1. Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
      April 2018
      8489 pages
      ISBN:9781450356206
      DOI:10.1145/3173574
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 April 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. auditory feedback
      2. blindness
      3. haptic feedback
      4. mobility
      5. virtual reality
      6. visual impairments
      7. white cane

      Qualifiers

      • Research-article

      Conference

      CHI '18
      Sponsor:

      Acceptance Rates

      CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Upcoming Conference

      CHI 2025
      ACM CHI Conference on Human Factors in Computing Systems
      April 26 - May 1, 2025
      Yokohama , Japan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)429
      • Downloads (Last 6 weeks)57
      Reflects downloads up to 23 Dec 2024

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media