research-article

NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment

Authors:

Hironobu Takagi,

Chieko AsakawaAuthors Info & Claims

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

Pages 270 - 279

https://doi.org/10.1145/3132525.3132535

Published: 19 October 2017 Publication History

Abstract

Navigating in unfamiliar environments is challenging for most people, especially for individuals with visual impairments. While many personal navigation tools have been proposed to enable in- dependent indoor navigation, they have insufficient accuracy (e.g., 5-10 m), do not provide semantic features about surroundings (e.g., doorways, shops, etc.), and may require specialized devices to function. Moreover, the deployment of many systems is often only evaluated in constrained scenarios, which may not precisely reflect the performance in the real world. Therefore, we have de- signed and implemented NavCog3, a smartphone-based indoor navigation assistant that has been evaluated in a 21,000 m2 shop- ping mall. In addition to turn-by-turn instructions, it provides in- formation on landmarks (e.g., tactile paving) and points of interests nearby. We first conducted a controlled study with 10 visually im- paired users to assess localization accuracy and the perceived use- fulness of semantic features. To understand the usability of the app in a real-world setting, we then conducted another study with 43 participants with visual impairments where they could freely nav- igate in the shopping mall using NavCog3. Our findings suggest that NavCog3 can open a new opportunity for users with visual im- pairments to independently find and visit large and complex places with confidence.

References

[1]

Draft development specification for spatial network model for pedestrians. Ministry of Land, Infrastructure, Transport and Tourism, Japan, 2017. http://www.mlit.go.jp/common/ 001177505.pdf visited at 2017-05-01.

[2]

A. Abdolrahmani, R. Kuber, and A. Hurst. An empirical investiga- tion of the situationally-induced impairments experienced by blind mobile device users. In Proceedings of the 13th Web for All Con- ference, page 21. ACM, 2016.

Digital Library

[3]

Microsoft indoor localization competition - IPSN 2016. https://www.microsoft.com/en-us/research/event/microsoft-in- door-localization-competition-ipsn-2016/ visited at 2017-05-01.

[4]

New version of Google Maps brings indoor floor plans to your phone. http://edition.cnn.com/2011/11/30/tech/mobile/new-ver- sion-google-maps-indoors-wired/ visited at 2017-05-01.

[5]

SFO app for visually impaired navigation by indoo.rs, 2014. https://indoo.rs/sfo/ visited at 2017-05-01.

[6]

Wayfinder app helps the blind navigate the tube, 2014. http://www.wired.co.uk/article/wayfindr-app visited at 2017-05-01.

[7]

BlindSquare indoor navigation, May 2015. http://blindsquare.com/indoor/ visited at 2017-05-01.

[8]

D. Adams, L. Morales, and S. Kurniawan. A qualitative study to support a blind photography mobile application. In Proceedings of the 6th International Conference on PErvasive Technologies Re- lated to Assistive Environments, page 25. ACM, 2013.

Digital Library

[9]

D. Ahmetovic, C. Gleason, K. M. Kitani, H. Takagi, and C. Asakawa. Navcog: turn-by-turn smartphonenavigation assistant for people with visualimpairments or blindness. In Proceedings of the Web for All Conference, page 9. ACM, 2016.

Digital Library

[10]

D. Ahmetovic, M. Masayuki, C. Gleason, E. Brady, H. Takagi, K. Kitani and C. Asakawa. Achieving Practical and Accurate Indoor Navigation for People with Visual Impairments. In Proceedings of the Web for All Conference, page 9. ACM, 2017.

Digital Library

[11]

S. Azenkot and N. B. Lee. Exploring the use of speech input by blind people on mobile devices. In Proceedings of the 15th Inter- national ACM SIGACCESS Conference on Computers and Acces- sibility, page 11. ACM, 2013.

Digital Library

[12]

P. M. Berry, K. Myers, T. E. Uribe, and N. Yorke-Smith. Task management under change and uncertainty. In Proceedings of the Workshop on Constraint Solving under Change, 2005.

[13]

A. Black, J. E. Lovie-Kitchin, R. L. Woods, N. Arnold, J. Byrnes, and J. Murrish. Mobility performance with retinitis pigmentosa. Clinical and Experimental Optometry, 80(1):1-12, 1997.

[14]

V. Braun and V. Clarke. Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77-101, 2006.

[15]

Y.-C. Cheng, Y. Chawathe, A. LaMarca, and J. Krumm. Accuracy characterization for metropolitan-scale wi-fi localization. In Pro- ceedings of the 3rd international conference on Mobile systems, applications, and services, pages 233-245. ACM, 2005.

Digital Library

[16]

M. B. Dias, E. A. Teves, G. J. Zimmerman, H. K. Gedawy, S. M. Belousov, and M. B. Dias. Indoor navigation challenges for visu- ally impaired people. In Indoor Wayfinding and Navigation, pages 141-164. CRC Press, 2015.

[17]

N. Fallah, I. Apostolopoulos, K. Bekris, and E. Folmer. The user as a sensor: Navigating users with visual impairments in indoor spaces using tactile landmarks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '12, pages 425-432, 2012. ACM.

Digital Library

[18]

N. Fallah, I. Apostolopoulos, K. Bekris, and E. Folmer. Indoor hu- man navigation systems: A survey. Interacting with Computers, page iws010, 2013.

[19]

J. Faria, S. Lopes, H. Fernandes, P. Martins, and J. Barroso. Elec- tronic white cane for blind people navigation assistance. In World Automation Congress (WAC), 2010, pages 1-7. IEEE, 2010.

[20]

N. A. Giudice and G. E. Legge. Blind navigation and the role of technology. The Engineering Handbook of Smart Technology for Aging, Disability, and Independence, pages 479-500, 2008.

[21]

C. Gleason, D. Ahmetovic. (2017) LuzDeploy: A Collective Ac- tion System for Installing Navigation Infrastructure for Blind Peo- ple. In Proceedings of International Web for All Conference

Digital Library

[22]

S. Helal, C. Giraldo, Y. Kaddoura, C. Lee, H. El Zabadani, and W. Mann. Smart phone based cognitive assistant. In UbiHealth 2003: The 2nd International Workshop on Ubiquitous Computing for Pervasive Healthcare Applications, 2003.

[23]

A. Hub, J. Diepstraten, and T. Ertl. Design and development of an indoor navigation and object identification system for the blind. In ACM Sigaccess Accessibility and Computing, number 77-78, pages 147-152. ACM, 2004.

Digital Library

[24]

T. Ishihara, J. Vongkulbhisal, K. M. Kitani, and C. Asakawa. Bea- con-guided structure from motion for smartphone-based naviga- tion. In Winter Conference on the Applications of Computer Vi- sion. IEEE, 2017.

[25]

S. H. Jung, B.-C. Moon, and D. Han. Performance evaluation of radio map construction methods for wi-fi positioning systems. IEEE Transactions on Intelligent Transportation Systems, 2016. {26} S. Kammoun, F. Dramas, B. Oriolaand, and C. Jouffrais. Route selection algorithm for blind pedestrian. In Control Automation and Systems (ICCAS), 2010 International Conference on, pages 2223-2228. IEEE, 2010.

Digital Library

[26]

S. K. Kane, C. Jayant, J. O. Wobbrock, and R. E. Ladner. Free- dom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility, pages 115-122. ACM, 2009.

Digital Library

[27]

J.-E. Kim, M. Bessho, S. Kobayashi, N. Koshizuka, and K. Saka- mura. Navigating visually impaired travelers in a large train sta- tion using smartphone and bluetooth low energy. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, pages 604-611. ACM, 2016.

Digital Library

[28]

D. G. Luca and M. Alberto. Towards accurate indoor localization using ibeacons, fingerprinting and particle filtering. In 2016 Inter- national Conference on IndoorPositioning and Indoor Navigation (IPIN), 2016.

[29]

R. Manduchi and J. M. Coughlan. The last meter: blind visual guidance to a target. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 3113-3122. ACM, 2014.

Digital Library

[30]

R. Manduchi and J. Coughlan. (computer) vision without sight. Communications of the ACM, 55(1):96-104, 2012.

Digital Library

[31]

R. Manduchi, S. Kurniawan, and H. Bagherinia. Blind guidance using mobile computer vision: A usability study. In Proceedings of the 12th International ACM SIGACCESS Conference on Com- puters and Accessibility, ASSETS '10, pages 241-242, 2010. ACM.

Digital Library

[32]

U. Oh and L. Findlater. Design of and subjective response to on- body input for people with visual impairments. In Proceedings of the 16th international ACM SIGACCESS conference on Comput- ers & accessibility, pages 115-122. ACM, 2014.

Digital Library

[33]

J. E. Pérez, M. Arrue, M. Kobayashi, H. Takagi, and C. Asakawa. Assessment of semantic taxonomies for blindindoor navigation based on a shopping center use case. In The 14th International Cross-Disciplinary Conference on Web Accessibility, W4A 2017. ACM, 2017.

Digital Library

[34]

P.-A. Quinones, T. Greene, R. Yang, and M. Newman. Supporting visually impaired navigation: a needs-finding study. In CHI'11 Extended Abstracts on Human Factors in Computing Systems, pages 1645-1650. ACM, 2011.

Digital Library

[35]

L. Ran, S. Helal, and S. Moore. Drishti: an integrated indoor/out- door blind navigation system and service. In Pervasive Computing and Communications, 2004. PerCom 2004. Proceedings of the Second IEEE Annual Conference on, pages 23-30. IEEE, 2004.

Digital Library

[36]

D. A. Ross. Implementing assistive technology on wearable com- puters. IEEE Intelligent systems, 16(3):47-53, 2001.

Digital Library

[37]

R. Velázquez. Wearable assistive devices for the blind. In Wear- able and autonomous biomedical devices and systems for smart environment, pages 331-349. Springer, 2010.

[38]

M. A. Williams, A. Hurst, and S. K. Kane. Pray before you step out: describing personal and situational blind navigation behav- iors. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, page 28. ACM, 2013

Digital Library

[39]

J. Xiao, Z. Zhou, Y. Yi, and L. M. Ni. A survey on wireless indoor localization from the device perspective. ACM Computing Surveys (CSUR), 49(2):25, 2016.

Digital Library

[40]

H. Ye, M. Malu, U. Oh, and L. Findlater. Current and future mo- bile and wearable device use by people with visual impairments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 3123-3132. ACM, 2014.

Digital Library

Cited By

Zhang HFalletta NXie JYu RLee SBillah SCarroll J(2025)Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation SystemCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699636(29-35)Online publication date: 12-Jan-2025
https://dl.acm.org/doi/10.1145/3688828.3699636
Panwar MDhankhar ARajoria HSoreng JBatsyas RKharangarh P(2024)Review—Innovations in Flexible Sensory Devices for the Visually ImpairedECS Journal of Solid State Science and Technology10.1149/2162-8777/ad658813:7(077011)Online publication date: 30-Jul-2024
https://doi.org/10.1149/2162-8777/ad6588
Tsai CElyasi FRen PManduchi R(2024)All the Way There and Back: Inertial-Based, Phone-in-Pocket Indoor Wayfinding and Backtracking Apps for Blind TravelersACM Transactions on Accessible Computing10.1145/369600517:4(1-35)Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1145/3696005
Show More Cited By

Index Terms

NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment

Recommendations

NavCog3 in the Wild: Large-scale Blind Indoor Navigation Assistant with Semantic Features
Special Issue on ASSETS 2017 (Part 2)

NavCog3 is a smartphone turn-by-turn navigation assistant system we developed specifically designed to enable independent navigation for people with visual impairments. Using off-the-shelf Bluetooth beacons installed in the surrounding environment and a ...
NavCog: a navigational cognitive assistant for the blind
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

Turn-by-turn navigation is a useful paradigm for assisting people with visual impairments during mobility as it reduces the cognitive load of having to simultaneously sense, localize and plan. To realize such a system, it is necessary to be able to ...
The Present and Future of Museum Accessibility for People with Visual Impairments
ASSETS '18: Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

People with visual impairments (PVI) have shown interest in visiting museums and enjoying visual art. Based on this knowledge, some museums provide tactile reproductions of artworks, specialized tours for PVI, or enable them to schedule accessible ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '17: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 2017

450 pages

ISBN:9781450349260

DOI:10.1145/3132525

General Chair:
Amy Hurst
University of Maryland, Baltimore County, USA
,
Program Chairs:
Leah Findlater
University of Washington, USA
,
Meredith Ringel Morris
Microsoft Research, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASSETS '17

Sponsor:

SIGACCESS

ASSETS '17: The 19th International ACM SIGACCESS Conference on Computers and Accessibility

October 20 - November 1, 2017

Maryland, Baltimore, USA

Acceptance Rates

ASSETS '17 Paper Acceptance Rate 28 of 126 submissions, 22%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

117
Total Citations
View Citations
1,050
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)16

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HFalletta NXie JYu RLee SBillah SCarroll J(2025)Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation SystemCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699636(29-35)Online publication date: 12-Jan-2025
https://dl.acm.org/doi/10.1145/3688828.3699636
Panwar MDhankhar ARajoria HSoreng JBatsyas RKharangarh P(2024)Review—Innovations in Flexible Sensory Devices for the Visually ImpairedECS Journal of Solid State Science and Technology10.1149/2162-8777/ad658813:7(077011)Online publication date: 30-Jul-2024
https://doi.org/10.1149/2162-8777/ad6588
Tsai CElyasi FRen PManduchi R(2024)All the Way There and Back: Inertial-Based, Phone-in-Pocket Indoor Wayfinding and Backtracking Apps for Blind TravelersACM Transactions on Accessible Computing10.1145/369600517:4(1-35)Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1145/3696005
Kamikubo RKacorri HAsakawa C(2024)"We are at the mercy of others' opinion": Supporting Blind People in Recreational Window Shopping with AI-infused TechnologyProceedings of the 21st International Web for All Conference10.1145/3677846.3677860(45-58)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677860
Cassidy CBranham S(2024)Dude, Where's My Luggage? An Autoethnographic Account of Airport Navigation by a Traveler with Residual VisionProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675624(1-13)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675624
Kameswaran VRobinson JSambasivan NAggarwal GMorris M(2024)Help and The Social Construction of Access: A Case-Study from IndiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675606(1-12)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675606
Liao CJin XCheon E(2024)EasyGO: A Field Study of Grocery Store Navigation Application Design for the Visually ImpairedCompanion Publication of the 2024 ACM Designing Interactive Systems Conference10.1145/3656156.3663719(214-218)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3656156.3663719
Herskovitz J(2024)DIY Assistive Software: End-User Programming for Personalized Assistive TechnologyACM SIGACCESS Accessibility and Computing10.1145/3654768.3654772(1-1)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1145/3654768.3654772
Nair VZhu HSong PWang JSmith B(2024)Surveyor: Facilitating Discovery Within Video Games for Blind and Low Vision PlayersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642615(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642615
Guan ZXiong ZFan M(2024)FetchAid: Making Parcel Lockers More Accessible to Blind and Low Vision People With Deep-learning Enhanced Touchscreen Guidance, Error-Recovery Mechanism, and AR-based Search SupportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642213(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642213
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents