short-paper

PhoneEar: interactions for mobile devices that hear high-frequency sound-encoded data

Authors:

Aditya Shekhar Nittala,

Xing-Dong Yang,

Saul GreenbergAuthors Info & Claims

EICS '15: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems

Pages 174 - 179

https://doi.org/10.1145/2774225.2775082

Published: 23 June 2015 Publication History

Abstract

We present PhoneEar, a new approach that enables mobile devices to understand the broadcasted audio and sounds that we hear every day using existing infrastructure. PhoneEar audio streams are embedded with sound-encoded data using nearly inaudible high frequencies. Mobile devices then listen for messages in the sounds around us, taking actions to ensure we don't miss any important info. In this paper, we detail our implementation of PhoneEar, describe a study demonstrating that mobile devices can effectively receive sound-based data, and describe the results of a user study that shows that embedding data in sounds is not detrimental to sound quality. We also exemplify the space of new interactions, through four PhoneEar-enabled applications. Finally, we discuss the challenges to deploying apps that can hear and react to data in the sounds around us.

Supplementary Material

M4V File (p174-nittala.m4v)

Download
33.53 MB

References

[1]

Buxton, W. Integrating the Periphery and Context: A New Model of Telematics. Proc. of GI, 1995, 239--246.

[2]

Rodden, Tom, Keith Cheverst, K. Davies, and Alan Dix. "Exploiting context in HCI design for mobile systems." In Workshop on human computer interaction with mobile devices, pp. 21--22. 1998.

[3]

Deshotels, L. Inaudible sound as a covert channel in mobile devices. Proc. USENIX conference on Offensive Technologies. 2014. pp 16--16.

Digital Library

[4]

Gerasimov, V. & Bender, W. Things that talk: using sound for device-to-device and device-to-human communication. IBM Syst. J., 39 (3-4), 530--546, 2000.

Digital Library

[5]

Google voice search. http://www.google.com/mobile/voicesearch/.

[6]

Hanspach, Mi., & Goetz, M. On covert acoustical mesh networks in air. 2014. arXiv preprint arXiv:1406.1213

[7]

Harrison, C. & Hudson, S. E. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. UIST'08, pp 205--208. ACM, 2008.

Digital Library

[8]

Ju, Wendy, Brian A. Lee, and Scott R. Klemmer. "Range: exploring implicit interaction through electronic whiteboard design." In Proceedings of the 2008 ACM conference on Computer supported cooperative work, pp. 17--26. ACM, 2008.

Digital Library

[9]

Lopes, C. V. and Aquiar, P. M. Q. Acoustic modems for ubiquitous computing. IEEE Pervasive Computing, Mobile and Ubiquitous Systems, 2(3), 62--71. 2003.

Digital Library

[10]

Lu, Hong, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury, and Andrew T. Campbell. "SoundSense: scalable sound sensing for people-centric applications on mobile phones." In Proceedings of the 7th international conference on Mobile systems, applications, and services, pp. 165--178. ACM, 2009.

Digital Library

[11]

Madhavapeddy, A., Scott, D. and Sharp, R. Context-Aware Computing with Sound. UbiComp'03. 315--332.

[12]

Moore, B. An Introduction to the Psychology of Hearing. Academic Press. 1997.

[13]

Nandakumar, Rajalakshmi, Krishna Kant Chintalapudi, Venkat Padmanabhan, and Ramarathnam Venkatesan. "Dhwani: secure peer-to-peer acoustic NFC." In ACM SIGCOMM Computer Communication Review, vol. 43, no. 4, pp. 63--74. ACM, 2013.

Digital Library

[14]

Rico, J. and Brewster, S. Usable gestures for mobile interfaces: evaluating social acceptability. Proc. CH'10. pp 887--896. ACM, 2010.

Digital Library

[15]

Shazam: http://www.shazam.com/.

[16]

Wang, A. An Industrial-Strength Audio Search Algorithm. Proc. ISMIR. 2003. 7--13.

[17]

Yang, Xing-Dong, Khalad Hasan, Neil Bruce, and Pourang Irani. "Surround-see: enabling peripheral vision on smartphones during active use." In Proceedings of the 26th annual ACM symposium on User interface software and technology, pp. 291--300. ACM, 2013.

Digital Library

Cited By

Lee JYang K(2022)Mobile Device-Based Struck-By Hazard Recognition in Construction Using a High-Frequency SoundSensors10.3390/s2209348222:9(3482)Online publication date: 3-May-2022
https://doi.org/10.3390/s22093482
Xu YDang FXu RChen XLiu Y(2022)LSync: A Universal Event-synchronizing Solution for Live StreamingIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796933(2188-2197)Online publication date: 2-May-2022
https://doi.org/10.1109/INFOCOM48880.2022.9796933
Wang YZhu XHan H(2021)ChirpMu: Chirp Based Imperceptible Information Broadcasting with Music2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS)10.1109/IWQOS52092.2021.9521353(1-9)Online publication date: 25-Jun-2021
https://doi.org/10.1109/IWQOS52092.2021.9521353
Show More Cited By

Index Terms

PhoneEar: interactions for mobile devices that hear high-frequency sound-encoded data
1. Human-centered computing

Recommendations

The effects of encumbrance on mobile interactions
MobileHCI '14: Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services

The popularity of touchscreen mobile devices gives users a variety of useful apps and functionality on the move. As a result, mobile devices are being used in a range of different contexts. One common scenario that has received little attention from ...
Augmenting spatial skills with mobile devices
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Mobile devices are increasingly providing novel ways for users to engage with the spaces around them. However, there are few systematic studies of enhancing spatial ability with mobile devices, and applications such as turn-by-turn navigation systems ...
Simplifying the making of probes, prototypes and toolkits in mobile interaction research using tasker
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

This paper presents a technique to support the making of mobile interaction interfaces for controlling the smartphone. We often use smartphones while moving, resulting in non-optimal or even unsafe interactions. Better mobile interactions need to be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EICS '15: Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems

June 2015

316 pages

ISBN:9781450336468

DOI:10.1145/2774225

General Chair:
Jürgen Ziegler
University of Duisburg-Essen, Germany
,
Program Chairs:
Michael Nebeling
Carnegie Melon University, USA
,
Laurence Nigay
Université Grenoble Alpes, France

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 June 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

EICS'15

Sponsor:

SIGCHI

EICS'15: ACM SIGCHI Symposium on Engineering Interactive Computing Systems

June 23 - 26, 2015

Duisburg, Germany

Acceptance Rates

EICS '15 Paper Acceptance Rate 19 of 64 submissions, 30%;

Overall Acceptance Rate 73 of 299 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
270
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)4

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee JYang K(2022)Mobile Device-Based Struck-By Hazard Recognition in Construction Using a High-Frequency SoundSensors10.3390/s2209348222:9(3482)Online publication date: 3-May-2022
https://doi.org/10.3390/s22093482
Xu YDang FXu RChen XLiu Y(2022)LSync: A Universal Event-synchronizing Solution for Live StreamingIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796933(2188-2197)Online publication date: 2-May-2022
https://doi.org/10.1109/INFOCOM48880.2022.9796933
Wang YZhu XHan H(2021)ChirpMu: Chirp Based Imperceptible Information Broadcasting with Music2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS)10.1109/IWQOS52092.2021.9521353(1-9)Online publication date: 25-Jun-2021
https://doi.org/10.1109/IWQOS52092.2021.9521353
Zhu XWang YQin Z(2021)RoV: receiving files from voice calls using dual-tone multi-frequency methodThe Journal of Supercomputing10.1007/s11227-021-04168-778:5(7304-7320)Online publication date: 8-Nov-2021
https://doi.org/10.1007/s11227-021-04168-7
Tan DLu YYan XLi L(2021)BAT: real-time inaudible sound capture with smartphonesMultimedia Tools and Applications10.1007/s11042-021-11372-3Online publication date: 17-Aug-2021
https://doi.org/10.1007/s11042-021-11372-3
Herrero R(2021) Repurposing acoustic modulation schemes to support ultrasonic IoT NFC Internet Technology Letters10.1002/itl2.3184:6Online publication date: 30-Aug-2021
https://doi.org/10.1002/itl2.318
Xiao RMayer SHarrison C(2020)VibroComm: Using Commodity Gyroscopes for Vibroacoustic Data Reception22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403540(1-9)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403540
Won KYeoum SKang BKim MShin YChoo H(2020)Inaudible Transmission System with Selective Dual Frequencies Robust to Noisy Surroundings2020 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE46568.2020.9042989(1-6)Online publication date: Jan-2020
https://doi.org/10.1109/ICCE46568.2020.9042989
Satapathy UChatterjee SChakraborty S(2020)SilentInformer: A Framework for Information Dispersion Using Inaudible Acoustic Signals2020 International Conference on COMmunication Systems & NETworkS (COMSNETS)10.1109/COMSNETS48256.2020.9027493(543-546)Online publication date: Jan-2020
https://doi.org/10.1109/COMSNETS48256.2020.9027493
Lee JKwak CKim SBahk S(2020)Reliable and Low-Complexity Chirp Spread Spectrum-Based Aerial Acoustic CommunicationIEEE Access10.1109/ACCESS.2020.30170978(151589-151601)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3017097
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents