research-article

Wireless computer vision using commodity radios

Authors:

Colleen Josephson,

Sachin KattiAuthors Info & Claims

IPSN '19: Proceedings of the 18th International Conference on Information Processing in Sensor Networks

Pages 229 - 240

https://doi.org/10.1145/3302506.3310403

Published: 16 April 2019 Publication History

Abstract

We introduce the design and implementation of BackCam, a low-power wireless camera sensor platform that supports continuous realtime vision applications, all using commodity radios. In the lowest power mode, our camera board consumes only 9.7mW and continuously transmits images for over one month on two AA batteries. We introduce a novel power management system that incorporates input from the camera itself to increase battery life up to 62%. Using images and system metadata as input, we designed a feedback system between the sensor and the gateway. This allows dynamic vision application requirements to be met while consuming as little power as possible. For example, our system can temporarily increase the resolution after an object of interest is detected, then reduce it again after it has disappeared. This increases the accuracy of simplistic facial recognition by at least 25% compared to operating constantly in the lowest power mode. We implement communications using a full-duplex WiFi backscatter radio, ensuring compatibility with commodity WiFi devices. We also designed an efficient data streaming and compression pipeline straight from the camera to the backscatter transmitter, allowing us to minimize latency and avoid expensive memory writes. We deployed BackCam in a real office environment, and as a proof-of-concept, implemented basic realtime face detection and recognition.

References

[1]

2019. Blink Camera. https://blinkforhome.com/collections/blink-security-camera.

[2]

ageitgey. {n. d.}. Python Face Recognition Library. https://github.com/ageitgey/face_recognition

[3]

Dinesh Bharadia, Emily McMilin, and Sachin Katti. 2013. Full Duplex Radios. SIGCOMM Comput. Commun. Rev. 43, 4 (Aug. 2013), 375--386.

Digital Library

[4]

cjosephson. 2019. BackCam Source. https://github.com/cjosephson/backcam.

[5]

Gerald Combs. {n. d.}. tshark terminal network analyzer. https://www.wireshark.org/docs/man-pages/tshark.html

[6]

Shyamnath Gollakota, Matthew S Reynolds, Joshua R Smith, and David J Wetherall. 2014. The emergence of RF-powered computing. 47, 1 (2014), 32--39.

Digital Library

[7]

HiMax. {n. d.}. HM01B1 Image Sensor. http://www.himax.com.tw/products/cmos-image-sensor/image-sensors/hm01b0/

[8]

Pan Hu, Pengyu Zhang, and Deepak Ganesan. 2014. Leveraging interleaved signal edges for concurrent backscatter. In Proceedings of the 1st ACM workshop on Hot topics in wireless. ACM, 13--18.

Digital Library

[9]

Pan Hu, Pengyu Zhang, and Deepak Ganesan. 2015. Laissez-Faire: Fully asymmetric backscatter communication. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. ACM, 255--267.

Digital Library

[10]

Pan Hu, Pengyu Zhang, Mohammad Rostami, and Deepak Ganesan. 2016. Braidio: An integrated active-passive radio for mobile devices with asymmetric energy budgets. In Proceedings of the 2016 conference on ACM SIGCOMM 2016 Conference. ACM, 384--397.

Digital Library

[11]

Timur Ibrayev, Ulan Myrzakhan, Olga Krestinskaya, Aidana Irmanova, and Alex Pappachen James. 2018. On-chip Face Recognition System Design with Memristive Hierarchical Temporal Memory. Journal of Intelligent and Fuzzy Systems 34 (2018), 1393--1402.

[12]

Texas Instruments. 2019. TIDA-01398. http://www.ti.com/litv/pdf/tiducu5.

[13]

Vikram Iyer, Vamsi Talla, Bryce Kellogg, Shyamnath Gollakota, and Joshua Smith. 2016. Inter-technology backscatter: Towards internet connectivity for implanted devices. In Proceedings of 2016 ACM SIGCOMM Conference. ACM, 356--369.

Digital Library

[14]

Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing gesture recognition to all devices. In Usenix NSDI, Vol. 14.

Digital Library

[15]

libpcap. {n. d.}. https://github.com/the-tcpdump-group/libpcap

[16]

Robert LiKamWa, Zhen Wang, Aaron Carroll, Felix Xiaozhu Lin, and Lin Zhong. 2014. Draining Our Glass: An Energy and Heat Characterization of Google Glass. In Proceedings of 5th Asia-Pacific Workshop on Systems (APSys '14). ACM, New York, NY, USA, Article 10, 7 pages.

Digital Library

[17]

Vincent Liu, Aaron Parks, Vamsi Talla, Shyamnath Gollakota, David Wetherall, and Joshua R. Smith. 2013. Ambient Backscatter: Wireless Communication out of Thin Air. SIGCOMM Comput. Commun. Rev. 43, 4 (Aug. 2013), 39--50.

Digital Library

[18]

Vincent Liu, Vamsi Talla, and Shyamnath Gollakota. 2014. Enabling Instantaneous Feedback with Full-duplex Backscatter. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (MobiCom '14). ACM, New York, NY, USA, 67--78.

Digital Library

[19]

Tomasz Marciniak, Agata Chmielewska, Radoslaw Weychan, Marianna Parzych, and Adam Dabrowski. 2015. Influence of low resolution of images on reliability of face detection and recognition. Multimedia Tools and Applications 74, 12 (01 Jun 2015), 4329--4349.

Digital Library

[20]

Saman Naderiparizi, Mehrdad Hessar, Vamsi Talla, Shyamnath Gollakota, and Joshua R Smith. 2018. Towards Battery-Free HD Video Streaming. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA, 233--247. https://www.usenix.org/conference/nsdi18/presentation/naderiparizi

Digital Library

[21]

S. Naderiparizi, A. N. Parks, Z. Kapetanovic, B. Ransford, and J. R. Smith. 2015. WISPCam: A battery-free RFID camera. In 2015 IEEE International Conference on RFID (RFID). 166--173.

[22]

Saman Naderiparizi, Pengyu Zhang, Matthai Philipose, Bodhi Priyantha, Jie Liu, and Deepak Ganesan. 2017. Glimpse: A programmable early-discard camera architecture for continuous mobile vision. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 292--305.

Digital Library

[23]

Aaron N Parks, Angli Liu, Shyamnath Gollakota, and Joshua R Smith. 2014. Turbocharging ambient backscatter communication. In Proceedings of the 2014 ACM conference on SIGCOMM. ACM, 619--630.

Digital Library

[24]

Vamsi Talla, Mehrdad Hessar, Bryce Kellogg, Ali Najafi, Joshua R. Smith, and Shyamnath Gollakota. 2017. LoRa Backscatter: Enabling The Vision of Ubiquitous Connectivity. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 105 (Sept. 2017), 24 pages.

Digital Library

[25]

N. Thomos, N. V. Boulgouris, and M. G. Strintzis. 2006. Optimized transmission of JPEG2000 streams over wireless channels. IEEE Transactions on Image Processing 15, 1 (Jan 2006), 54--67.

Digital Library

[26]

Jue Wang, Haitham Hassanieh, Dina Katabi, and Piotr Indyk. 2012. Efficient and reliable low-power backscatter networks. In Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication. ACM, 61--72.

Digital Library

[27]

Xieyang Xu, Yang Shen, Junrui Yang, Chenren Xu, Guobin Shen, Guojun Chen, and Yunzhe Ni. 2017. PassiveVLC: Enabling Practical Visible Light Backscatter Communication for Battery-free IoT Applications. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (MobiCom '17). ACM, 180--192.

Digital Library

[28]

C. Yang, J. Gummeson, and A. Sample. 2017. Riding the airways: Ultra-wideband ambient backscatter via commercial broadcast systems. In IEEE INFOCOM 2017 - IEEE Conference on Computer Communications. 1--9.

[29]

zeromq. {n. d.}. ZeroMQ. https://github.com/zeromq/libzmq

[30]

PENGYU ZHANG, Dinesh Bharadia, Kiran Joshi, and Sachin Katti. 2016. Enabling Backscatter Communication Among Commodity WiFi Radios. In Proceedings of the 2016 Conference on ACM SIGCOMM 2016 Conference (SIGCOMM '16). ACM, 611--612.

Digital Library

[31]

Pengyu Zhang, Dinesh Bharadia, Kiran Joshi, and Sachin Katti. 2016. HitchHike: Practical Backscatter Using Commodity WiFi. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM (SenSys '16). ACM, 259--271.

Digital Library

[32]

Pengyu Zhang, Colleen Josephson, Dinesh Bharadia, and Sachin Katti. 2017. FreeRider: Backscatter Communication Using Commodity Radios. In Proceedings of the 13th International Conference on Emerging Networking Experiments and Technologies. ACM, 389--401.

Digital Library

[33]

Lin Zhong. 2011. Power Consumption By Wireless Communication. http://www.ruf.rice.edu/~mobile/elec518/lectures/3-wireless.pdf

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/367207623:6(1-26)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3672076
Nayar SKlotz JNanda NFridberg M(2024)Cricket: A Self-Powered Chirping PixelACM Transactions on Graphics10.1145/365819643:4(1-11)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658196
Naeem NRademacher JPatnaik RBoroushaki TAdib FGanesan DLane NShi W(2024)SeaScan: An Energy-Efficient Underwater Camera for Wireless 3D Color ImagingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690661(785-799)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3690661
Show More Cited By

Index Terms

Wireless computer vision using commodity radios
1. Networks
  1. Network types
    1. Cyber-physical networks
      1. Sensor networks

Recommendations

EkhoNet: high speed ultra low-power backscatter for next generation sensors
MobiCom '14: Proceedings of the 20th annual international conference on Mobile computing and networking

This paper argues for a clean-slate redesign of wireless sensor systems to take advantage of the extremely low power consumption of backscatter communication and emerging ultra-low power sensor modalities. We make the case that existing sensing ...
Simple, Accurate, and Robust Projector-Camera Calibration
3DIMPVT '12: Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission

Structured-light systems are simple and effective tools to acquire 3D models. Built with off-the-shelf components, a data projector and a camera, they are easy to deploy and compare in precision with expensive laser scanners. But such a high precision ...
Optimization of Power Allocation in Multimedia Wireless Sensor Networks

Wireless Sensor Networks WSN have been widely applied in monitoring and surveillance fields in recent years and have dramatically changed the methodologies and technologies in monitoring and surveillance. However, the sensor nodes in WSN have very ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IPSN '19: Proceedings of the 18th International Conference on Information Processing in Sensor Networks

April 2019

365 pages

ISBN:9781450362849

DOI:10.1145/3302506

General Chair:
Rasit Eskicioglu
University of Manitoba, Canada
,
Program Chairs:
Luca Mottola
Politecnico di Milano, Italy
,
Bodhi Priyantha
Microsoft Research

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGBED: ACM Special Interest Group on Embedded Systems

In-Cooperation

IEEE-SPS: Signal Processing Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IPSN '19

Sponsor:

SIGBED

IPSN '19: The 18th International Conference on Information Processing in Sensor Networks

April 16 - 18, 2019

Quebec, Montreal, Canada

Acceptance Rates

IPSN '19 Paper Acceptance Rate 25 of 91 submissions, 27%;

Overall Acceptance Rate 143 of 593 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
446
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/367207623:6(1-26)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3672076
Nayar SKlotz JNanda NFridberg M(2024)Cricket: A Self-Powered Chirping PixelACM Transactions on Graphics10.1145/365819643:4(1-11)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658196
Naeem NRademacher JPatnaik RBoroushaki TAdib FGanesan DLane NShi W(2024)SeaScan: An Energy-Efficient Underwater Camera for Wireless 3D Color ImagingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690661(785-799)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3690661
Le DZhou SYang JChen JHo DTan R(2024)Artificial Intelligence of Things for industrial Visual sensing systems in HP's factoriesDigital Manufacturing10.1016/B978-0-443-13812-6.00003-8(133-168)Online publication date: 2024
https://doi.org/10.1016/B978-0-443-13812-6.00003-8
Kodukula VManetta MLiKamWa RCosta XAl Hassanieh HAsadi ACox LPerino DWidmer JGiustiniano D(2023)Squint: A Framework for Dynamic Voltage Scaling of Image Sensors Towards Low Power IoT VisionProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613303(1-15)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3570361.3613303
Veluri BPernu CSaffari ASmith JTaylor MGollakota SCosta XAl Hassanieh HAsadi ACox LPerino DWidmer JGiustiniano D(2023)NeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT CamerasProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3592523(1-17)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3570361.3592523
Abbood IIdrees A(2023)Data reduction techniques for wireless multimedia sensor networks: a systematic literature reviewThe Journal of Supercomputing10.1007/s11227-023-05842-880:7(10044-10089)Online publication date: 19-Dec-2023
https://doi.org/10.1007/s11227-023-05842-8
Zhou SLe DTan RYang JHo D(2022)Configuration-Adaptive Wireless Visual Sensing System with Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2022.3175182(1-1)Online publication date: 2022
https://doi.org/10.1109/TMC.2022.3175182
Katanbaf MSaffari ASmith J(2021)MultiScatterProceedings of the 19th ACM Conference on Embedded Networked Sensor Systems10.1145/3485730.3485939(69-83)Online publication date: 15-Nov-2021
https://dl.acm.org/doi/10.1145/3485730.3485939
Josephson CKotaru MWinstein KKatti SChandra R(2021)Low-cost In-ground Soil Moisture Sensing with Radar Backscatter TagsProceedings of the 4th ACM SIGCAS Conference on Computing and Sustainable Societies10.1145/3460112.3472326(299-311)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3460112.3472326
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents