research-article

Human Sensing Using Visible Light Communication

Authors:

Andrew T. Campbell,

Xia ZhouAuthors Info & Claims

MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking

Pages 331 - 344

https://doi.org/10.1145/2789168.2790110

Published: 07 September 2015 Publication History

Abstract

We present LiSense, the first-of-its-kind system that enables both data communication and fine-grained, real-time human skeleton reconstruction using Visible Light Communication (VLC). LiSense uses shadows created by the human body from blocked light and reconstructs 3D human skeleton postures in real time. We overcome two key challenges to realize shadow-based human sensing. First, multiple lights on the ceiling lead to diminished and complex shadow patterns on the floor. We design light beacons enabled by VLC to separate light rays from different light sources and recover the shadow pattern cast by each individual light. Second, we design an efficient inference algorithm to reconstruct user postures using 2D shadow information with a limited resolution collected by photodiodes embedded in the floor. We build a 3 m x 3 m LiSense testbed using off-the-shelf LEDs and photodiodes. Experiments show that LiSense reconstructs the 3D user skeleton at 60 Hz in real time with 10 degrees mean angular error for five body joints.

References

[1]

Google Project Jacquard. https://www.google.com/atap/project-jacquard/.

[2]

Adib, F., Kabelac, Z., and Katabi, D. Multi-Person Motion Tracking via RF Body Reflections. In Proc. of NSDI (2015).

Digital Library

[3]

Adib, F., Kabelac, Z., Katabi, D., and Miller, R. C. 3D Tracking via Body Radio Reflections. In Proc. of NSDI (2014).

Digital Library

[4]

Adib, F., and Katabi, D. See through walls with WiFi! In Proc. of SIGCOMM (2013).

Digital Library

[5]

Afgani, M. Z., Haas, H., Elgala, H., and Knipp, D. Visible light communication using OFDM. In Proc. of TRIDENTCOM (2006).

[6]

Arai, S., et al. Experimental on hierarchical transmission scheme for visible light communication using LED traffic light and high-speed camera. In Proc. of VTC (2007).

[7]

Ashok, A., et al. Challenge: Mobile optical networks through visual MIMO. In Proc. of MobiCom (2010).

Digital Library

[8]

Association, I. S., et al. IEEE Std. for Local and metropolitan area networks-Part 15.7: Short-Rang Wireless Optical Communication Using Visible Light. IEEE Computer Society (2011).

[9]

Beaudoin, P., Poulin, P., and van de Panne, M. Adapting wavelet compression to human motion capture clips. In Proc. of GI (2007).

Digital Library

[10]

Chen, V. C., Li, F., Ho, S.-S., and Wechsler, H. Analysis of micro-doppler signatures. IEE Proceedings-Radar, Sonar and Navigation 150, 4 (2003), 271--276.

[11]

Cheung, G. K., Kanade, T., Bouguet, J.-Y., and Holler, M. A real time system for robust 3D voxel reconstruction of human motions. In Proc. of CVPR (2000).

[12]

Dietz, P., Yerazunis, W., and Leigh, D. Very low-cost sensing and communication using bidirectional LEDs. In Proc. of UbiComp (2003).

[13]

Duhamel, P., and Hollmann, H. Split radix FFT algorithm. Electronics letters 20, 1 (1984), 14--16.

[14]

Elgala, H., Mesleh, R., Haas, H., and Pricope, B. OFDM visible light wireless communication based on white LEDs. In Proc. of VTC (2007).

[15]

Fernandez-Baena, A., Susın, A., and Lligadas, X. Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In Proc. of INCoS (2012).

Digital Library

[16]

Ghaddar, M., Talbi, L., and Denidni, T. Human body modelling for prediction of effect of people on indoor propagation channel. Electronics Letters 40, 25 (2004), 1592--1594.

[17]

Gupta, S., Morris, D., Patel, S., and Tan, D. SoundWave: Using the Doppler effect to sense gestures. In Proc. of CHI (2012).

Digital Library

[18]

Hao, T., Zhou, R., and Xing, G. COBRA: Color barcode streaming for smartphone systems. In Proc. of MobiSys (2012).

Digital Library

[19]

Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proc. of ISER (2010).

[20]

Herda, L., Fua, P., Plankers, R., Boulic, R., and Thalmann, D. Skeleton-based motion capture for robust reconstruction of human motion. In Proc. of International Conference on Computer Animation (2000).

Digital Library

[21]

Howe, N. R., Leventon, M. E., and Freeman, W. T. Bayesian Reconstruction of 3D Human Motion from Single-Camera Video. In Proc. of NIPS (1999).

[22]

Hu, W., Gu, H., and Pu, Q. LightSync: Unsynchronized visual communication over screen-camera links. In Proc. of MobiCom (2013).

Digital Library

[23]

Hu, W., Mao, J., Huang, Z., Xue, Y., She, J., Bian, K., and Shen, G. Strata: Layered Coding for Scalable Visual Communication. In Proc. of MobiCom (2014).

Digital Library

[24]

Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. of UIST (2011).

Digital Library

[25]

Jerri, A. J. The Shannon sampling theorem--Its various extensions and applications: A tutorial review. Proc. of the IEEE (1977).

[26]

Jhuang, H., Gall, J., Zuffi, S., Schmid, C., and Black, M. J. Towards understanding action recognition. In Proc. of ICCV (2013).

Digital Library

[27]

Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P. Digits: Freehand 3D Interactions Anywhere Using a Wrist-worn Gloveless Sensor. In Proc. of UIST (2012).

Digital Library

[28]

Kim, H.-S., et al. An indoor visible light communication positioning system using a RF carrier allocation technique. Journal of Lightwave Technology 31, 1 (2013), 134--144.

[29]

Komine, T., and Nakagawa, M. Integrated system of white LED visible-light communication and power-line communication. IEEE Transactions on Consumer Electronics 49, 1 (2003), 71--79.

Digital Library

[30]

Komine, T., and Nakagawa, M. Fundamental analysis for visible-light communication system using LED lights. IEEE Transactions on Consumer Electronics 50, 1 (2004), 100--107.

Digital Library

[31]

Krosshaug, T., and Bahr, R. A model-based image-matching technique for three-dimensional reconstruction of human motion from uncalibrated video sequences. Journal of biomechanics 38, 4 (2005), 919--929.

[32]

Kuo, Y.-S., Pannuto, P., Hsiao, K.-J., and Dutta, P. Luxapose: Indoor positioning with mobile phones and visible light. In Proc. of MobiCom (2014).

Digital Library

[33]

Le-Minh, H., et al. 100-Mb/s NRZ visible light communications using a postequalized white LED. Photonics Technology Letters, IEEE 21, 15 (2009), 1063--1065.

[34]

Le Minh, H., O'Brien, D., Faulkner, G., Zeng, L., Lee, K., Jung, D., Oh, Y., and Won, E. T. 100-Mb/s NRZ visible light communications using a postequalized white LED. Photonics Technology Letters, IEEE 21, 15 (2009), 1063--1065.

[35]

Lee, K., and Park, H. Modulations for visible light communications with dimming control. Photonics Technology Letters, IEEE 23, 16 (2011), 1136--1138.

[36]

Li, L., Hu, P., Peng, C., Shen, G., and Zhao, F. Epsilon: A visible light based positioning system. In Proc. of NSDI (2014).

Digital Library

[37]

Li, T., An, C., Xiao, X., Campbell, A. T., and Zhou, X. Real-Time Screen-Camera Communication Behind Any Scene. In Proc. of MobiSys (2015).

Digital Library

[38]

Little, T. D. C., et al. Using LED lighting for ubiquitous indoor wireless networking. In Proc. of WiMob (2008).

Digital Library

[39]

Liu, C. B., Sadeghi, B., and Knightly, E. W. Enabling vehicular visible light communication (V2LC) networks. In Proc. of VANET (2011).

Digital Library

[40]

McKinney, J. C., and Hopkins, C. D. R. ATSC digital television standard. Advanced Television System Committee (1995).

[41]

Moeslund, T. B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Computer vision and image understanding 104, 2 (2006), 90--126.

Digital Library

[42]

Mohr, R., Quan, L., and Veillon, F. Relative 3D reconstruction using multiple uncalibrated images. The International Journal of Robotics Research 14, 6 (1995), 619--632.

Digital Library

[43]

Moschini, D., and Fusiello, A. Tracking human motion with multiple cameras using an articulated model. In Computer Vision/Computer Graphics Collaboration Techniques. Springer, 2009, pp. 1--12.

Digital Library

[44]

Perli, S. D., Ahmed, N., and Katabi, D. PixNet: Interference-free wireless links using LCD-camera pairs. In Proc. of MobiCom (2010).

Digital Library

[45]

Post, E. R., Orth, M., Russo, P. R., and Gershenfeld, N. E-broidery: Design and fabrication of textile-based computing. IBM Systems Journal 39, 3.4 (2000), 840--860.

Digital Library

[46]

Pu, Q., Gupta, S., Gollakota, S., and Patel, S. Whole-home gesture recognition using wireless signals. In Proc. of MobiCom (2013).

Digital Library

[47]

Quintana, C., et al. Reading lamp-based visible light communication system for in-flight entertainment. Consumer Electronics, IEEE Transactions on 59, 1 (2013), 31--37.

[48]

Rajagopal, N., Lazik, P., and Rowe, A. Visual light landmarks for mobile devices. In Proc. of IPSN (2014).

Digital Library

[49]

Rajagopal, S., Roberts, R., and Lim, S.-K. IEEE 802.15.7 visible light communication: modulation schemes and dimming support. Communications Magazine, IEEE 50, 3 (2012), 72--82.

[50]

Ryckaert, J., De Doncker, P., Meys, R., de Le Hoye, A., and Donnay, S. Channel model for wireless communication around human body. Electronics Letters 40, 9 (2004), 543--544.

[51]

Saxena, A., Chung, S. H., and Ng, A. Y. 3-D depth reconstruction from a single still image. International journal of computer vision 76, 1 (2008), 53--69.

Digital Library

[52]

Sb\^ırlea, D., Burke, M. G., Guarnieri, S., Pistoia, M., and Sarkar, V. Automatic detection of inter-application permission leaks in android applications. IBM Journal of Research and Development 57, 6 (2013), 10--1.

Digital Library

[53]

Schill, F., Zimmer, U. R., and Trumpf, J. Visible spectrum optical communication and distance sensing for underwater applications. In In Proc. of Australasian Conference on Robotics and Automation (2004).

[54]

Schmid, S., Corbellini, G., Mangold, S., and Gross, T. R. LED-to-LED visible light communication networks. In Proc. of MobiHoc (2013).

Digital Library

[55]

Segen, J., and Kumar, S. Shadow gestures: 3D hand pose estimation using a single camera. In Proc. of CVPR (1999).

[56]

Shoemaker, G., Tang, A., and Booth, K. S. Shadow reaching: a new perspective on interaction for large displays. In Proc. of UIST (2007).

Digital Library

[57]

Shotton, J., et al. Real-time human pose recognition in parts from single depth images. In Proc. of CVPR (2011).

Digital Library

[58]

Silaghi, M.-C., Plankers, R., Boulic, R., Fua, P., and Thalmann, D. Local and global skeleton fitting techniques for optical motion capture. In Modelling and Motion Capture Techniques for Virtual Environments. Springer, 1998, pp. 26--40.

[59]

Tsonev, D., et al. A 3-Gb/s Single-LED OFDM-based Wireless VLC Link Using a Gallium Nitride μLED. Photonics Technology Letters, IEEE PP, 99 (2014), 1--1.

[60]

Wall, M. 'Li-fi' via LED light bulb data speed breakthrough. BBC News, 2013.

[61]

Wang, A., Ma, S., Hu, C., Huai, J., Peng, C., and Shen, G. Enhancing Reliability to Boost the Throughput over Screen-camera Links. In Proc. of MobiCom (2014).

Digital Library

[62]

Wang, J., Vasisht, D., and Katabi, D. RF-IDraw: Virtual Touch Screen in the Air Using RF Signals. In Proc. of SIGCOMM (2014).

Digital Library

[63]

Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., and Liu, H. E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures. In Proc. of MobiCom (2014).

Digital Library

[64]

Weinberg, Z., Chen, E. Y., Jayaraman, P. R., and Jackson, C. I still know what you visited last summer: Leaking browsing history via user interaction and side channel attacks. In Proc. of IEEE Symposium on Security and Privacy (2011).

Digital Library

[65]

Welch, T., Musselman, R., Emessiene, B., Gift, P., Choudhury, D., Cassadine, D., and Yano, S. The effects of the human body on UWB signal propagation in an indoor environment. IEEE Journal on Selected Areas in Communications 20, 9 (2002), 1778--1782.

Digital Library

[66]

Whitaker, R. T. A level-set approach to 3D reconstruction from range data. International journal of computer vision 29, 3 (1998), 203--231.

Digital Library

[67]

Yeung, K.-Y., Kwok, T.-H., and Wang, C. C. Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications. Journal of Computing and Information Science in Engineering 13, 4 (2013), 041007.

[68]

Zeng, L., et al. High data rate multiple input multiple output (MIMO) optical wireless communications using white LED lighting. IEEE Journal on Selected Areas in Communications 27, 9 (2009), 1654--1662.

Digital Library

[69]

Zhang, W., and Kavehrad, M. Comparison of VLC-based indoor positioning techniques. In Proc. of SPIE (2013).

[70]

Zhang, Z., et al. I Am the Antenna: Accurate Outdoor AP Location Using Smartphones. In Proc. of MobiCom (2011).

Digital Library

[71]

Zhou, X., and Campbell, A. Visible Light Networking and Sensing. In Proc. of HotWireless (2014).

Digital Library

Cited By

Nie MZou LCui HZhou XWan Y(2024)Enhancing Human Activity Recognition with LoRa Wireless RF Signal Preprocessing and Deep LearningElectronics10.3390/electronics1302026413:2(264)Online publication date: 6-Jan-2024
https://doi.org/10.3390/electronics13020264
Xiao RZhao LQian FYang LHan JOkoshi TKo JLiKamWa R(2024)Practical Optical Camera Communication Behind Unseen and Complex BackgroundsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661866(113-126)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661866
Zhang XMariani JXiao LMutka M(2024)Exploiting Fine-grained Dimming with Improved LiFi ThroughputACM Transactions on Sensor Networks10.1145/364381420:3(1-24)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3643814
Show More Cited By

Index Terms

Human Sensing Using Visible Light Communication
1. Networks
  1. Network types
    1. Mobile networks
    2. Wireless access networks

Recommendations

Practical Human Sensing in the Light
MobiSys '16: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services

We present StarLight, an infrastructure-based sensing system that reuses light emitted from ceiling LED panels to reconstruct fine-grained user skeleton postures continuously in real time. It relies on only a few (e.g., 20) photodiodes placed at ...
LED-to-LED visible light communication networks
MobiHoc '13: Proceedings of the fourteenth ACM international symposium on Mobile ad hoc networking and computing

Visible Light Communication (VLC) with Light Emitting Diodes (LEDs) as transmitters and receivers enables low bitrate wireless adhoc networking. LED-to-LED VLC adhoc networks with VLC devices communicating with each other over free-space optical links ...
Using consumer LED light bulbs for low-cost visible light communication systems
VLCS '14: Proceedings of the 1st ACM MobiCom workshop on Visible light communication systems

LED-to-LED Visible Light Communication (VLC) based on Light Emitting Diodes (LEDs) and microcontrollers provide a foundation for networking using visible light as communication medium. We describe a low-complexity smart LED light bulb prototype that is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and Networking

September 2015

638 pages

ISBN:9781450336192

DOI:10.1145/2789168

General Chairs:
Serge Fdida
UPMC/LIP6, France
,
Giovanni Pau
UPMC/LIP6, France
,
Program Chairs:
Sneha Kumar Kasera
University of Utah, USA
,
Heather Zheng
University of California, Santa Barbara, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

MobiCom'15

Sponsor:

SIGMOBILE

MobiCom'15: The 21th Annual International Conference on Mobile Computing and Networking

September 7 - 11, 2015

Paris, France

Acceptance Rates

MobiCom '15 Paper Acceptance Rate 38 of 207 submissions, 18%;

Overall Acceptance Rate 440 of 2,972 submissions, 15%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

174
Total Citations
View Citations
1,586
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nie MZou LCui HZhou XWan Y(2024)Enhancing Human Activity Recognition with LoRa Wireless RF Signal Preprocessing and Deep LearningElectronics10.3390/electronics1302026413:2(264)Online publication date: 6-Jan-2024
https://doi.org/10.3390/electronics13020264
Xiao RZhao LQian FYang LHan JOkoshi TKo JLiKamWa R(2024)Practical Optical Camera Communication Behind Unseen and Complex BackgroundsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661866(113-126)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661866
Zhang XMariani JXiao LMutka M(2024)Exploiting Fine-grained Dimming with Improved LiFi ThroughputACM Transactions on Sensor Networks10.1145/364381420:3(1-24)Online publication date: 13-Apr-2024
https://dl.acm.org/doi/10.1145/3643814
Zeng HLi GLi T(2024)PyroSenseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314357:4(1-32)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631435
Liao ZLuo ZHuang QZhang LWu FZhang QChen G(2024)Gesture Recognition Using Visible Light on Mobile DevicesIEEE/ACM Transactions on Networking10.1109/TNET.2024.336999632:4(2920-2935)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3369996
Thaddeus LReddy CBhadauria YShah DGulati MVarshney A(2024)TunnelSense: Low-Power, Non-Contact Sensing Using Tunnel Diodes2024 IEEE International Conference on RFID (RFID)10.1109/RFID62091.2024.10582671(154-159)Online publication date: 4-Jun-2024
https://doi.org/10.1109/RFID62091.2024.10582671
Ahmad IUllah AChoi W(2024)WiFi-Based Human Sensing With Deep Learning: Recent Advances, Challenges, and OpportunitiesIEEE Open Journal of the Communications Society10.1109/OJCOMS.2024.34115295(3595-3623)Online publication date: 2024
https://doi.org/10.1109/OJCOMS.2024.3411529
Caputo SJayousi SBorghini GHämäläinen MPaso THernandez MTanaka HAnzai DIto TMucchi L(2024)ETSI SmartBAN Vision on the use of Visible Light Communications in Body Area Networks2024 18th International Symposium on Medical Information and Communication Technology (ISMICT)10.1109/ISMICT61996.2024.10738176(16-21)Online publication date: 15-May-2024
https://doi.org/10.1109/ISMICT61996.2024.10738176
Liu YJiang JWang J(2024)LoMu: Enable Long-Range Multi-Target Backscatter Sensing for Low-Cost TagsIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621272(2069-2078)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621272
Wei ZHuang NLiu WXu Z(2024)A PCA-Based Material Color Classification Method via LED Sensing2024 IEEE 12th International Conference on Information, Communication and Networks (ICICN)10.1109/ICICN62625.2024.10761343(288-292)Online publication date: 21-Aug-2024
https://doi.org/10.1109/ICICN62625.2024.10761343
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

EPUB

View this article in ePub.

Media

Figures

Other

Tables

View Table of Contents