skip to main content
research-article

Energy-efficient Collaborative Sensing: Learning the Latent Correlations of Heterogeneous Sensors

Published: 21 June 2021 Publication History

Abstract

With the proliferation of Internet of Things (IoT) devices in the consumer market, the unprecedented sensing capability of IoT devices makes it possible to develop advanced sensing and complex inference tasks by leveraging heterogeneous sensors embedded in IoT devices. However, the limited power supply and the restricted computation capability make it challenging to conduct seamless sensing and continuous inference tasks on resource-constrained devices. How to conduct energy-efficient sensing and perform rich-sensor inference tasks on IoT devices is crucial for the success of IoT applications. Therefore, we propose a novel energy-efficient collaborative sensing framework to optimize the energy consumption of IoT devices. Specifically, we explore the latent correlations among heterogeneous sensors via an attention mechanism in temporal convolutional network to quantify the dependency among sensors, and characterize the heterogeneous sensors in terms of energy consumption to categorize them into low-power sensors and energy-intensive sensors. Finally, to decrease the sampling frequency of energy-intensive sensors, we propose a multi-task learning strategy to predict the statuses of energy-intensive sensors based on the low-power sensors. To evaluate the performance of the proposed collaborative sensing framework, we develop a mobile application to collect concurrent heterogeneous data streams from all sensors embedded in Huawei Mate 8. The experimental results show that latent correlation learning is greatly helpful to understand the latent correlations among heterogeneous streams, and it is feasible to predict the statuses of energy-intensive sensors by low-power sensors with high accuracy and fast convergence. In terms of energy consumption, the proposed collaborative sensing framework is able to preserve the energy consumption of IoT devices by nearly 50% for continuous data acquisition tasks.

References

[1]
Muhammad Aftab, Sid Chi-Kin Chau, and Prashant Shenoy. 2020. Efficient online classification and tracking on resource-constrained iot devices. ACM Trans. Internet Things 1, 3, Article 20 (July 2020), 29 pages.
[2]
A. Alsharoa, H. Ghazzai, A. Kadri, and A. E. Kamal. 2020. Spatial and temporal management of cellular HetNets with multiple solar powered drones. IEEE Trans. Mobile Comput. 19, 4 (2020), 954–968.
[3]
Tanapat Anusas-amornkul. 2019. Strengthening password authentication using keystroke dynamics and smartphone sensors. In Proceedings of the 9th International Conference on Information Communication and Management. Association for Computing Machinery, New York, NY, 70–74.
[4]
A. H. Azni, Abdul Fuad Abdul Rahman, Najwa Hayaati Mohd Alwi, and Kamaruzzaman Seman. 2017. Measuring sensor to cloud energy consumption. In Proceedings of the Second International Conference on Internet of Things, Data and Cloud Computing (ICC’17). Association for Computing Machinery, New York, NY, Article 186, 6 pages.
[5]
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. ArXiv, abs/1803.01271.
[6]
Sourav Bhattacharya and Nicholas D. Lane. 2016. Sparsification and separation of deep learning layers for constrained resource inference on wearables. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM (SenSys’16). ACM, New York, NY, 176–189.
[7]
Bokai Cao, Lei Zheng, Chenwei Zhang, Philip S. Yu, Andrea Piscitello, John Zulueta, Olu Ajilore, Kelly Ryan, and Alex D. Leow. 2017. DeepMood: Modeling mobile phone typing dynamics for mood detection. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’17). Association for Computing Machinery, New York, NY, 747–755.
[8]
J. Chen and X. Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107, 8 (Aug. 2019), 1655–1674.
[9]
M. W. Condry and C. B. Nelson. 2016. Using smart edge iot devices for safer, rapid response with industry IoT control operations. Proc. IEEE 104, 5 (2016), 938–946.
[10]
R. Du, L. Gkatzikis, C. Fischione, and M. Xiao. 2015. Energy efficient sensor activation for water distribution networks based on compressive sensing. IEEE J. Select. Areas Commun. 33, 12 (2015), 2997–3010.
[11]
Angelo Feraudo, Poonam Yadav, Vadim Safronov, Diana Andreea Popescu, Richard Mortier, Shiqiang Wang, Paolo Bellavista, and Jon Crowcroft. 2020. CoLearn: Enabling federated learning in MUD-compliant IoT edge networks. In Proceedings of the 3rd ACM International Workshop on Edge Systems, Analytics and Networking (EdgeSys’20). Association for Computing Machinery, New York, NY, 25–30.
[12]
Surjya Ghosh, Kaustubh Hiware, Niloy Ganguly, Bivas Mitra, and Pradipta De. 2019. Does emotion influence the use of auto-suggest during smartphone typing?. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI’19). Association for Computing Machinery, New York, NY, 144–149.
[13]
B. Guo, Y. Liu, L. Wang, V. O. K. Li, J. C. K. Lam, and Z. Yu. 2018. Task allocation in spatial crowdsourcing: Current state and future directions. IEEE Internet Things J. 5, 3 (2018), 1749–1764.
[14]
Sandeep Gupta, Attaullah Buriro, and Bruno Crispo. 2019. DriverAuth: A risk-based multi-modal biometric-based driver authentication scheme for ride-sharing platforms. Comput. Secur. 83 (2019), 122–139.
[15]
Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both weights and connections for efficient neural networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS’15). MIT Press, Cambridge, MA, 1135–1143.
[16]
Tatsuhito Hasegawa and Makoto Koshino. 2019. Representation learning by convolutional neural network for smartphone sensor based activity recognition. In Proceedings of the 2nd International Conference on Computational Intelligence and Intelligent Systems (CIIS’19). Association for Computing Machinery, New York, NY, 99–104.
[17]
Leila Ismail and Huned Materwala. 2020. Computing server power modeling in a data center: Survey, taxonomy, and performance evaluation. ACM Comput. Surv. 53, 3, Article 58 (June 2020), 34 pages.
[18]
Wen Jiang, Chan Huang, and Xinyang Deng. 2019. A new probability transformation method based on a correlation coefficient of belief functions. Int. J. Intell. Syst. 34, 6 (2019), 1337–1347.
[19]
Meng Jin, Yuan He, Xin Meng, Dingyi Fang, and Xiaojiang Chen. 2018. Parallel backscatter in the wild: When burstiness and randomness play with you. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking (MobiCom’18). Association for Computing Machinery, New York, NY, 471–485.
[20]
Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. SIGARCH Comput. Archit. News 45, 1 (Apr. 2017), 615–629.
[21]
H. Khaleel, D. Conzon, P. Kasinathan, P. Brizzi, C. Pastrone, F. Pramudianto, M. Eisenhauer, P. A. Cultrona, F. Rusinà, G. Lukáč, and M. Paralic. 2017. Heterogeneous applications, tools, and methodologies in the car manufacturing industry through an IoT approach. IEEE Syst. J. 11, 3 (2017), 1412–1423.
[22]
N. D. Lane, S. Bhattacharya, P. Georgiev, C. Forlivesi, L. Jiao, L. Qendro, and F. Kawsar. 2016. DeepX: A software accelerator for low-power deep learning inference on mobile devices. In Proceedings of the 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN’16). 1–12.
[23]
Nicholas D. Lane, Sourav Bhattacharya, Petko Georgiev, Claudio Forlivesi, and Fahim Kawsar. 2015. An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices. In Proceedings of the International Workshop on Internet of Things Towards Applications (IoT-App’15). ACM, New York, NY, 7–12.
[24]
Y. Liang, S. Samtani, B. Guo, and Z. Yu. 2020. Behavioral biometrics for continuous authentication in the internet of things era: An artificial intelligence perspective. IEEE Internet Things J. 7, 9 (2020), 9128–9143.
[25]
Yunji Liang, Zheng Xiaolong, and Daniel Dajun Zeng. 2019. A survey on big data-driven digital phenotyping of mental health. Info. Fusion 52 (11 2019), 290–307.
[26]
Yunji Liang, Xingshe Zhou, Zhiwen Yu, and Bin Guo. 2014. Energy-efficient motion related activity recognition on mobile devices for pervasive healthcare. Mobile Netw. Appl. 19, 3 (June 2014), 303–317.
[27]
Zhihan Lv, Alaa Halawani, Shengzhong Feng, Haibo Li, and Shafiq Ur Réhman. 2014. Multimodal hand and foot gesture interaction for handheld devices. ACM Trans. Multimedia Comput. Commun. Appl. 11, 1s, Article 10 (Oct. 2014), 19 pages.
[28]
D. Ma, G. Lan, M. Hassan, W. Hu, and S. K. Das. 2020. Sensing, computing, and communications for energy harvesting IoTs: A survey. IEEE Commun. Surveys Tutor. 22, 2 (2020), 1222–1250.
[29]
Dong Ma, Guohao Lan, Mahbub Hassan, Wen Hu, Mushfika B. Upama, Ashraf Uddin, and Moustafa Youssef. 2019. SolarGest: Ubiquitous and battery-free gesture recognition using solar cells. In Proceedings of the 25th Annual International Conference on Mobile Computing and Networking (MobiCom’19). Association for Computing Machinery, New York, NY, Article 12, 15 pages.
[30]
Renato Mancuso, Or D. Dantsker, Marco Caccamo, and Michael S. Selig. 2014. A low-power architecture for high frequency sensor acquisition in Many-DOF UAVs. In Proceedings of the ACM/IEEE 5th International Conference on Cyber-Physical Systems (ICCPS’14). IEEE Computer Society, 103–114.
[31]
Héctor A. Cordourier Maruri, Paulo Lopez-Meyer, Jonathan Huang, Willem Marco Beltman, Lama Nachman, and Hong Lu. 2018. V-Speech: Noise-robust speech capturing glasses using vibration sensors. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 180 (Dec. 2018), 23 pages.
[32]
Sven Mayer, Gierad Laput, and Chris Harrison. 2020. Enhancing mobile voice assistants with worldgaze. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’20). Association for Computing Machinery, New York, NY, 1–10.
[33]
David K. McGookin, Laura Maye, Ling Chen, and Mikko Kytö. 2018. An initial study of multisensory interaction for outdoor heritage sites. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA’18). Association for Computing Machinery, New York, NY, 1–6.
[34]
Mohammad Motamedi, Daniel Fong, and Soheil Ghiasi. 2017. Machine intelligence on resource-constrained IoT devices: The case of thread granularity optimization for CNN inference. ACM Trans. Embed. Comput. Syst. 16, 5s, Article 151 (Sept. 2017), 19 pages.
[35]
T. Murakami, S. Kurihara, S. Fukuda, M. Oguchi, and S. Yamaguchi. 2018. Saving power consumption of smartphones in the screen-off state with disabling the Wi-Fi. In Proceedings of the IEEE International Conference on Consumer Electronics (ICCE’18). 1–6.
[36]
T. D. Nguyen, J. Y. Khan, and D. T. Ngo. 2018. A distributed energy-harvesting-aware routing algorithm for heterogeneous IoT networks. IEEE Trans. Green Commun. Netw. 2, 4 (2018), 1115–1127.
[37]
A. Ometov, V. Petrov, S. Bezzateev, S. Andreev, Y. Koucheryavy, and M. Gerla. 2019. Challenges of multi-factor authentication for securing advanced IoT applications. IEEE Netw. 33, 2 (2019), 82–88.
[38]
Sujata Pal, Anindo Ghosh, and Vivek Sethi. 2018. Vehicle air pollution monitoring using IoTs. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems (SenSys’18). Association for Computing Machinery, New York, NY, 400–401.
[39]
Meng-Shiuan Pan and Kuan-Ying Li. 2019. Precise trajectories derivation using smartphone sensors. In Proceedings of the 11th International Conference on Computer Modeling and Simulation (ICCMS’19). Association for Computing Machinery, New York, NY, 196–199.
[40]
Emirhan Poyraz and Gokhan Memik. 2019. Using built-in sensors to predict and utilize user satisfaction for CPU settings on smartphones. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 1, Article 21 (Mar. 2019), 25 pages.
[41]
Swadhin Pradhan, Wei Sun, Ghufran Baig, and Lili Qiu. 2019. Combating replay attacks against voice assistants. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 3, Article 100 (Sept. 2019), 26 pages.
[42]
P. K. D. Pramanik, N. Sinhababu, B. Mukherjee, S. Padmanaban, A. Maity, B. K. Upadhyaya, J. B. Holm-Nielsen, and P. Choudhury. 2019. Power consumption analysis, measurement, management, and issues: A state-of-the-art review of smartphone battery and energy usage. IEEE Access 7 (2019), 182113–182172.
[43]
Lina Qiu, Alexander De Luca, Ildar Muslukhov, and Konstantin Beznosov. 2019. Towards understanding the link between age and smartphone authentication. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–10.
[44]
Rajib Kumar Rana, Chun Tung Chou, Salil Kanhere, Nirupama Bulusu, and Wen Hu. 2009. Ear-phone assessment of noise pollution with mobile phones. In Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems (SenSys’09). Association for Computing Machinery, New York, NY, 395–396.
[45]
M. Rani, S. B. Dhok, and R. B. Deshmukh. 2018. A systematic review of compressive sensing: Concepts, implementations and applications. IEEE Access 6 (2018), 4875–4894.
[46]
David N. Reshef, Yakir A. Reshef, Hilary K. Finucane, Sharon R. Grossman, Gilean Mcvean, Peter J. Turnbaugh, Eric S. Lander, Michael Mitzenmacher, and Pardis C. Sabeti. 2011. Detecting novel associations in large data sets. Science 334, 6062 (2011), 1518–1524.
[47]
Nirupam Roy and Romit Roy Choudhury. 2016. Listening through a vibration motor. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’16). Association for Computing Machinery, New York, NY, 57–69.
[48]
Mintra Ruensuk. 2018. Detecting emotions using smartphone sensors: Technique to raise self-awareness for social media users. In Proceedings of the ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers (UbiComp’18). Association for Computing Machinery, New York, NY, 1813–1816.
[49]
Zhanna Sarsenbayeva, Gabriele Marini, Niels van Berkel, Chu Luo, Weiwei Jiang, Kangning Yang, Greg Wadley, Tilman Dingler, Vassilis Kostakos, and Jorge Goncalves. 2020. Does smartphone use drive our emotions or vice versa? a causal analysis. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’20). Association for Computing Machinery, New York, NY, 1–15.
[50]
Yusuke Sato, Shun Kurihara, Shoki Fukuda, Masato Oguchi, and Saneyasu Yamaguchi. 2017. Height estimation based on sensor data on smartphone. In Proceedings of the 15th International Conference on Advances in Mobile Computing and Multimedia (MoMM’17). Association for Computing Machinery, New York, NY, 102–106.
[51]
Ryota Sawano and Kazuya Murao. 2019. Automatic annotation for human activity and device state recognition using smartphone notification. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers (UbiComp/ISWC’19 Adjunct). Association for Computing Machinery, New York, NY, 819–824.
[52]
Z. Sitová, J. Šeděnka, Q. Yang, G. Peng, G. Zhou, P. Gasti, and K. S. Balagani. 2016. HMOG: New behavioral biometric features for continuous authentication of smartphone users. IEEE Trans. Info. Forens. Secur. 11, 5 (2016), 877–892.
[53]
Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 3645–3650.
[54]
Yoshihiko Suhara, Yinzhan Xu, and Alex “Sandy” Pentland. 2017. DeepMood: Forecasting depressed mood based on self-reported histories via recurrent neural networks. In Proceedings of the 26th International Conference on World Wide Web (WWW’17). International World Wide Web Conferences Steering Committee, 715–724.
[55]
P. Sun, Z. Tian, Z. Wang, and Z. Wang. 2019. PRSS: A prejudiced random sensing strategy for energy-efficient information collection in the internet of things. IEEE Internet Things J. 6, 2 (2019), 2717–2728.
[56]
Arsalan Tavakoli, Aman Kansal, and Suman Nath. 2010. On-line sensing task optimization for shared sensors. In Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN’10). Association for Computing Machinery, New York, NY, 47–57.
[57]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefinedukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). Curran Associates Inc., Red Hook, NY, 6000–6010.
[58]
Haotian Wang. 2019. Residual mask based on mobilenet-v2 for driver’s dangerous behavior recognition. In Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence (CSAI’9). Association for Computing Machinery, New York, NY, USA, 196–199.
[59]
J. Wang, Y. Wang, D. Zhang, and S. Helal. 2018. Energy saving techniques in mobile crowd sensing: Current state and future opportunities. IEEE Commun. Mag. 56, 5 (2018), 164–169.
[60]
Qing Wang, Chenren Xu, Supeng Leng, and Sofie Pollin. 2018. When autonomous drones meet driverless cars. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’18). Association for Computing Machinery, New York, NY, 514.
[61]
Rui Wang, Min S. H. Aung, Saeed Abdullah, Rachel Brian, Andrew T. Campbell, Tanzeem Choudhury, Marta Hauser, John Kane, Michael Merrill, Emily A. Scherer, Vincent W. S. Tseng, and Dror Ben-Zeev. 2016. CrossCheck: Toward passive sensing and detection of mental health changes in people with schizophrenia. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’16). Association for Computing Machinery, New York, NY, 886–897.
[62]
Shixuan Wang, Jiabin Yuan, and Shudi Chen. 2020. Quality-based score level fusion for continuous authentication with motion sensor and face. In Proceedings of the 4th International Conference on Cryptography, Security and Privacy. 58–62.
[63]
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. 2019. Dynamic graph cnn for learning on point clouds. ACM Trans. Graph. 38, 5 (2019), 1–12.
[64]
Fang-Jing Wu, Yu-Chee Tseng, and Wen-Chih Peng. 2016. Activity sense organs: Energy-efficient activity sensing with adaptive duty cycle control. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp’16). Association for Computing Machinery, New York, NY, 229–232.
[65]
Liwen Xu, Xiaohong Hao, Nicholas D. Lane, Xin Liu, and Thomas Moscibroda. 2015. Cost-Aware compressive sensing for networked sensing systems. In Proceedings of the 14th International Conference on Information Processing in Sensor Networks (IPSN’15). Association for Computing Machinery, New York, NY, 130–141.
[66]
M. Xu, F. Qian, M. Zhu, F. Huang, S. Pushp, and X. Liu. 2019. DeepWear: Adaptive local offloading for on-wearable deep learning. IEEE Trans. Mobile Comput. (2019), 1–1.
[67]
Y. Xun, J. Liu, N. Kato, Y. Fang, and Y. Zhang. 2020. Automobile driver fingerprinting: A new machine learning based authentication scheme. IEEE Trans. Industr. Informatics 16, 2 (2020), 1417–1426.
[68]
T. Yang, Y. Chen, and V. Sze. 2017. Designing energy-efficient convolutional neural networks using energy-aware pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6071–6079.
[69]
A. Zaeemzadeh, N. Rahnavard, and M. Shah. 2020. Norm-Preservation: Why residual networks can become extremely deep?IEEE Trans. Pattern Anal. Mach. Intell. (2020), 1–1.
[70]
Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang. 2019. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE 107, 8 (Aug. 2019), 1738–1762.

Cited By

View all

Index Terms

  1. Energy-efficient Collaborative Sensing: Learning the Latent Correlations of Heterogeneous Sensors

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Sensor Networks
      ACM Transactions on Sensor Networks  Volume 17, Issue 3
      August 2021
      333 pages
      ISSN:1550-4859
      EISSN:1550-4867
      DOI:10.1145/3470624
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Journal Family

      Publication History

      Published: 21 June 2021
      Accepted: 01 January 2021
      Revised: 01 November 2020
      Received: 01 July 2020
      Published in TOSN Volume 17, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Energy efficiency
      2. latent correlation learning
      3. collaboration sensing
      4. internet of things
      5. temporal convolutional network
      6. attention mechanism
      7. multi-task learning

      Qualifiers

      • Research-article
      • Refereed

      Funding Sources

      • National Major Program for Technological Innovation 2030-New Generation Artificial Intelligence
      • Natural Science Foundation of China
      • The Central Universities

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)35
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 09 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media