skip to main content
research-article

FG-LiquID: A Contact-less Fine-grained Liquid Identifier by Pushing the Limits of Millimeter-wave Sensing

Published: 14 September 2021 Publication History

Abstract

Contact-less liquid identification via wireless sensing has diverse potential applications in our daily life, such as identifying alcohol content in liquids, distinguishing spoiled and fresh milk, and even detecting water contamination. Recent works have verified the feasibility of utilizing mmWave radar to perform coarse-grained material identification, e.g., discriminating liquid and carpet. However, they do not fully exploit the sensing limits of mmWave in terms of fine-grained material classification. In this paper, we propose FG-LiquID, an accurate and robust system for fine-grained liquid identification. To achieve the desired fine granularity, FG-LiquID first focuses on the small but informative region of the mmWave spectrum, so as to extract the most discriminative features of liquids. Then we design a novel neural network, which uncovers and leverages the hidden signal patterns across multiple antennas on mmWave sensors. In this way, FG-LiquID learns to calibrate signals and finally eliminate the adverse effect of location interference caused by minor displacement/rotation of the liquid container, which ensures robust identification towards daily usage scenarios. Extensive experimental results using a custom-build prototype demonstrate that FG-LiquID can accurately distinguish 30 different liquids with an average accuracy of 97%, under 5 different scenarios. More importantly, it can discriminate quite similar liquids, such as liquors with the difference of only 1% alcohol concentration by volume.

References

[1]
2021. Pytorch. https://pytorch.org/.
[2]
Fadel Adib, Zachary Kabelac, Dina Katabi, and Robert C. Miller. 2014. 3D Tracking via Body Radio Reflections. In Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2014, Seattle, WA, USA, April 2-4, 2014. 317--329.
[3]
S. Bhadra, D. J. Thomson, and G. E. Bridges. 2012. A wireless passive pH sensor for real-time in vivo milk quality monitoring. In SENSORS, 2012 IEEE. 1--4.
[4]
Binu Melit Devassy, Sony George, and Peter Nussbaum. 2020. Unsupervised Clustering of Hyperspectral Paper Data Using t-SNE. J. Imaging 6, 5 (2020), 29.
[5]
Ashutosh Dhekne, Mahanth Gowda, Yixuan Zhao, Haitham Hassanieh, and Romit Roy Choudhury. 2018. LiquID: A Wireless Liquid IDentifier. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2018, Munich, Germany, June 10-15, 2018. 442--454.
[6]
Chao Feng, Jie Xiong, Liqiong Chang, Ju Wang, Xiaojiang Chen, Dingyi Fang, and Zhanyong Tang. 2019. WiMi: Target Material Identification with Commodity Wi-Fi Devices. In 39th IEEE International Conference on Distributed Computing Systems, ICDCS 2019, Dallas, TX, USA, July 7-10, 2019. 700--710.
[7]
Carlo Fischione, Dimitrios Koutsonikolas, Sundeep Rangan, Ljiljana Simic, Joerg Widmer, Xinyu Zhang, and Anfu Zhou. 2019. Guest Editorial Millimeter-Wave Networking. IEEE J. Sel. Areas Commun. 37, 12 (2019), 2649--2652.
[8]
J. Fu, H. Zheng, and T. Mei. 2017. Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4476--4484.
[9]
Yasaman Ghasempour, Muhammad Kumail Haider, Carlos Cordeiro, Dimitrios Koutsonikolas, and Edward W. Knightly. 2018. Multi-Stream Beam-Training for mmWave MIMO Networks. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking, MobiCom 2018, New Delhi, India, October 29 - November 02, 2018. 225--239.
[10]
Unsoo Ha, Junshan Leng, Alaa Khaddaj, and Fadel Adib. 2020. Food and Liquid Sensing in Practical Environments using RFIDs. In 17th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2020, Santa Clara, CA, USA, February 25-27, 2020. 1083--1100.
[11]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. 770--778.
[12]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-Excitation Networks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. 7132--7141.
[13]
Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6--11 July 2015. 448--456.
[14]
Xinrui Jiang, Ye Zhang, Qi Yang, Bin Deng, and Hongqiang Wang. 2020. Millimeter-Wave Array Radar-Based Human Gait Recognition Using Multi-Channel Three-Dimensional Convolutional Neural Network. Sensors 20, 19 (2020), 5466.
[15]
Jr. King, LeRoy H., Kent P. Bradley, and Jr. Shires, Dana L. 1970. Hemodialysis for Isopropyl Alcohol Poisoning. JAMA 211, 11 (03 1970), 1855--1855.
[16]
Takumi Kobayashi. 2019. Large Margin In Softmax Cross-Entropy Loss. In 30th British Machine Vision Conference 2019, BMVC 2019, Cardiff, UK, September 9-12, 2019. 139.
[17]
Jonathan Krause, Hailin Jin, Jianchao Yang, and Fei-Fei Li. 2015. Fine-grained recognition without part annotations. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. 5546--5555.
[18]
Soo Min Kwon, Song Yang, Jian Liu, Xin Yang, Wesam Saleh, Shreya Patel, Christine Mathews, and Yingying Chen. 2019. Demo: Hands-Free Human Activity Recognition Using Millimeter-Wave Sensors. In 2019 IEEE International Symposium on Dynamic Spectrum Access Networks, DySPAN 2019, Newark, NJ, USA, November 11-14, 2019. 1--2.
[19]
B. Langen, G. Lober, and W. Herzig. 1994. Reflection and transmission behaviour of building materials at 60 GHz. In 5th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, Wireless Networks - Catching the Mobile Future., Vol. 2. 505-509 vol.2.
[20]
Yuanzhi Li and Yang Yuan. 2017. Convergence Analysis of Two-layer Neural Networks with ReLU Activation. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4--9 December 2017, Long Beach, CA, USA. 597--607.
[21]
Jaime Lien, Nicholas Gillian, Mustafa Emre Karagozler, Patrick Amihood, Carsten Schwesig, Erik Olson, Hakim Raja, and Ivan Poupyrev. 2016. Soli: ubiquitous gesture sensing with millimeter wave radar. ACM Trans. Graph. 35, 4 (2016), 142:1-142:19.
[22]
Haipeng Liu, Yuheng Wang, Anfu Zhou, Hanyue He, Wei Wang, Kunpeng Wang, Peilin Pan, Yixuan Lu, Liang Liu, and Huadong Ma. 2020. Real-time Arm Gesture Recognition in Smart Home Scenarios via Millimeter Wave Sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4 (2020), 140:1-140:28.
[23]
Chris Xiaoxuan Lu, Stefano Rosa, Peijun Zhao, Bing Wang, Changhao Chen, John A. Stankovic, Niki Trigoni, and Andrew Markham. 2020. See through smoke: robust indoor mapping with low-cost mmWave radar. In MobiSys '20: The 18th Annual International Conference on Mobile Systems, Applications, and Services, Toronto, Ontario, Canada, June 15-19, 2020. 14--27.
[24]
Michael Lu and Nam Sun Wang. 2017. Chapter 7 - Spoilage of Milk and Dairy Products. In The Microbiological Quality of Food, Antonio Bevilacqua, Maria Rosaria Corbo, and Milena Sinigaglia (Eds.). Woodhead Publishing, 151--178.
[25]
Mukesh Kumar Maheshwari, Mamta Agiwal, Navrati Saxena, and Abhishek Roy. 2019. Directional Discontinuous Reception (DDRX) for mmWave Enabled 5G Communications. IEEE Trans. Mob. Comput. 18, 10 (2019), 2330--2343.
[26]
Zhen Meng, Song Fu, Jie Yan, Hongyuan Liang, Anfu Zhou, Shilin Zhu, Huadong Ma, Jianhua Liu, and Ning Yang. 2020. Gait Recognition for Co-Existing Multiple People Using Millimeter Wave Sensing. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, New York, NY, USA, February 7-12, 2020. 849--856.
[27]
Arshak Poghossian, Hanno Geissler, and Michael J. Schöning. 2019. Rapid methods and sensors for milk quality monitoring and spoilage detection. Biosensors and Bioelectronics 140 (2019), 111272.
[28]
Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 77--85.
[29]
Tauhidur Rahman, Alexander Travis Adams, Perry Schein, Aadhar Jain, David Erickson, and Tanzeem Choudhury. 2016. Nutrilyzer: A Mobile System for Characterizing Liquid Food with Photoacoustic Effect. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems, SenSys 2016, Stanford, CA, USA, November 14-16, 2016. 123--136.
[30]
R. Raju, G. E. Bridges, and S. Bhadra. 2020. Wireless Passive Sensors for Food Quality Monitoring: Improving the Safety of Food Products. IEEE Antennas and Propagation Magazine 62, 5 (2020), 76--89.
[31]
Chenjun Shi, Ji Zhu, Mingqian Xu, Xu Wu, Yan Peng, and Wenzheng Bao. 2020. An Approach of Spectra Standardization and Qualitative Identification for Biomedical Materials Based on Terahertz Spectroscopy. Sci. Program. 2020 (2020), 8841565:1-8841565:8.
[32]
Akash Deep Singh, Sandeep Singh Sandha, Luis Garcia, and Mani B. Srivastava. 2019. RadHAR: Human Activity Recognition from Point Clouds Generated through a Millimeter-wave Radar. In Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems, mmNets@MobiCom 2019, October 25, 2019, Los Cabos, Mexico. 51--56.
[33]
Project Soli. 2019. https://www.google.com/atap/project-soli/.
[34]
Monika Szymanska-Chargot, Justyna Cybulska, and Artur Zdunek. 2011. Sensing the Structural Differences in Cellulose from Apple and Bacterial Cell Wall Materials by Raman and FT-IR Spectroscopy. Sensors 11, 6 (2011), 5543--5560.
[35]
Georgios Tsiminis, Fenghong Chu, Stephen C. Warren-Smith, Nigel A. Spooner, and Tanya M. Monro. 2013. Identification and Quantification of Explosives in Nanolitre Solution Volumes by Raman Spectroscopy in Suspended Core Optical Fibers. Sensors 13, 10 (2013), 13163--13177.
[36]
Fengyu Wang, Feng Zhang, Chenshu Wu, Beibei Wang, and K. J. Ray Liu. 2020. ViMo: Vital Sign Monitoring Using Commodity Millimeter Wave Radio. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020. 8304--8308.
[37]
Ju Wang, Jie Xiong, Xiaojiang Chen, Hongbo Jiang, Rajesh Krishna Balan, and Dingyi Fang. 2017. TagScan: Simultaneous Target Imaging and Material Identification with Commodity RFID Devices. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, MobiCom 2017, Snowbird, UT, USA, October 16 - 20, 2017. 288--300.
[38]
Saiwen Wang, Jie Song, Jaime Lien, Ivan Poupyrev, and Otmar Hilliges. 2016. Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST 2016, Tokyo, Japan, October 16-19, 2016. 851--860.
[39]
J. Weiß and A. Santra. 2018. One-Shot Learning for Robust Material Classification Using Millimeter-Wave Radar System. IEEE Sensors Letters 2, 4 (2018), 1--4.
[40]
Chenshu Wu, Feng Zhang, Beibei Wang, and K.J. Ray Liu. 2020. mSense: Towards Mobile Material Sensing with a Single Millimeter-Wave Radio. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3 (2020), 106:1-106:20.
[41]
Jeffries. Wyman. 1931. THE DIELECTRIC CONSTANT OF MIXTURES OF ETHYL ALCOHOL AND WATER FROM -5 TO 40°. Journal of the American Chemical Society 53, 9 (1931), 3292--3301.
[42]
Binbin Xie, Jie Xiong, Xiaojiang Chen, Eugene Chai, Liyao Li, Zhanyong Tang, and Dingyi Fang. 2019. Tagtag: material sensing with commodity RFID. In Proceedings of the 17th Conference on Embedded Networked Sensor Systems, SenSys 2019, New York, NY, USA, November 10-13, 2019. 338--350.
[43]
Dongzhu Xu, Anfu Zhou, Xinyu Zhang, Guixian Wang, Xi Liu, Congkai An, Yiming Shi, Liang Liu, and Huadong Ma. 2020. Understanding Operational 5G: A First Measurement Study on Its Coverage, Performance and Energy Consumption. In SIGCOMM '20: Proceedings of the 2020 Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication, Virtual Event, USA, August 10-14, 2020. 479 -494.
[44]
Ke Yan, Yonghong Tian, Yaowei Wang, Wei Zeng, and Tiejun Huang. 2017. Exploiting Multi-grain Ranking Constraints for Precisely Searching Visually-similar Vehicles. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017. 562--570.
[45]
Xin Yang, Jian Liu, Yingying Chen, Xiaonan Guo, and Yucheng Xie. 2020. MU-ID: Multi-user Identification Through Gaits Using Millimeter Wave Radios. In 39th IEEE Conference on Computer Communications, INFOCOM 2020, Toronto, ON, Canada, July 6-9, 2020. 2589--2598.
[46]
Zhicheng Yang, Parth H. Pathak, Yunze Zeng, Xixi Liran, and Prasant Mohapatra. 2016. Monitoring vital signs using millimeter wave. In Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc 2016, Paderborn, Germany, July 4-8, 2016. 211--220.
[47]
Zhicheng Yang, Parth H. Pathak, Yunze Zeng, Xixi Liran, and Prasant Mohapatra. 2017. Vital Sign and Sleep Monitoring Using Millimeter Wave. ACM Trans. Sens. Networks 13, 2 (2017), 14:1-14:32.
[48]
Hui-Shyong Yeo, Gergely Flamich, Patrick Schrempf, David Harris-Birtill, and Aaron Quigley. 2016. RadarCat: Radar Categorization for Input & Interaction. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST 2016, Tokyo, Japan, October 16-19, 2016. 833--841.
[49]
Shichao Yue and Dina Katabi. 2019. Liquid Testing with Your Smartphone. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys 2019, Seoul, Republic of Korea, June 17-21, 2019. 275--286.
[50]
Fahim Zaman, Aslam Pervez, and Kenneth Abreo. 2002. Isopropyl alcohol intoxication: A diagnostic challenge. American Journal of Kidney Diseases 40, 3 (2002), e12.1-e12.4.
[51]
Yunze Zeng, Parth H. Pathak, Zhicheng Yang, and Prasant Mohapatra. 2016. Poster Abstract: Human Tracking and Activity Monitoring Using 60 GHz mmWave. In 15th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2016, Vienna, Austria, April 11-14, 2016. 25:1-25:2.
[52]
Daqing Zhang, Hao Wang, and Dan Wu. 2017. Toward Centimeter-Scale Human Activity Sensing with Wi-Fi Signals. Computer 50, 1 (2017), 48--57.
[53]
Diana Zhang, Jingxian Wang, Junsu Jang, Junbo Zhang, and Swarun Kumar. 2019. On the Feasibility of Wi-Fi Based Material Sensing. In The 25th Annual International Conference on Mobile Computing and Networking, MobiCom 2019, Los Cabos, Mexico, October 21-25, 2019. 41:1-41:16.
[54]
Anfu Zhou, Leilei Wu, Shaoqing Xu, Huadong Ma, Teng Wei, and Xinyu Zhang. 2018. Following the Shadow: Agile 3-D Beam-Steering for 60 GHz Wireless Networks. In 2018 IEEE Conference on Computer Communications, INFOCOM 2018, Honolulu, HI, USA, April 16-19, 2018. 2375--2383.
[55]
Anfu Zhou, Shaoqing Xu, Song Wang, Jingqi Huang, Shaoyuan Yang, Teng Wei, Xinyu Zhang, and Huadong Ma. 2019. Robot Navigation in Radio Beam Space: Leveraging Robotic Intelligence for Seamless mmWave Network Coverage. In Proceedings of the Twentieth ACM International Symposium on Mobile Ad Hoc Networking and Computing, Mobihoc 2019, Catania, Italy, July 2-5, 2019. 161--170.
[56]
Anfu Zhou, Shaoqing Xu, Song Wang, Jingqi Huang, Shaoyuan Yang, Teng Wei, Xinyu Zhang, and Huadong Ma. 2020. Robotic Millimeter-Wave Wireless Networks. IEEE/ACM Trans. Netw. 28, 4 (2020), 1534--1549.
[57]
Anfu Zhou, Shaoyuan Yang, Yi Yang, Yuhang Fan, and Huadong Ma. 2019. Autonomous Environment Mapping Using Commodity Millimeter-wave Network Device. In 2019 IEEE Conference on Computer Communications, INFOCOM 2019, Paris, France, April 29 - May 2, 2019. 1126--1134.
[58]
Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning Deep Features for Discriminative Localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. 2921--2929.
[59]
Bingpeng Zhou, An Liu, and Vincent K. N. Lau. 2019. Successive Localization and Beamforming in 5G mmWave MIMO Communication Systems. IEEE Trans. Signal Process. 67, 6 (2019), 1620--1635.
[60]
Yibo Zhu, Yanzi Zhu, Zengbin Zhang, Ben Y. Zhao, and Haitao Zheng. 2015. 60GHz Mobile Imaging Radar. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, HotMobile 2015, Santa Fe, NM, USA, February 12-13, 2015. 75--80.
[61]
Yanzi Zhu, Yibo Zhu, Ben Y. Zhao, and Haitao Zheng. 2015. Reusing 60GHz Radios for Mobile Radar Imaging. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, MobiCom 2015, Paris, France, September 7-11, 2015. 103--116.

Cited By

View all
  • (2024)mm-CUR: A Novel Ubiquitous, Contact-free, and Location-aware Counterfeit Currency Detection in Bundles Using Millimeter-Wave SensorACM Transactions on Sensor Networks10.1145/3694970Online publication date: 5-Sep-2024
  • (2024)GrainSense: A Wireless Grain Moisture Sensing System Based on Wi-Fi SignalsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785898:3(1-25)Online publication date: 9-Sep-2024
  • (2024)airTac: A Contactless Digital Tactile Receptor for Detecting Material and Roughness via Terahertz SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785868:3(1-37)Online publication date: 9-Sep-2024
  • Show More Cited By

Index Terms

  1. FG-LiquID: A Contact-less Fine-grained Liquid Identifier by Pushing the Limits of Millimeter-wave Sensing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 5, Issue 3
    Sept 2021
    1443 pages
    EISSN:2474-9567
    DOI:10.1145/3486621
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 September 2021
    Published in IMWUT Volume 5, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 60GHz mmWave
    2. Contactless Sensing
    3. FMCW Radar
    4. Liquid Identification
    5. Material Identification
    6. Neural Networks
    7. Wireless Sensing

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • the Funds for Creative Research Groups of China
    • NSFC
    • the Youth Top Talent Support Program
    • the 111 Project
    • BUPT Excellent Ph.D. Students Foundation

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)298
    • Downloads (Last 6 weeks)25
    Reflects downloads up to 14 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media