skip to main content
research-article
Public Access

FootprintID: Indoor Pedestrian Identification through Ambient Structural Vibration Sensing

Published: 11 September 2017 Publication History

Abstract

We present FootprintID, an indoor pedestrian identification system that utilizes footstep-induced structural vibration to infer pedestrian identities for enabling various smart building applications. Previous studies have explored other sensing methods, including vision-, RF-, mobile-, and acoustic-based methods. They often require specific sensing conditions, including line-of-sight, high sensor density, and carrying wearable devices. Vibration-based methods, on the other hand, provide easy-to-install sparse sensing and utilize gait to distinguish different individuals. However, the challenge for these methods is that the signals are sensitive to the gait variations caused by different walking speeds and the floor variations caused by structural heterogeneity.
We present FootprintID, a vibration-based approach that achieves robust pedestrian identification. The system uses vibration sensors to detect footstep-induced vibrations. It then selects vibration signals and classifiers to accommodate sensing variations, taking step location and frequency into account. We utilize the physical insight on how individual step signal changes with walking speeds and introduce an iterative transductive learning algorithm (ITSVM) to achieve robust classification with limited labeled training data. When trained only on the average walking speed and tested on different walking speeds, FootprintID achieves up to 96% accuracy and a 3X improvement in extreme speeds compared to the Support Vector Machine. Furthermore, it achieves up to 90% accuracy (1.5X improvement) in uncontrolled experiments.

References

[1]
Jan Achenbach. 2012. Wave propagation in elastic solids. Vol. 16. Elsevier, Atlanta, GA, USA.
[2]
David T Alpert and Martin Allen. 2010. Acoustic gait recognition on a staircase. In World Automation Congress (WAC), 2010. IEEE, New York, USA, 1--6.
[3]
Thomas P Andriacchi, Seungbum Koo, and Sean F Scanlan. 2009. Gait mechanics influence healthy cartilage morphology and osteoarthritis of the knee. The Journal of Bone 8 Joint Surgery 91, Supplement 1 (2009), 95--101.
[4]
Hugo Bachmann. 1992. Case studies of structures with man-induced vibrations. Journal of Structural Engineering 118, 3 (1992), 631--647.
[5]
Jezekiel Ben-Arie and Dibyendu Nandy. 1998. A volumetric/iconic frequency domain representation for objects with application for pose invariant face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 5 (1998), 449--457.
[6]
Chiraz BenAbdelkader, Ross Cutler, and Larry Davis. 2002. Person identification using automatic height and stride estimation. In Pattern Recognition, 2002. Proceedings. 16th International Conference on, Vol. 4. IEEE, New York, USA, 377--380.
[7]
Richard Bono. 2008. Transducer Mounting and Test Setup Configurations. In IMAC XXVI TUTORIAL. SEM, Orlando, Florida USA.
[8]
Bernhard E Boser, Isabelle M Guyon, and Vladimir N Vapnik. 1992. A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory. ACM, 144--152.
[9]
Nikolaos V Boulgouris, Dimitrios Hatzinakos, and Konstantinos N Plataniotis. 2005. Gait recognition: a challenging signal processing technology for biometric identification. signal processing magazine, IEEE 22, 6 (2005), 78--90.
[10]
Alessandra Carriero, Amy Zavatsky, Julie Stebbins, Tim Theologis, and Sandra J Shefelbine. 2009. Correlation between lower limb bone morphology and gait characteristics in children with spastic diplegic cerebral palsy. Journal of Pediatric Orthopaedics 29, 1 (2009), 73--79.
[11]
Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2, 3 (2011), 27.
[12]
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009. Semi-supervised learning (chapelle, o. et al., eds.; 2006){book reviews}. IEEE Transactions on Neural Networks 20, 3 (2009), 542--542.
[13]
Meng-Jung Chung and Mao-Jiun J Wang. 2010. The change of gait parameters during walking at different percentage of preferred walking speed for healthy adults aged 20--60 years. Gait 8 posture 31, 1 (2010), 131--135.
[14]
Federal Facilities Council, National Research Council, and others. 1974. Expansion Joints in Buildings: Technical Report. National Academies Press, Washington, DC, USA.
[15]
Frederic Danion, E Varraine, Mireille Bonnard, and Jean Pailhous. 2003. Stride variability in human gait: the effect of stride frequency and stride length. Gait 8 Posture 18, 1 (2003), 69--77.
[16]
Alexander Ekimov and James M Sabatier. 2006. Vibration and sound signatures of human footsteps in buildingsa). The Journal of the Acoustical Society of America 120, 2 (2006), 762--768.
[17]
Davrondzhon Gafurov, Einar Snekkenes, and Patrick Bours. 2007. Gait authentication and identification using wearable accelerometer sensor. In Automatic Identification Advanced Technologies, 2007 IEEE Workshop on. IEEE, Alghero, Italy, 220--225.
[18]
James R Gage, Peter A Deluca, and Thomas S Renshaw. 1995. Gait analysis: principles and applications. J Bone Joint Surg Am 77, 10 (1995), 1607--1623.
[19]
Alexander Gammerman, Volodya Vovk, and Vladimir Vapnik. 1998. Learning by transduction. In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., Madison, Wisconsin, USA, 148--155.
[20]
Jürgen T Geiger, Maximilian Kneißl, Björn W Schuller, and Gerhard Rigoll. 2014. Acoustic gait-based person identification using hidden Markov models. In Proceedings of the 2014 Workshop on Mapping Personality Traits Challenge and Workshop. ACM, Istanbul, Turkey, 25--30.
[21]
King-Shy Goh, Edward Chang, and Kwang-Ting Cheng. 2001. SVM Binary Classifier Ensembles for Image Classification. In Proceedings of the Tenth International Conference on Information and Knowledge Management (CIKM). ACM, Atlanta, GA, USA, 395--402.
[22]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[23]
William G Halvorsen and David L Brown. 1977. Impulse technique for structural frequency response testing. Sound and Vibration 11, 11 (1977), 8--21.
[24]
Chih-Wei Hsu and Chih-Jen Lin. 2002. A comparison of methods for multiclass support vector machines. IEEE transactions on Neural Networks 13, 2 (2002), 415--425.
[25]
Input/Output, Inc. 2006. SM-24 Geophone Element. Input/Output, Inc. Rev. 3.
[26]
T. Joachims. 1999. Making large-Scale SVM Learning Practical. In Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola (Eds.). MIT Press, Cambridge, MA, Chapter 11, 169--184.
[27]
Stephen C Johnson. 1967. Hierarchical clustering schemes. Psychometrika 32, 3 (1967), 241--254.
[28]
Sukun Kim, Shamim Pakzad, David Culler, James Demmel, Gregory Fenves, Steven Glaser, and Martin Turon. 2007. Health monitoring of civil infrastructures using wireless sensor networks. In Information Processing in Sensor Networks, 2007. IPSN 2007. 6th International Symposium on. IEEE, Cambridge, Massachusetts, USA, 254--263.
[29]
Mike Lam, Mostafa Mirshekari, Shijia Pan, Pei Zhang, and Hae Young Noh. 2016. Robust Occupant Detection Through Step-Induced Floor Vibration By Incorporating Structural Characteristics. In IMAC XXXIV A Conference and Exposition on Structural Dynamics. Vol. 54. SEM, Orlando, Florida USA, 1--13.
[30]
Heyoung Lee, Jung Wook Park, and Abdelsalam Sumi Helal. 2009. Estimation of indoor physical activity level based on footstep vibration signal measured by MEMS accelerometer for personal health care under smart home environments. Control Instrum 5801, 5 (2009).
[31]
Jennifer L Lelas, Gregory J Merriman, Patrick O Riley, and D Casey Kerrigan. 2003. Predicting peak kinematic and kinetic parameters from gait speed. Gait 8 posture 17, 2 (2003), 106--112.
[32]
Jani Mantyjarvi, Mikko Lindholm, Elena Vildjiounaite, S-M Makela, and HA Ailisto. 2005. Identifying users of portable devices from gait pattern with accelerometers. In Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP’05). IEEE International Conference on, Vol. 2. IEEE, Philadelphia, PA, USA, ii--973.
[33]
Lee Middleton, Alex Buss, Alex Bazin, Mark S Nixon, and others. 2005. A floor sensor system for gait recognition. In Automatic Identification Advanced Technologies, 2005. Fourth IEEE Workshop on. IEEE, Buffalo, NY, USA, USA, 171--176.
[34]
Mostafa Mirshekari, Shijia Pan, Adeola Bannis, Yan Pui Mike Lam, Pei Zhang, and Hae Young Noh. 2015. Step-level person localization through sparse sensing of structural vibration. In Proceedings of the 14th International Conference on Information Processing in Sensor Networks. ACM, Seattle, WA, USA, 376--377.
[35]
Mostafa Mirshekari, Shijia Pan, Pei Zhang, and Hae Young Noh. 2016. Characterizing wave propagation to improve indoor step-level person localization using floor vibration. In SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring. International Society for Optics and Photonics, SPIE, Las Vegas, NV, USA, 98034--98034.
[36]
Mostafa Mirshekari, Pei Zhang, and HaeYoung Noh. 2017. Calibration-Free Footstep Frequency Estimation using Structural Vibration. In IMAC XXXV A Conference and Exposition on Structural Dynamics. SEM.
[37]
Friedrich Moser, Laurence J Jacobs, and Jianmin Qu. 1999. Modeling elastic wave propagation in waveguides with the finite element method. Ndt 8 E International 32, 4 (1999), 225--234.
[38]
Le T Nguyen, Yu Seung Kim, Patrick Tague, and Joy Zhang. 2014. IdentityLink: user-device linking through visual and RF-signal cues. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, Seattle, WA, USA, 529--539.
[39]
Jens Bo Nielsen. 2003. How we walk: central control of muscle activity during human walking. The Neuroscientist 9, 3 (2003), 195--204.
[40]
H Noh, R Rajagopal, and AS Kiremidjian. 2013. Sequential structural damage diagnosis algorithm using a change point detection method. Journal of Sound and Vibration 332, 24 (2013), 6419--6433.
[41]
Hae Young Noh, Dimitrios G Lignos, K Krishnan Nair, and Anne S Kiremidjian. 2012. Development of fragility functions as a damage classification/prediction method for steel moment-resisting frames using a wavelet-based damage sensitive feature. Earthquake Engineering 8 Structural Dynamics 41, 4 (2012), 681--696.
[42]
Hae Young Noh, K Krishnan Nair, Anne S Kiremidjian, and CH Loh. 2009. Application of time series based damage detection algorithms to the benchmark experiment at the National Center for Research on Earthquake Engineering (NCREE) in Taipei, Taiwan. Smart Structures and Systems 5, 1 (2009), 95--117.
[43]
Hae Young Noh, Ram Rajagopal, and Anne S Kiremidjian. 2012. Damage diagnosis algorithm using a sequential change point detection method with an unknown distribution for damage. In SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring. International Society for Optics and Photonics, SPIE, San Diego, CA, USA, 834507--834507.
[44]
T Öberg, Alek Karsznia, and K Öberg. 1993. Basic gait parameters: reference data for normal subjects, 10-79 years of age. Journal of rehabilitation research and development 30 (1993), 210--210.
[45]
Tommy Oberg, Alek Karsznia, and Kurt Oberg. 1994. Joint angle parameters in gait: reference data for normal subjects, 10-79 years of age. Journal of rehabilitation Research and Development 31, 3 (1994), 199--213.
[46]
Sandra J Olney, Malcolm P Griffin, and Ian D McBride. 1994. Temporal, kinematic, and kinetic variables related to gait speed in subjects with hemiplegia: a regression approach. Physical therapy 74, 9 (1994), 872--885.
[47]
Robert J Orr and Gregory D Abowd. 2000. The smart floor: a mechanism for natural user identification and tracking. In CHI’00 extended abstracts on Human factors in computing systems. ACM, The Hague, The Netherlands, 275--276.
[48]
Shijia Pan, Amelie Bonde, Jie Jing, Lin Zhang, Pei Zhang, and HaeYoung Noh. 2014. BOES: building occupancy estimation system using sparse ambient vibration monitoring. In SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring. International Society for Optics and Photonics, SPIE, San Diego, CA, USA, 90611O--90611O.
[49]
Shijia Pan, An Chen, and Pei Zhang. 2013. Securitas: user identification through rgb-nir camera pair on mobile devices. In Proceedings of the Third ACM workshop on Security and privacy in smartphones 8 mobile devices. ACM, ACM, Berlin, Germany, 99--104.
[50]
Shijia Pan, Mostafa Mirshekari, Hae Young Noh, and Pei Zhang. 2015. Structural sensing system with networked dynamic sensing configuration. In Proceedings of the 14th International Conference on Information Processing in Sensor Networks. ACM, ACM, Seattle, WA, USA, 344--345.
[51]
Shijia Pan, Mostafa Mirshekari, Pei Zhang, and Hae Young Noh. 2016. Occupant Traffic Estimation through Structural Vibration Sensing. In SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring. International Society for Optics and Photonics, SPIE, Las Vegas, NV, USA, 98035--98035.
[52]
Shijia Pan, Ningning Wang, Yuqiu Qian, Irem Velibeyoglu, Hae Young Noh, and Pei Zhang. 2015. Indoor person identification through footstep induced structural vibration. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. ACM, Santa Fe, NM, USA, 81--86.
[53]
Shijia Pan, Susu Xu, Mostafa Mirshekari, Pei Zhang, and Hae Young Noh. 2017. Collaboratively Adaptive Vibration Sensing System for High Fidelity Monitoring of Structural Responses Induced by Pedestrians. Frontiers in Built Environment 3 (2017), 28.
[54]
Jeffrey D Poston, Javier Schloemann, R Michael Buehrer, VVN Sriram Malladi, Americo G Woolard, and Pablo A Tarazaga. 2015. Towards indoor localization of pedestrians via smart building vibration sensing. In Localization and GNSS (ICL-GNSS), 2015 International Conference on. IEEE, Gothenburg, Sweden, 1--6.
[55]
Boris I Prilutsky and Robert J Gregor. 2001. Swing-and support-related muscle actions differentially trigger human walk--run and run--walk transitions. Journal of Experimental Biology 204, 13 (2001), 2277--2287.
[56]
Liu Rong, Duan Zhiguo, Zhou Jianzhong, and Liu Ming. 2007. Identification of individual walking patterns using gait acceleration. In Bioinformatics and Biomedical Engineering, 2007. ICBBE 2007. The 1st International Conference on. IEEE, Wuhan, China, 543--546.
[57]
Raul Sanchez-Reillo. 2000. Hand geometry pattern recognition through gaussian mixture modelling. In Pattern Recognition, 2000. Proceedings. 15th International Conference on, Vol. 2. IEEE, Barcelona, Spain, Spain, 937--940.
[58]
Mary Sansalone, Nicholas J Carino, and Nelson N Hsu. 1987. A finite element study of transient wave propagation in plates. Journal of research of the National Bureau of Standards 92, 4 (1987), 267--278.
[59]
Pamela L Sheridan, Judi Solomont, Neil Kowall, and Jeffrey M Hausdorff. 2003. Influence of executive function on locomotor function: divided attention increases gait variability in Alzheimer’s disease. Journal of the American Geriatrics Society 51, 11 (2003), 1633--1637.
[60]
Martin L Smith and FA Dahlen. 1973. The azimuthal dependence of Love and Rayleigh wave propagation in a slightly anisotropic medium. Journal of Geophysical Research 78, 17 (1973), 3321--3333.
[61]
Budi Sugandi, Hyoungseop Kim, Joo Kooi Tan, and Seiji Ishikawa. 2009. Real time tracking and identification of moving persons by using a camera in outdoor environment. Int. J. In nov. Comput. Inf. Control 5, 5 (2009), 1179--1188.
[62]
David H Sutherland. 2005. The evolution of clinical gait analysis part III--kinetics and energy assessment. Gait 8 Posture 21, 4 (2005), 447--461.
[63]
Rawesak Tanawongsuwan and Aaron Bobick. 2003. A study of human gaits across different speeds. Technical Report. Georgia Tech, Tech. Rep.
[64]
Thiago Teixeira, Deokwoo Jung, and Andreas Savvides. 2010. Tasking networked cctv cameras and mobile phones to identify and localize multiple people. In Proceedings of the 12th ACM international conference on Ubiquitous computing. ACM, Copenhagen, Denmark, 213--222.
[65]
Y-I Tian, Takeo Kanade, and Jeffrey F Cohn. 2001. Recognizing action units for facial expression analysis. IEEE Transactions on pattern analysis and machine intelligence 23, 2 (2001), 97--115.
[66]
Stephen Timoshenko, Stephen Prokofevich Timoshenko, Stephen Prokofevich Timoshenko, and Stephen Prokofevich Timoshenko. 1956. Strength of materials. Vol. 210. D. Van Nostrand Company, Inc., New York, USA.
[67]
Joe Verghese, Richard B Lipton, Charles B Hall, Gail Kuslansky, Mindy J Katz, and Herman Buschke. 2002. Abnormality of gait as a predictor of non-Alzheimer’s dementia. New England Journal of Medicine 347, 22 (2002), 1761--1768.
[68]
Liang Wang, Tieniu Tan, Huazhong Ning, and Weiming Hu. 2003. Silhouette analysis-based gait recognition for human identification. IEEE transactions on pattern analysis and machine intelligence 25, 12 (2003), 1505--1518.
[69]
Wei Wang, Alex X Liu, and Muhammad Shahzad. 2016. Gait recognition using wifi signals. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 363--373.
[70]
David A Winter. 1984. Kinematic and kinetic patterns in human gait: variability and compensating effects. Human Movement Science 3, 1 (1984), 51--76.
[71]
Chenren Xu, Bernhard Firner, Robert S Moore, Yanyong Zhang, Wade Trappe, Richard Howard, Feixiong Zhang, and Ning An. 2013. Scpl: Indoor device-free multi-subject counting and localization using radio signal strength. In Information Processing in Sensor Networks (IPSN), 2013 ACM/IEEE International Conference on. IEEE, Philadelphia, USA, 79--90.
[72]
Jaynie F Yang and Monica Gorassini. 2006. Spinal and brain control of human walking: implications for retraining of walking. The Neuroscientist 12, 5 (2006), 379--389.
[73]
P Constance Yang, Charles H Norris, and Yehuda Stavsky. 1966. Elastic wave propagation in heterogeneous plates. International Journal of solids and structures 2, 4 (1966), 665--684.
[74]
Hae Young Noh, K Krishnan Nair, Dimitrios G Lignos, and Anne S Kiremidjian. 2011. Use of wavelet-based damage-sensitive features for structural damage diagnosis using strong motion data. Journal of Structural Engineering 137, 10 (2011), 1215--1228.
[75]
Lee Yung-Hui and Hong Wei-Hsien. 2005. Effects of shoe inserts and heel height on foot pressure, impact force, and perceived comfort during walking. Applied ergonomics 36, 3 (2005), 355--362.
[76]
Yunze Zeng, Parth H Pathak, and Prasant Mohapatra. 2016. WiWho: wifi-based person identification in smart spaces. In Proceedings of the 15th International Conference on Information Processing in Sensor Networks. IEEE Press, Vienna, Austria, 4.
[77]
Wei-Shi Zheng, Shaogang Gong, and Tao Xiang. 2011. Person re-identification by probabilistic relative distance comparison. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, Colorado Springs, CO, USA, USA, 649--656.

Cited By

View all

Index Terms

  1. FootprintID: Indoor Pedestrian Identification through Ambient Structural Vibration Sensing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 3
    September 2017
    2023 pages
    EISSN:2474-9567
    DOI:10.1145/3139486
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 September 2017
    Accepted: 01 July 2017
    Revised: 01 May 2017
    Received: 01 February 2017
    Published in IMWUT Volume 1, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. ambient vibration sensing
    2. pedestrian identification
    3. structural vibration

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)373
    • Downloads (Last 6 weeks)45
    Reflects downloads up to 05 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Full Access

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media