research-article

Free access

A Robust Sign Language Recognition System with Multiple Wi-Fi Devices

Authors:

Jiacheng Shang,

Jie WuAuthors Info & Claims

MobiArch '17: Proceedings of the Workshop on Mobility in the Evolving Internet Architecture

Pages 19 - 24

https://doi.org/10.1145/3097620.3097624

Published: 11 August 2017 Publication History

PDF eReader

Abstract

Sign language is important since it provides a way for us to the deaf culture and more opportunities to communicate with those who are deaf or hard of hearing. Since sign language chiefly uses body languages to convey meaning, Human Activity Recognition (HAR) techniques can be used to recognize them for some sign language translation applications. In this paper, we show for the first time that Wi-Fi signals can be used to recognize sign language. The key intuition is that different hand and arm motions introduce different multi-path distortions in Wi-Fi signals and generate different unique patterns in the time-series of Channel State Information (CSI). More specifically, we propose a Wi-Fi signal-based sign language recognition system called WiSign. Different from existing Wi-Fi signal-based human activity recognition systems, WiSign uses 3 Wi-Fi devices to improve the recognition performance. We implemented the WiSign using a TP-Link TL-WR1043ND Wi-Fi router and two Lenovo X100e laptops. The evaluation results show that our system can achieve a mean prediction accuracy of 93.8% and mean false positive of 1.55%.

Supplementary Material

WEBM File (arobustsignlanguagerecognitionsystemwithmultiplewifidevices.webm)

Download
57.97 MB

References

[1]

http://dhalperi.github.io/linux-80211n-csitool/.

Google Scholar

[2]

http://www.castlerockfamilyenrichmentcenter.com/workshops/baby-signing/.

Google Scholar

[3]

H. Abdelnasser, M. Youssef, and K. A. Harras. Wigest: A ubiquitous wifi-based gesture recognition system. In Proceedings of the INFOCOM, pages 1472--1480. IEEE, 2015.

Crossref

Google Scholar

[4]

F. Adib, Z. Kabelac, D. Katabi, and R. C. Miller. 3d tracking via body radio reflections. In Proceedings of the NSDI, pages 317--329, 2014.

Digital Library

Google Scholar

[5]

K. Ali, A. X. Liu, W. Wang, and M. Shahzad. Keystroke recognition using wifi signals. In Proceedings of the MobiCom, pages 90--102. ACM, 2015.

Digital Library

Google Scholar

[6]

M. Kotaru, K. Joshi, D. Bharadia, and S. Katti. Spotfi: Decimeter level localization using wifi. In Proceedings of the SIGCOMM, pages 269--282. ACM, 2015.

Digital Library

Google Scholar

[7]

Q. Pu, S. Gupta, S. Gollakota, and S. Patel. Whole-home gesture recognition using wireless signals. In Proceedings of the MobiCom, pages 27--38. ACM, 2013.

Digital Library

Google Scholar

[8]

J. Shang and J. Wu. Fine-grained vital signs estimation using commercial wi-fi devices. In Proceedings of the Eighth Wireless of the Students, by the Students, and for the Students Workshop, pages 30--32. ACM, 2016.

Digital Library

Google Scholar

[9]

G. Wang, Y. Zou, Z. Zhou, K. Wu, and L. M. Ni. We can hear you with wi-fi! In Proceedings of the 20th annual international conference on Mobile computing and networking, pages 593--604. ACM, 2014.

Digital Library

Google Scholar

[10]

W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the MobiCom, pages 65--76. ACM, 2015.

Digital Library

Google Scholar

[11]

W. Xi, J. Zhao, X.-Y. Li, K. Zhao, S. Tang, X. Liu, and Z. Jiang. Electronic frog eye: Counting crowd using wifi. In Proceedings of the INFOCOM, pages 361--369. IEEE, 2014.

Crossref

Google Scholar

Cited By

View all

Ghosian Moghaddam MAsghar Nazari Shirehjini AShirmohammadi S(2025)Device-Free Human Activity Recognition: A Systematic Literature ReviewIEEE Open Journal of Instrumentation and Measurement10.1109/OJIM.2024.35028854(1-34)Online publication date: 2025
https://doi.org/10.1109/OJIM.2024.3502885
Pranav Katarya R(2025)Effi-CNN: real-time vision-based system for interpretation of sign language using CNN and transfer learningMultimedia Tools and Applications10.1007/s11042-024-20585-1Online publication date: 7-Jan-2025
https://doi.org/10.1007/s11042-024-20585-1
Zhou HLu TDeHaan KGowda M(2024)ASLRing: American Sign Language Recognition with Meta-Learning on Wearables2024 IEEE/ACM Ninth International Conference on Internet-of-Things Design and Implementation (IoTDI)10.1109/IoTDI61053.2024.00022(203-214)Online publication date: 13-May-2024
https://doi.org/10.1109/IoTDI61053.2024.00022
Show More Cited By

Index Terms

A Robust Sign Language Recognition System with Multiple Wi-Fi Devices
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Kernel methods
        Support vector machines
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile devices
      1. Mobile devices

Recommendations

Wi-Fi Sensing Techniques for Human Activity Recognition: Brief Survey, Potential Challenges, and Research Directions
Recent advancements in wireless communication technologies have made Wi-Fi signals indispensable in both personal and professional settings. The utilization of these signals for Human Activity Recognition (HAR) has emerged as a cutting-edge technology. By ...
Human Activity Recognition based on Wi-Fi CSI Data -A Deep Neural Network Approach
Abstract
Using Wi-Fi Channel State Information (CSI) is a novel way of environmental sensing and human activity recognition (HAR). These methods can be used for several safety and security applications by (re)using Wi-Fi routers without the need for ...
Grammatical facial expression recognition in sign language discourse: a study at the syntax level

Facial Expression Recognition is an already well-developed research area, mainly due to its applicability in the construction of different system types. Facial expressions are especially important in the area which relates to the construction of ...

Comments

Information & Contributors

Information

Published In

MobiArch '17: Proceedings of the Workshop on Mobility in the Evolving Internet Architecture

August 2017

53 pages

ISBN:9781450350594

DOI:10.1145/3097620

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SIGCOMM '17

Sponsor:

SIGCOMM

SIGCOMM '17: ACM SIGCOMM 2017 Conference

August 25, 2017

CA, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 47 of 92 submissions, 51%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
654
Total Downloads

Downloads (Last 12 months)116
Downloads (Last 6 weeks)19

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ghosian Moghaddam MAsghar Nazari Shirehjini AShirmohammadi S(2025)Device-Free Human Activity Recognition: A Systematic Literature ReviewIEEE Open Journal of Instrumentation and Measurement10.1109/OJIM.2024.35028854(1-34)Online publication date: 2025
https://doi.org/10.1109/OJIM.2024.3502885
Pranav Katarya R(2025)Effi-CNN: real-time vision-based system for interpretation of sign language using CNN and transfer learningMultimedia Tools and Applications10.1007/s11042-024-20585-1Online publication date: 7-Jan-2025
https://doi.org/10.1007/s11042-024-20585-1
Zhou HLu TDeHaan KGowda M(2024)ASLRing: American Sign Language Recognition with Meta-Learning on Wearables2024 IEEE/ACM Ninth International Conference on Internet-of-Things Design and Implementation (IoTDI)10.1109/IoTDI61053.2024.00022(203-214)Online publication date: 13-May-2024
https://doi.org/10.1109/IoTDI61053.2024.00022
He QYang EFang S(2024)A Survey on Human Profile Information Inference via Wireless SignalsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.337339726:4(2577-2610)Online publication date: Dec-2025
https://doi.org/10.1109/COMST.2024.3373397
van Berlo BOerlemans CMarogna FOzcelebi TMeratnia N(2023)Mini-Batch Alignment: A Deep-Learning Model for Domain Factor-Independent Feature Extraction for Wi-Fi–CSI DataSensors10.3390/s2323953423:23(9534)Online publication date: 30-Nov-2023
https://doi.org/10.3390/s23239534
Ji SZhang XZheng YLi MEskicioglu RHuang PPatwari N(2023)Construct 3D Hand Skeleton with Commercial WiFiProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems10.1145/3625687.3625812(322-334)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3625687.3625812
Li JHuang LShah SJones SJin YWang DRussell AChoi SGao YYuan JJin Z(2023)SignRingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108817:3(1-29)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610881
Jin YZhang SGao YXu XChoi SLi ZAdler HJin Z(2023)SmartASLProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962557:2(1-21)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596255
Lei SSun ZYu ZWang ZGuo B(2023)CrowdFi: A Communication Efficient Multi-device Wi-Fi Sensing SystemProceedings of the 25th International Conference on Mobile Human-Computer Interaction10.1145/3565066.3608697(1-7)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1145/3565066.3608697
Lin GJiang WXu SZhou XGuo XZhu YHe X(2023)Human Activity Recognition Using Smartphones With WiFi SignalsIEEE Transactions on Human-Machine Systems10.1109/THMS.2022.318872653:1(142-153)Online publication date: Feb-2023
https://doi.org/10.1109/THMS.2022.3188726
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Wi-Fi Sensing Techniques for Human Activity Recognition: Brief Survey, Potential Challenges, and Research Directions

Human Activity Recognition based on Wi-Fi CSI Data -A Deep Neural Network Approach

Grammatical facial expression recognition in sign language discourse: a study at the syntax level

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

PDF

eReader

Login options

Full Access

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Wi-Fi Sensing Techniques for Human Activity Recognition: Brief Survey, Potential Challenges, and Research Directions

Human Activity Recognition based on Wi-Fi CSI Data -A Deep Neural Network Approach

Grammatical facial expression recognition in sign language discourse: a study at the syntax level

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations