article

Signer independent isolated Italian sign recognition based on hidden Markov models

Authors:

Emanuele Principi,

Stefano Squartini,

Francesco PiazzaAuthors Info & Claims

Pattern Analysis & Applications, Volume 18, Issue 2

Pages 385 - 402

https://doi.org/10.1007/s10044-014-0400-z

Published: 01 May 2015 Publication History

Abstract

Sign languages represent the most natural way to communicate for deaf and hard of hearing. However, there are often barriers between people using this kind of languages and hearing people, typically oriented to express themselves by means of oral languages. To facilitate the social inclusiveness in everyday life for deaf minorities, technology can play an important role. Indeed many attempts have been recently made by the scientific community to develop automatic translation tools. Unfortunately, not many solutions are actually available for the Italian Sign Language (Lingua Italiana dei Segni--LIS) case study, specially for what concerns the recognition task. In this paper, the authors want to face such a lack, in particular addressing the signer-independent case study, i.e., when the signers in the testing set are to included in the training set. From this perspective, the proposed algorithm represents the first real attempt in the LIS case. The automatic recognizer is based on Hidden Markov Models (HMMs) and video features have been extracted using the OpenCV open source library. The effectiveness of the HMM system is validated by a comparative evaluation with Support Vector Machine approach. The video material used to train the recognizer and testing its performance consists in a database that the authors have deliberately created by involving 10 signers and 147 isolated-sign videos for each signer. The database is publicly available. Computer simulations have shown the effectiveness of the adopted methodology, with recognition accuracies comparable to those obtained by the automatic tools developed for other sign languages.

References

[1]

von Agris, U, Kraiss, KF (2007) Towards a video corpus for signer-independent continuous sign language recognition. In: Proceedings of the International Workshop on Gesture in Human---Computer Interaction and Simulation, Lisbon

[2]

von Agris U, Zieren J, Canzler U, Bauer B, Kraiss KF (2008) Recent developments in visual sign language recognition. Univers Access Inf Soc 6(4):323---362

Digital Library

[3]

Al-Ahdal M, Tahir N (2012) Review in sign language recognition systems. In: Proceedings of IEEE Symposium on Computers Informatics, pp 52---57

[4]

Bertoldi N, Tiotto G, Prinetto P, Piccolo E, Nunnari F, Lombardo V, Mazzei A, Damiano R, Lesmo L, Del Principe A (2010) On the creation and the annotation of a large-scale Italian-LIS parallel corpus. In: Proceedings of the International Conference on Language Resources and Evaluation, pp 19---22

[5]

Bishop C (2006) Pattern recognition and machine learning. Springer Science+Business Media, LLC, New York

Digital Library

[6]

Bradski G (2000) The OpenCV library. Dr Dobb's J Softw Tools 25(11):120 (122---125)

[7]

Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O'Reilly Media Inc., Sebastopol

[8]

Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679---698

Digital Library

[9]

Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2:1---27

Digital Library

[10]

Cortes C, Vapnik V (1995) Support-vector networks. In: Machine Learning, pp 273---297

Digital Library

[11]

Dreuw P, Forster J, Deselaers T, Ney H (2008) Efficient approximations to model-based joint tracking and recognition of continuous sign language. In: Proceedings of the IEEE International Conference on Automatic Face Gesture Recognition, Los Alamitos, pp 1---6

[12]

Dreuw P, Neidle C, Sclaroff VAS, Ney H (2008) Proc Int Conf Lang Resour Eval., Benchmark databases for video-based automatic sign language recognitionEuropean Language Resources Association, Marrakech

[13]

Dreuw P, Ney H (2008) Visual modeling and feature adaptation in sign language recognition. In: ITG Conference on Voice Communication (SprachKommunikation), pp 1---4

[14]

Dreuw P, Ney H, Martinez G, Crasborn O, Piater J, Moya JM, Wheatley M (2010) The signspeak project--bridging the gap between signers and speakers. In: Proceedings of the International Conference on Language Resources and Evaluation. Valletta, Malta

[15]

Dreuw P, Rybach D, Deselaers T, Zahedi M, Ney H (2007) Speech recognition techniques for a sign language recognition system. In: Proceedings of Interspeech, pp 2513---2516

[16]

Fagiani M, Principi E, Squartini S, Piazza F (2012) A new Italian sign language database. In: Zhang H, Hussain A, Liu D, Wang Z (eds) Advances in brain inspired cognitive systems, Lecture Notes in Computer Science, vol 7366. Springer, pp 164---173

[17]

Fagiani M, Principi E, Squartini S, Piazza F (2013) A new system for automatic recognition of italian sign language. In: Apolloni B, Bassis S, Esposito A, Morabito FC (eds) Neural nets and surroundings, smart innovation, systems and technologies, vol 19. Springer, Berlin, pp 69---79

[18]

Fang G, Gao W, Ma J (2001) Signer-independent sign language recognition based on SOFM/HMM. In: Proceedings of the IEEE ICCV workshop on recognition, analysis, and tracking of faces and gestures in real-time systems, pp 90---95

Digital Library

[19]

Gonzalez RC, Woods RE (2007) Digital image processing, 3rd edn. Prentice Hall, USA

Digital Library

[20]

Gweth Y, Plahl C, Ney H (2012) Enhanced continuous sign language recognition using PCA and neural network features. In: Proceedings of the computer vision and pattern recognition workshops, pp 55---60

[21]

Haasdonk B, Bahlmann C (2004) Pattern recognition. In: Rasmussen C, Bülthoff HH, Schölkopf B, Giese MA (eds) Learning with distance substitution kernels., Lecture notes in computer scienceSpringer, Berlin, pp 220---227

[22]

Hanke T (2004) HamNoSys-representing sign language data in language resources and language processing contexts. In: Proceedings of LREC, pp 1---6

[23]

Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8(2):179---187

[24]

Infantino I, Rizzo R, Gaglio S (2007) A framework for sign language sentence recognition by commonsense context. IEEE Trans Syst Man Cybern C Appl Rev 37(5):1034---1039

Digital Library

[25]

Kelly D, McDonald J, Markham C (2010) A person independent system for recognition of hand postures used in sign language. Pattern Recognit Lett 31(11):1359---1368

Digital Library

[26]

Kukharev G, Nowosielski A (2004) Visitor identification--elaborating real time face recognition system. In: Proceedings of the international conference on computer graphics, visualization and computer vision. UNION Agency, Plzen, pp 157---164

[27]

Maebatake M, Suzuki I, Nishida M, Horiuchi Y, Kuroiwa S (2008) Sign language recognition based on position and movement using multi-stream HMM. In: Proceedings of the international symposium on universal communication. Los Alamitos, pp 478---481

Digital Library

[28]

Quan Y (2010) Chinese sign language recognition based on video sequence appearance modeling. In: Proceedings of the IEEE Conference on Industrial Electronics and Applications, pp 1537---1542

[29]

Rabiner L, Juang BH (1993) Fundamentals of speech recognition. Prentice-Hall Inc., Upper Saddle River

Digital Library

[30]

Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(62):257---286

[31]

Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

Digital Library

[32]

Sandler W, Lillo-Martin D (2006) Sign language and linguistic universals. Cambridge University Press, Cambridge

[33]

Saon G, Chien JT (2012) Large-vocabulary continuous speech recognition systems: a look at some recent advances. IEEE Signal Process Mag 29(6):18---33

[34]

Serra J (1983) Image analysis and mathematical morphology. Academic Press Inc., Orlando

Digital Library

[35]

Starner T, Pentland A (1995) Real-time American sign language recognition from video using hidden Markov models. In: Proceedings of the international symposium on computer vision, pp 265---270

Digital Library

[36]

Starner T, Weaver J, Pentland A (1998) Real-time American sign language recognition using desk and wearable computer based video. IEEE Trans Pattern Anal Mach Intell 20(12):1371---1375

Digital Library

[37]

Theodorakis S, Katsamanis A, Maragos P (2009) Product-HMMs for automatic sign language recognition. In: Proceedings of the international conference on acoustics, speech and signal processing. Taipei, Taiwan, pp. 1601---1604

Digital Library

[38]

Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, Burlington

Digital Library

[39]

Vezhnevets V, Sazonov V, Andreeva A (2003) A survey on pixel-based skin color detection techniques. In: Proceedings of GraphiCon, pp 85---92

[40]

Young SJ, Evermann G, Gales MJF, Hain T, Kershaw D, Moore G, Odell J, Ollason D, Povey D, Valtchev V, Woodland PC (2006) The HTK Book, version 3.4. Cambridge University Press, Cambridge

[41]

Zahedi M, Keysers D, Deselaers T, Ney H (2005) Pattern recognition. In: Kropatsch W, Sablatnig R, Hanbury A (eds) Combination of tangent distance and an image distortion model for appearance-based sign language recognition., Lecture Notes in Computer ScienceSpringer, Berlin, pp 401---408

Digital Library

[42]

Zaki MM, Shaheen SI (2011) Sign language recognition using a combination of new vision based features. Pattern Recognit. Lett. 32(4):572---577

Digital Library

Cited By

Ferhat RChelali F(2024)Textural feature descriptors for a static and dynamic hand gesture recognition systemMultimedia Tools and Applications10.1007/s11042-023-15410-083:3(8165-8187)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15410-0
Khellas KSeghir R(2023)Alabib-65: A Realistic Dataset for Algerian Sign Language RecognitionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/359690922:6(1-23)Online publication date: 10-May-2023
https://dl.acm.org/doi/10.1145/3596909
Roy PKumar PKim B(2021)An Efficient Sign Language Recognition (SLR) System Using Camshift Tracker and Hidden Markov Model (HMM)SN Computer Science10.1007/s42979-021-00485-z2:2Online publication date: 6-Feb-2021
https://dl.acm.org/doi/10.1007/s42979-021-00485-z
Show More Cited By

Signer independent isolated Italian sign recognition based on hidden Markov models
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Machine learning
    1. Machine learning algorithms

Recommendations

Real-time American Sign Language recognition from video using hidden Markov models
ISCV '95: Proceedings of the International Symposium on Computer Vision

Hidden Markov models (HMMs) have been used prominently and successfully in speech recognition and, more recently, in handwriting recognition. Consequently, they seem ideal for visual recognition of complex, structured hand gestures such as are found in ...
A SRN/HMM System for Signer-Independent Continuous Sign Language Recognition
FGR '02: Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition

Sign language recognition is to provide an efficient and accurate mechanism to transcribe sign language into text or speech. State-of-the-art sign language recognition should be able to solve the signer-independent continuous problem for practical ...
Video-based signer-independent Arabic sign language recognition using hidden Markov models

Sign language in Arab World has been recently recognized and documented. There have been no serious attempts to develop a recognition system that can be used as a communication means between hearing-impaired and other people. This paper introduces the ...

Comments

Information & Contributors

Information

Published In

cover image Pattern Analysis & Applications

Pattern Analysis & Applications Volume 18, Issue 2

May 2015

242 pages

ISSN:1433-7541

EISSN:1433-755X

Issue’s Table of Contents

Copyright © Copyright © 2015 Springer-Verlag London.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 May 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ferhat RChelali F(2024)Textural feature descriptors for a static and dynamic hand gesture recognition systemMultimedia Tools and Applications10.1007/s11042-023-15410-083:3(8165-8187)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15410-0
Khellas KSeghir R(2023)Alabib-65: A Realistic Dataset for Algerian Sign Language RecognitionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/359690922:6(1-23)Online publication date: 10-May-2023
https://dl.acm.org/doi/10.1145/3596909
Roy PKumar PKim B(2021)An Efficient Sign Language Recognition (SLR) System Using Camshift Tracker and Hidden Markov Model (HMM)SN Computer Science10.1007/s42979-021-00485-z2:2Online publication date: 6-Feb-2021
https://dl.acm.org/doi/10.1007/s42979-021-00485-z
Rezende TAlmeida SGuimarães F(2021)Development and validation of a Brazilian sign language database for human gesture recognitionNeural Computing and Applications10.1007/s00521-021-05802-433:16(10449-10467)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1007/s00521-021-05802-4
Pezzuoli FCorona DCorradini M(2020)Recognition and Classification of Dynamic Hand Gestures by a Wearable Data-GloveSN Computer Science10.1007/s42979-020-00396-52:1Online publication date: 19-Nov-2020
https://dl.acm.org/doi/10.1007/s42979-020-00396-5
Anami BBhandage V(2019)A Comparative Study of Suitability of Certain Features in Classification of Bharatanatyam Mudra Images Using Artificial Neural NetworkNeural Processing Letters10.1007/s11063-018-9921-650:1(741-769)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s11063-018-9921-6
Anami BBhandage V(2019)Combined Hu moments, orientation knowledge, and grid intersections feature based identification of Bharatanatyam mudra imagesPattern Analysis & Applications10.1007/s10044-018-0715-222:4(1439-1454)Online publication date: 1-Nov-2019
https://dl.acm.org/doi/10.1007/s10044-018-0715-2
Anami BBhandage V(2018)A vertical-horizontal-intersections feature based method for identification of bharatanatyam double hand mudra imagesMultimedia Tools and Applications10.5555/3288443.328849177:23(31021-31040)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.5555/3288443.3288491
Anami BBhandage V(2018)A vertical-horizontal-intersections feature based method for identification of bharatanatyam double hand mudra imagesMultimedia Tools and Applications10.1007/s11042-018-6223-y77:23(31021-31040)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s11042-018-6223-y

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents