Multi-Task Learning in Deep Neural Networks for Mandarin-English Code-Mixing Speech Recognition

Mengzhe CHEN; Jielin PAN; Qingwei ZHAO; Yonghong YAN

doi:10.1587/transinf.2016SLL0004

Special Section on Recent Advances in Machine Learning for Spoken Language Processing

Multi-Task Learning in Deep Neural Networks for Mandarin-English Code-Mixing Speech Recognition

Mengzhe CHEN, Jielin PAN, Qingwei ZHAO, Yonghong YAN

Author information

Keywords: multi-task learning, deep neural network, Mandarin-English code mixing, speech recognition

JOURNAL FREE ACCESS

2016 Volume E99.D Issue 10 Pages 2554-2557

DOI https://doi.org/10.1587/transinf.2016SLL0004

Details

Abstract

Multi-task learning in deep neural networks has been proven to be effective for acoustic modeling in speech recognition. In the paper, this technique is applied to Mandarin-English code-mixing recognition. For the primary task of the senone classification, three schemes of the auxiliary tasks are proposed to introduce the language information to networks and improve the prediction of language switching. On the real-world Mandarin-English test corpus in mobile voice search, the proposed schemes enhanced the recognition on both languages and reduced the relative overall error rates by 3.5%, 3.8% and 5.8% respectively.

Corresponding author

Register with J-STAGE for free!