Word-Phrase Fusion Encoding Model for Natural Language Understanding in the Electric Power Field

Intelligent question-answering systems have been widely applied in the field of electric power. However, with the complexity of terms in the electric power field, the generic word-based encoding approach of natural language understanding model fails to identify those domain phrases even if they appear in the training samples. To improve the semantic accuracy of special content labeling, this paper proposes a words-phrase fusion encoding NLU model with the help of domain corpus. We pre-train a phrase-level Bert model in the electric power field which is involved during the model encoding step to accurately capture the semantics of domain terms and perceive the boundary of phrases. Additionally, continuous consistency loss of sequences is added to the model to reduce the to reduce the misclassification of individual words in the phrases. Experiment with real dataset in power QA system demonstrates that our model helps improve semantic parsing accuracy on both training and untrained items, alleviates the complete dependency of the phrase encoding model on word segmentation results.