Cited By
View all- Kaur RGinige JObst O(2023)AI-based ICD coding and classification approaches using discharge summariesExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118997213:PBOnline publication date: 1-Mar-2023
Towards enhanced hierarchical attention networks in ICD-9 tagging of clinical notes
Pages 146 - 150
Abstract
Text is an important element in document classification in many natural language applications. Natural language processing (NLP) is today's computational advancement that provides many significant modern uses of text documents such as efficient information retrieval. In this paper, we describe the theoretical framework of predicting ICD-9 codes through tagging of clinical notes using our improved framework in deep learning called EnHANs. This proposed model improvement covers combination of word and topic embedding, as well as adding character-level representation of a document in a hierarchical attention neural networks. This paper also present the use of sigmoid activation function in the last layer of the enhanced neural network in order to arrive with a multi-label, multi-class prediction of clinical notes with ICD-9 codes.
References
[1]
Lane, H., Howard, C., & Hapke, H. M. 2017. Natural Language Processing in Action (Manning Early Access Program (MEAP)). Manning Publications.
[2]
Ayyar, S., Don', O. B., & Iv, W. 2016. Tagging Patient Notes with ICD-9 Codes. In Proceedings of the 29th Conference on Neural Information Processing Systems (NIPS 2016).
[3]
Lenc, L., & Král, P. 2016. Deep Neural Networks for Czech Multi-label Document Classification. In Proceedings of the 17th International Conference on. Intelligent Text Processing and Computational Linguistics. arXiv preprint arXiv:1701.03849.
[4]
Perotte, A., Pivovarov, R., Natarajan, K., Weiskopf, N., Wood, F., & Elhadad, N. 2014. Diagnosis code assignment: models and evaluation metrics. Journal of the American Medical Informatics Association : JAMIA, 21(2), 231--7.
[5]
Zhang, M. L., & Zhou, Z. H. 2014. A review on multi-label learning algorithms. In Proceedings of IEEE Transactions on Knowledge and Data Engineering, 26(8), 1819--1837.
[6]
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. 2014. A Convolutional Neural Network for Modelling Sentences. In Proceedings of Association of Computational Linguistics (ACL), 655--665.
[7]
Lipton, Z. C., Kale, D. C., Elkan, C., & Wetzell, R. 2016. Learning to Diagnose with LSTM Recurrent Neural Networks. In Proceedings of the 2016 International Conference on Learning Representations (ICLR 2016), 1--18.
[8]
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. 2016. Hierarchical Attention Networks for Document Classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1480--1489.
[9]
Collet, F. 2017. Deep Learning with Python. Manning Publishing. October 28 2017
[10]
Liu, P., Qiu, X., & Huang, X. 2016. Recurrent Neural Network for Text Classification with Multi-Task Learning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI'16). New York, New York, USA --- July 09 -- 15, 2016. 2873--2879
[11]
Ren, Y., Zhang, Y., Zhang, M., & Ji, D. 2016. Improving Twitter Sentiment Classification Using Topic-Enriched Multi-Prototype Word Embeddings. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-2016), 3038--3044. February 12--17, 2016, Phoenix, Arizona USA
[12]
Liu, Y., Liu, Z., Chua, T.-S., & Sun, M. 2015. Topical Word Embeddings. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI'15), 2(C), 2418--2424.
[13]
Shi, B., Lam, W., Jameel, S., Schockaert, S., & Lai, K. P. 2017. Jointly Learning Word Embeddings and Latent Topics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information (375--384). ACM.
[14]
Luong, M.-T., Pham, H., & Manning, C. D. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the Empirical Methods on Natural Language Processing (EMNLP), (Lisbon, Portugal, 17--21 September 2015)
[15]
Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F., Sun, J., Schuetz, A., Sun, J. 2016. Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. In Proceedings of the 1st Machine Learning for Healthcare 2016, (JMLR Workshop Conf Proc. 2016 Aug; 56: 301--318) http://arxiv.org/abs/1511.05942
[16]
Esteban, C., Staeck, O., Yang, Y., & Tresp, V. 2016. Predicting Clinical Events by Combining Static and Dynamic Information Using Recurrent Neural Networks. In Proceedings of 2016 IEEE International Conference on Healthcare Informatics (ICHI2016) (Chicago, Illinois, USA on October 4--7, 2016) 93--101.
[17]
Bahdanau, D., Cho, K., & Bengio, Y. 2014. Neural Machine Translation By Jointly Learning To Align and Translate. In Proceedings of the 2015 International Conference on Learning Representations (ICLR2015), 1--15.
[18]
Chen, D., & Manning, C. D. 2014. A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 740--750.
[19]
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. In Proceedings of the, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-8), 103--111.
[20]
Chollet, F. (n.d.). Keras: The Python Deep Learning library. Retrieved from http://keras.io/
[21]
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. In Proceedings of the 2014 Conference on Neural Information Processing Systems (NIPS2014) - Deep Learning and Representation Learning Workshop. http://arxiv.org/abs/1412.3555
[22]
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. 2015. Deep learning applications and challenges in big data analytics. Journal of Big Data, 2(1), 1.
[23]
Yu, L., Hermann, K. M., Blunsom, P., & Pulman, S. 2014. Deep Learning for Answer Sentence Selection. In Proceedings of the 2014 Conference on Neural Information Processing Systems (NIPS2014
[24]
LeCun, Y., Yoshua, B., & Geoffrey, H. 2015. Deep learning. Nature 521, 436--444 (28 May 2015).
[25]
Sutskever, I., Vinyals, O., & Le, Q. V. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the 2014 Conference on Neural Information Processing Systems (NIPS2014) 3104--3112.
[26]
Bartunov, S., Kondrashkin, D., Osokin, A., & Vetrov, D. 2015. Breaking Sticks and Ambiguities with Adaptive Ski-pgram. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics - AISTATS 2015 (San Diego, California, USA - May 9 -- 12, 2015).
[27]
Kim, Y. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the Empirical Methods on Natural Language Processing (EMNLP 2014).
[28]
Liu, P., Qiu, X., & Huang, X. 2015. Learning context-sensitive word embeddings with neural tensor skip-gram model. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI2015) 1284--1290
[29]
Huang, Y., Wang, W., Wang, L., & Tan, T. 2013. Multi-task deep neural network for multi-label learning. In Proceedings of the 2013 IEEE International Conference on Image Processing (ICIP 2013) 2897--2900.
[30]
Rekabsaz, N., & Navid. 2016. Enhancing Information Retrieval with Adapted Word Embedding. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '16, 1169--1169.
[31]
Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. 2015. End-To-End Memory Networks. In Proceedings of the 2015 Conference on Neural Information Processing Systems (NIPS2015)
[32]
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L. H., Feng, M., Ghassemi, M., Mark, R. G. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035.
[33]
Zhang, Y. 2008. A hierarchical approach to encoding medical concepts for clinical notes. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Student Research Workshop, (June2008), 67--72.
Index Terms
- Towards enhanced hierarchical attention networks in ICD-9 tagging of clinical notes
Recommendations
ICD-9 Tagging of Clinical Notes Using Topical Word Embedding
ICIEB '18: Proceedings of the 2018 1st International Conference on Internet and e-BusinessMedical records, which contains text, has been dramatically increasing everyday. This means that there is a greater need of analyzing health information in a better way. And this can be done through document classification in natural language ...
Speculation detection for Chinese clinical notes
Display Omitted Chinese speculation detection is approached by a supervised sequence-labeling.Embedding features can enhance system accuracy, especially word embedding.Domain specific word segmentation is critical to Chinese speculation detection. ...
Part-of-Speech (POS) Tagging Using Deep Learning-Based Approaches on the Designed Khasi POS Corpus
Part-of-speech (POS) tagging is one of the research challenging fields in natural language processing (NLP). It requires good knowledge of a particular language with large amounts of data or corpora for feature engineering, which can lead to achieving a ...
Comments
Information & Contributors
Information
Published In
November 2017
545 pages
ISBN:9781450353656
DOI:10.1145/3162957
- Conference Chairs:
- Jalel Ben-Othman,
- Feng Gang,
- Program Chairs:
- Jain-Shing Liu,
- Masayuki Arai
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 November 2017
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ICCIP 2017
ICCIP 2017: 2017 the 3rd International Conference on Communication and Information Processing
November 24 - 26, 2017
Tokyo, Japan
Acceptance Rates
Overall Acceptance Rate 61 of 301 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 144Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Jan 2025
Other Metrics
Citations
Cited By
View all- Kaur RGinige JObst O(2023)AI-based ICD coding and classification approaches using discharge summariesExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118997213:PBOnline publication date: 1-Mar-2023
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in