skip to main content
research-article

Uncertainty-Aware Semi-Supervised Method Using Large Unlabeled and Limited Labeled COVID-19 Data

Published: 15 November 2021 Publication History

Abstract

The new coronavirus has caused more than one million deaths and continues to spread rapidly. This virus targets the lungs, causing respiratory distress which can be mild or severe. The X-ray or computed tomography (CT) images of lungs can reveal whether the patient is infected with COVID-19 or not. Many researchers are trying to improve COVID-19 detection using artificial intelligence. Our motivation is to develop an automatic method that can cope with scenarios in which preparing labeled data is time consuming or expensive. In this article, we propose a Semi-supervised Classification using Limited Labeled Data (SCLLD) relying on Sobel edge detection and Generative Adversarial Networks (GANs) to automate the COVID-19 diagnosis. The GAN discriminator output is a probabilistic value which is used for classification in this work. The proposed system is trained using 10,000 CT scans collected from Omid Hospital, whereas a public dataset is also used for validating our system. The proposed method is compared with other state-of-the-art supervised methods such as Gaussian processes. To the best of our knowledge, this is the first time a semi-supervised method for COVID-19 detection is presented. Our system is capable of learning from a mixture of limited labeled and unlabeled data where supervised learners fail due to a lack of sufficient amount of labeled data. Thus, our semi-supervised training method significantly outperforms the supervised training of Convolutional Neural Network (CNN) when labeled training data is scarce. The 95% confidence intervals for our method in terms of accuracy, sensitivity, and specificity are 99.56 ± 0.20%, 99.88 ± 0.24%, and 99.40 ± 0.18%, respectively, whereas intervals for the CNN (trained supervised) are 68.34 ± 4.11%, 91.2 ± 6.15%, and 46.40 ± 5.21%.

References

[5]
Tao Ai, Zhenlu Yang, Hongyan Hou, Chenao Zhan, Chong Chen, Wenzhi Lv, Qian Tao, Ziyong Sun, and Liming Xia. 2020. Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases. Radiology 296, 2 (2020), E32–E40.
[6]
Roohallah Alizadehsani, Zahra Alizadeh Sani, Mohaddeseh Behjati, Zahra Roshanzamir, Sadiq Hussain, Niloofar Abedini, Fereshteh Hasanzadeh, Abbas Khosravi, Afshin Shoeibi, Mohamad Roshanzamir, et al. 2020. Risk factors prediction, clinical outcomes, and mortality in COVID-19 patients. Journal of Medical Virology 93 (2020), 2307–2320.
[7]
Roohallah Alizadehsani, Abbas Khosravi, Mohamad Roshanzamir, Moloud Abdar, Nizal Sarrafzadegan, Davood Shafie, Fahime Khozeimeh, Afshin Shoeibi, Saeid Nahavandi, Maryam Panahiazar, et al. 2020. Coronary artery disease detection using artificial intelligence techniques: A survey of trends, geographical differences and diagnostic features 1991–2020. Computers in Biology and Medicine 128 (2020), 104095.
[8]
Md Zahangir Alom, M. M. Rahman, Mst Shamima Nasrin, Tarek M. Taha, and Vijayan K. Asari. 2020. COVID_MTNet: Covid-19 detection with multi-task deep learning approaches. arXiv:2004.03747. https://arxiv.org/abs/2004.03747.
[9]
Marios Anthimopoulos, Stergios Christodoulidis, Lukas Ebner, Andreas Christe, and Stavroula Mougiakakou. 2016. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Transactions on Medical Imaging 35, 5 (2016), 1207–1216.
[10]
Ioannis D. Apostolopoulos and Tzani A. Mpesiana. 2020. Covid-19: Automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine 43, 2 (2020), 635–640.
[11]
Ali Abbasian Ardakani, U. Rajendra Acharya, Sina Habibollahi, and Afshin Mohammadi. 2021. COVIDiag: A clinical CAD system to diagnose COVID-19 pneumonia based on CT findings. European Radiology 31, 1 (2021), 121–130.
[12]
Ali Abbasian Ardakani, Alireza Rajabzadeh Kanafi, U. Rajendra Acharya, Nazanin Khadem, and Afshin Mohammadi. 2020. Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks. Computers in Biology and Medicine 121 (2020), 103795.
[13]
Hamzeh Asgharnezhad, Afshar Shamsi, Roohallah Alizadehsani, Abbas Khosravi, Saeid Nahavandi, Zahra Alizadeh Sani, and Dipti Srinivasan. 2020. Objective evaluation of deep uncertainty predictions for COVID-19 detection. arXiv:2012.11840. https://arxiv.org/abs/2012.11840.
[14]
Mikhail Belyaev, Evgeny Burnaev, and Yermek Kapushev. 2014. Exact inference for Gaussian process regression in case of big data with the Cartesian product structure. arXiv:1403.6573. https://arxiv.org/abs/1403.6573.
[15]
Nanshan Chen, Min Zhou, Xuan Dong, Jieming Qu, Fengyun Gong, Yang Han, Yang Qiu, Jingli Wang, Ying Liu, Yuan Wei, et al. 2020. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: A descriptive study. The Lancet 395, 10223 (2020), 507–513.
[16]
Debabrata Dansana, Raghvendra Kumar, Aishik Bhattacharjee, D. Jude Hemanth, Deepak Gupta, Ashish Khanna, and Oscar Castillo. 2020. Early diagnosis of COVID-19-affected patients based on X-ray and computed tomography images using deep learning algorithm. Soft Computing (2020), 1–9.
[17]
Adrien Depeursinge, Anne S. Chin, Ann N. Leung, Donato Terrone, Michael Bristow, Glenn Rosen, and Daniel L. Rubin. 2015. Automated classification of usual interstitial pneumonia using regional volumetric texture analysis in high-resolution CT. Investigative Radiology 50, 4 (2015), 261.
[18]
Rahimeh Eskandarian, Zahra Alizadeh Sani, Mohaddeseh Behjati, Mehrdad Zahmatkesh, Azadeh Haddadi, Kourosh Kakhi, Mohamad Roshanzamir, Afshin Shoeibi, Roohallah Alizadehsani, Sadiq Hussain, et al. 2021. Identification of clinical features associated with mortality in COVID-19 patients. medRxiv.https://doi.org/10.1101/2021.04.19.21255715
[19]
Yicheng Fang, Huangqi Zhang, Jicheng Xie, Minjie Lin, Lingjun Ying, Peipei Pang, and Wenbin Ji. 2020. Sensitivity of chest CT for COVID-19: Comparison to RT-PCR. Radiology 296, 2 (2020), E115–E117.
[20]
Zoubin Ghahramani. 2003. Unsupervised learning. In Summer School on Machine Learning. Springer, 72–112.
[21]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. arXiv:1406.2661. https://arxiv.org/abs/1406.2661.
[22]
Juan M. Górriz, Javier Ramírez, Andrés Ortíz, Francisco J. Martínez-Murcia, Fermin Segovia, John Suckling, Matthew Leming, Yu-Dong Zhang, Jose Ramón Álvarez-Sánchez, Guido Bologna, et al. 2020. Artificial intelligence within the interplay between natural and artificial computation: Advances in data science, trends and applications. Neurocomputing 410 (2020), 237–270.
[23]
Ophir Gozes, Maayan Frid-Adar, Hayit Greenspan, Patrick D. Browning, Huangqi Zhang, Wenbin Ji, Adam Bernheim, and Eliot Siegel. 2020. Rapid AI development cycle for the coronavirus (COVID-19) pandemic: Initial results for automated detection & patient monitoring using deep learning CT image analysis. arXiv:2003.05037. https://arxiv.org/abs/2003.05037.
[24]
Ezz El-Din Hemdan, Marwa A. Shouman, and Mohamed Esmail Karar. 2020. COVIDX-Net: A framework of deep learning classifiers to diagnose COVID-19 in x-ray images. arXiv:2003.11055. https://arxiv.org/abs/2003.11055.
[25]
Taocheng Hu and Jinhui Yu. 2015. Generalized entropy based semi-supervised learning. In 2015 IEEE/ACIS 14th International Conference on Computer and Information Science (ICIS’15). IEEE, 259–263.
[26]
Aayush Jaiswal, Neha Gianchandani, Dilbag Singh, Vijay Kumar, and Manjit Kaur. 2020. Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning. Journal of Biomolecular Structure and Dynamics 39 (2020), 5682-5689.
[27]
Tahereh Javaheri, Morteza Homayounfar, Zohreh Amoozgar, Reza Reiazi, Fatemeh Homayounieh, Engy Abbas, Azadeh Laali, Amir Reza Radmard, Mohammad Hadi Gharib, Seyed Ali Javad Mousavi, et al. 2021. CovidCTNet: An open-source deep learning approach to identify covid-19 using small cohort of CT images. npj Digit. Med. 4, 29 (2021). https://doi.org/10.1038/s41746-021-00399-3
[28]
Marjane Khodatars, Afshin Shoeibi, Navid Ghassemi, Mahboobeh Jafari, Ali Khadem, Delaram Sadeghi, Parisa Moridian, Sadiq Hussain, Roohallah Alizadehsani, Assef Zare, et al. 2020. Deep learning for neuroimaging-based diagnosis and rehabilitation of autism spectrum disorder: A review. arXiv:2007.01285. https://arxiv.org/abs/2007.01285.
[29]
Fahime Khozeimeh, Danial Sharifrazi, Navid Hoseini Izadi, Javad Hassannataj Joloudari, Afshin Shoeibi, Roohallah Alizadehsani, Juan M Gorriz, Sadiq Hussain, Zahra Alizadeh Sani, Hossein Moosaei, et al. 2021. CNN AE: Convolution neural network combined with autoencoder approach to detect survival chance of COVID 19 patients. Scientific Reports 11, 1 (2021), 1–18.
[30]
Hoon Ko, Heewon Chung, Wu Seong Kang, Kyung Won Kim, Youngbin Shin, Seung Ji Kang, Jae Hoon Lee, Young Jun Kim, Nan Yeol Kim, Hyunseok Jung, et al. 2020. COVID-19 pneumonia diagnosis using a simple 2D deep learning framework with a single chest CT image: Model development and validation. Journal of Medical Internet Research 22, 6 (2020), e19569.
[31]
Rajesh Kumar, Abdullah Aman Khan, Sinmin Zhang, WenYong Wang, Yousif Abuidris, Waqas Amin, and Jay Kumar. 2020. Blockchain-federated-learning and deep learning models for covid-19 detection using CT imaging. IEEE Sensors Journal 21, 14 (2021), 16301–16314. DOI:https://doi.org/10.1109/JSEN.2021.3076767
[32]
Lin Li, Lixin Qin, Zeguo Xu, Youbing Yin, Xin Wang, Bin Kong, Junjie Bai, Yi Lu, Zhenghan Fang, Qi Song, et al. 2020. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology 296 (2020), E65–E71.
[33]
Longze Li and Aleksandar Vakanski. 2018. Generative adversarial networks for generation and classification of physical rehabilitation movement episodes. International Journal of Machine Learning and Computing 8, 5 (2018), 428.
[34]
Mu Li, Tong Zhang, Yuqiang Chen, and Alexander J. Smola. 2014. Efficient mini-batch training for stochastic optimization. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 661–670.
[35]
Xiuming Liu, Dave Zachariah, Johan Wågberg, and Thomas B. Schön. 2018. Reliable semi-supervised learning when labels are missing at random. arXiv:1811.10947. https://arxiv.org/abs/1811.10947.
[36]
Halgurd S. Maghdid, Aras T. Asaad, Kayhan Zrar Ghafoor, Ali Safaa Sadiq, Seyedali Mirjalili, and Muhammad Khurram Khan. 2021. Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms. In Multimodal Image Exploitation and Learning 2021, Vol. 11734. International Society for Optics and Photonics, 117340E.
[37]
Ali Narin, Ceren Kaya, and Ziynet Pamuk. 2021. Automatic detection of coronavirus disease (COVID-19) using x-ray images and deep convolutional neural networks. Pattern Anal Applic 24 (2021), 1207–1220. https://doi.org/10.1007/s10044-021-00984-y
[38]
Qianqian Ni, Zhi Yuan Sun, Li Qi, Wen Chen, Yi Yang, Li Wang, Xinyuan Zhang, Liu Yang, Yi Fang, Zijian Xing, et al. 2020. A deep learning approach to characterize 2019 coronavirus disease (COVID-19) pneumonia in chest CT images. European Radiology 30, 12 (2020), 6517–6527.
[39]
Emilio Soria Olivas, Jos David Mart Guerrero, Marcelino Martinez-Sober, Jose Rafael Magdalena-Benedito, L Serrano, et al. 2009. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques. IGI Global.
[40]
Harsh Panwar, P. K. Gupta, Mohammad Khubeb Siddiqui, Ruben Morales-Menendez, Prakhar Bhardwaj, and Vaishnavi Singh. 2020. A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images. Chaos, Solitons & Fractals 140 (2020), 110190.
[41]
Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434. https://arxiv.org/abs/1511.06434.
[42]
Carl Edward Rasmussen and C. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press. Cambridge, MA.
[43]
Abdolkarim Saeedi, Maryam Saeedi, and Arash Maghsoudi. 2020. A novel and reliable deep learning web-based tool to detect COVID-19 infection form chest CT-scan. arXiv:2006.14419. https://arxiv.org/abs/2006.14419.
[44]
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training GANs. Advances in Neural Information Processing Systems 29 (2016), 2234–2242.
[45]
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618–626.
[46]
Prabira Kumar Sethy, Santi Kumari Behera, Pradyumna Kumar Ratha, and Preesat Biswas. 2020. Detection of coronavirus disease (COVID-19) based on deep features and support vector machine. DOI:https://doi.org/10.33889/IJMEMS.2020.5.4.052
[47]
Vruddhi Shah, Rinkal Keniya, Akanksha Shridharani, Manav Punjabi, Jainam Shah, and Ninad Mehendale. 2021. Diagnosis of COVID-19 using CT scan images and deep learning techniques. Emergency Radiology (2021), 1–9. DOI:https://doi.org/10.1007/s10140-020-01886-y
[48]
Danial Sharifrazi, Roohallah Alizadehsani, Javad Hassannataj Joloudari, Shahab Shamshirband, Sadiq Hussain, Zahra Alizadeh Sani, Fereshteh Hasanzadeh, Afshin Shoaibi, Abdollah Dehzangi, and Hamid Alinejad-Rokny. 2020. CNN-KCL: Automatic myocarditis diagnosis using convolutional neural network combined with K-means clustering. DOI: https://doi.org/10.20944/preprints202007.0650.v1
[49]
Danial Sharifrazi, Roohallah Alizadehsani, Mohamad Roshanzamir, Javad Hassannataj Joloudari, Afshin Shoeibi, Mahboobeh Jafari, Sadiq Hussain, Zahra Alizadeh Sani, Fereshteh Hasanzadeh, Fahime Khozeimeh, et al. 2021. Fusion of convolution neural network, support vector machine and Sobel filter for accurate detection of COVID-19 patients using X-ray images. Biomedical Signal Processing and Control 68 (2021), 102622.
[50]
Weiya Shi, Xueqing Peng, Tiefu Liu, Zenghui Cheng, Hongzhou Lu, Shuyi Yang, Jiulong Zhang, Mei Wang, Yaozong Gao, Yuxin Shi, et al. 2021. A deep learning-based quantitative computed tomography model for predicting the severity of COVID-19: A retrospective study of 196 patients. Annals of Translational Medicine 9, 3 (2021).
[51]
Afshin Shoeibi, Navid Ghassemi, Marjane Khodatars, Mahboobeh Jafari, Sadiq Hussain, Roohallah Alizadehsani, Parisa Moridian, Abbas Khosravi, Hossein Hosseini-Nejad, Modjtaba Rouhani, and A. Khosravi. 2021. Epileptic seizure detection using deep learning techniques: A review. International Journal of Environmental Research and Public Health 18, 11 (2021), 5780.
[52]
Afshin Shoeibi, Marjane Khodatars, Roohallah Alizadehsani, Navid Ghassemi, Mahboobeh Jafari, Parisa Moridian, Ali Khadem, Delaram Sadeghi, Sadiq Hussain, Assef Zare, et al. 2020. Automated detection and forecasting of COVID-19 using deep learning techniques: A review. arXiv:2007.10785. https://arxiv.org/abs/2007.10785.
[53]
Ying Song, Shuangjia Zheng, Liang Li, Xiang Zhang, Xiaodong Zhang, Ziwang Huang, Jianwen Chen, Ruixuan Wang, Huiying Zhao, Yunfei Zha, et al. 2021. Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. IEEE/ACM Transactions on Computational Biology and Bioinformatics (2021).
[54]
Sijia Tian, Nan Hu, Jing Lou, Kun Chen, Xuqin Kang, Zhenjun Xiang, Hui Chen, Dali Wang, Ning Liu, Dong Liu, et al. 2020. Characteristics of COVID-19 infection in Beijing. Journal of Infection 80, 4 (2020), 401–406.
[55]
Yu Tingting, Wang Junqian, Wu Lintai, and Xu Yong. 2019. Three-stage network for age estimation. CAAI Transactions on Intelligence Technology 4, 2 (2019), 122–126.
[56]
Linda Wang, Zhong Qiu Lin, and Alexander Wong. 2020. OVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest x-ray images. Scientific Reports 10, 1 (2020), 1–12.
[57]
Shuai Wang, Bo Kang, Jinlu Ma, Xianjun Zeng, Mingming Xiao, Jia Guo, Mengjiao Cai, Jingyi Yang, Yaodong Li, Xiangfei Meng, et al. 2021. A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19). European Radiology (2021), 1–9.
[58]
Shui-Hua Wang, Vishnu Varthanan Govindaraj, Juan Manuel Górriz, Xin Zhang, and Yu-Dong Zhang. 2021. COVID-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network. Information Fusion 67 (2021), 208–229.
[59]
Xiaowei Xu, Xiangao Jiang, Chunlian Ma, Peng Du, Xukun Li, Shuangzhi Lv, Liang Yu, Qin Ni, Yanfei Chen, Junwei Su, et al. 2020. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering 6, 10 (2020), 1122–1129.
[60]
Rabab Yasin and Walaa Gouda. 2020. Chest X-ray findings monitoring COVID-19 disease course and severity. Egyptian Journal of Radiology and Nuclear Medicine 51, 1 (2020), 1–18.
[61]
Haider Hashim Zalzala. 2020. Diagnosis of COVID-19: Facts and challenges. New Microbes and New Infections (2020), 100761.
[62]
Jianpeng Zhang, Yutong Xie, Yi Li, Chunhua Shen, and Yong Xia. 2020. COVID-19 screening on chest x-ray images using deep learning based anomaly detection. https://europepmc.org/article/ppr/ppr346108.
[63]
Yu-Dong Zhang, Suresh Chandra Satapathy, Li-Yao Zhu, Juan Manuel Górriz, and Shui-Hua Wang. 2020. A seven-layer convolutional neural network for chest CT based COVID-19 diagnosis using stochastic pooling. IEEE Sensors Journal (2020).
[64]
Jinyu Zhao, Yichen Zhang, Xuehai He, and Pengtao Xie. 2020. CVID-CT-Dataset: A CT scan dataset about COVID-19. arXiv:2003.13865. https://arxiv.org/abs/2003.13865.

Cited By

View all

Index Terms

  1. Uncertainty-Aware Semi-Supervised Method Using Large Unlabeled and Limited Labeled COVID-19 Data

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
    October 2021
    324 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/3492435
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 November 2021
    Accepted: 01 April 2021
    Revised: 01 March 2021
    Received: 01 December 2020
    Published in TOMM Volume 17, Issue 3s

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Semi-supervised learning
    2. generative adversarial networks
    3. COVID-19
    4. supervised learning
    5. deep learning

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • MINECO/FEDER

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)92
    • Downloads (Last 6 weeks)11
    Reflects downloads up to 22 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media