RSDNet: A New Multiscale Rail Surface Defect Detection Model
Abstract
:1. Introduction
- (1)
- Proposed CDConv (Cascade Dilated Convolution), a module based on feature reuse. It was introduced into Backbone to realize multi-scale feature extraction without increasing the number of too many parameters.
- (2)
- Based on the idea of BiFPN (Bi-directional Feature Pyramids Network), change the feature fusion method of Head, add jump connections, and utilize more original feature information for feature fusion to improve the network’s ability to recognize defective edges.
- (3)
- Incorporate the EMA (Efficient Multi-Scale Attention) module into Head to enhance the feature extraction network’s attention to defect detail information, thus improving the detection accuracy of rail surface defects.
2. YOLOv8
3. RSDNet: YOLOv8n-CDConv-BiFPN-EMA
3.1. CDConv Module Proposed in This Study
3.2. Feature Extraction Method in This Study
3.3. Head Network with EMA
4. Experiments
4.1. Experimental Details
4.1.1. Dataset
4.1.2. Experimental Environment
4.1.3. Evaluation Index
4.2. Results and Discussion
4.2.1. Ablation Experiments
4.2.2. Improved Model Comparison Experiments
- Location and number of CDConv
- Location and number of the EMA
4.2.3. Performance Comparison Experiments
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Gong, W.; Akbar, M.F.; Jawad, G.N.; Mohamed, M.F.P.; Wahab, M.N.A. Nondestructive testing technologies for rail inspection: A review. Coatings 2022, 12, 1790. [Google Scholar] [CrossRef]
- Oh, K.; Yoo, M.; Jin, N.; Ko, J.; Seo, J.; Joo, H.; Ko, M. A Review of Deep Learning Applications for Railway Safety. Appl. Sci. 2022, 12, 10572. [Google Scholar] [CrossRef]
- Abbas, M.; Shafiee, M. Structural health monitoring (SHM) and determination of surface defects in large metallic structures using ultrasonic guided waves. Sensors 2018, 18, 3958. [Google Scholar] [CrossRef] [PubMed]
- Park, J.W.; Lee, T.G.; Back, I.C.; Park, S.J.; Seo, J.M.; Choi, W.J.; Kwon, S.G. Rail surface defect detection and analysis using multi-channel eddy current method based algorithm for defect evaluation. J. Nondestruct. Eval. 2021, 40, 83. [Google Scholar] [CrossRef]
- Alvarenga, T.A.; Carvalho, A.L.; Honorio, L.M.; Cerqueira, A.S.; Filho, L.M.; Nobrega, R.A. Detection and classification system for rail surface defects based on Eddy current. Sensors 2021, 21, 7937. [Google Scholar] [CrossRef]
- Hao, S.; Gao, S.; Ma, X.; An, B.; He, T. Anchor-free infrared pedestrian detection based on cross-scale feature fusion and hierarchical attention mechanism. Infrared Phys. Technol. 2023, 131, 104660. [Google Scholar] [CrossRef]
- Hao, S.; An, B.; Ma, X.; Sun, X.; He, T.; Sun, S. PKAMNet: A Transmission Line Insulator Parallel-Gap Fault Detection Network Based on Prior Knowledge Transfer and Attention Mechanism. IEEE Trans. Power Deliv. 2023, 38, 3387–3397. [Google Scholar] [CrossRef]
- Shim, J.; Koo, J.; Park, Y. A Methodology of Condition Monitoring System Utilizing Supervised and Semi-Supervised Learning in Railway. Sensors 2023, 23, 9075. [Google Scholar] [CrossRef]
- Fu, W.; He, Q.; Feng, Q.; Li, J.; Zheng, F.; Zhang, B. Recent advances in wayside railway wheel flat detection techniques: A review. Sensors 2023, 23, 3916. [Google Scholar] [CrossRef]
- Xie, Q.; Tao, G.; He, B.; Wen, Z. Rail corrugation detection using one-dimensional convolution neural network and data-driven method. Measurement 2022, 200, 111624. [Google Scholar] [CrossRef]
- Li, H.; Wang, Y.; Zeng, J.; Li, F.; Yang, Z.; Mei, G.; Ye, Y. Virtual point tracking method for online detection of relative wheel-rail displacement of railway vehicles. Reliab. Eng. Syst. Saf. 2024, 246, 110087. [Google Scholar] [CrossRef]
- Ye, Y.; Zhu, B.; Huang, P.; Peng, B. OORNet: A deep learning model for on-board condition monitoring and fault diagnosis of out-of-round wheels of high-speed trains. Measurement 2022, 199, 111268. [Google Scholar] [CrossRef]
- Xing, Z.; Zhang, Z.; Yao, X.; Qin, Y.; Jia, L. Rail wheel tread defect detection using improved YOLOv3. Measurement 2022, 203, 111959. [Google Scholar] [CrossRef]
- Yang, H.; Wang, Y.; Hu, J.; He, J.; Yao, Z.; Bi, Q. Deep learning and machine vision-based inspection of rail surface defects. IEEE Trans. Instrum. Meas. 2021, 71, 5005714. [Google Scholar] [CrossRef]
- Zhuang, L.; Qi, H.; Zhang, Z. The automatic rail surface multi-flaw identification based on a deep learning powered framework. IEEE Trans. Intell. Transp. Syst. 2021, 23, 12133–12143. [Google Scholar] [CrossRef]
- Kou, L. A review of research on detection and evaluation of the rail surface defects. Acta Polytech. Hung. 2022, 19, 167. [Google Scholar] [CrossRef]
- Gan, J.; Li, Q.; Wang, J.; Yu, H. A hierarchical extractor-based visual rail surface inspection system. IEEE Sens. J. 2017, 17, 7935–7944. [Google Scholar] [CrossRef]
- Zhang, H.; Jin, X.; Wu, Q.J.; Wang, Y.; He, Z.; Yang, Y. Automatic visual detection system of railway surface defects with curvature filter and improved Gaussian mixture model. IEEE Trans. Instrum. Meas. 2018, 67, 1593–1608. [Google Scholar] [CrossRef]
- Nieniewski, M. Morphological detection and extraction of rail surface defects. IEEE Trans. Instrum. Meas. 2020, 69, 6870–6879. [Google Scholar] [CrossRef]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 38, 142–158. [Google Scholar] [CrossRef]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, Montreal, QC, Canada, 7–12 December 2015. [Google Scholar]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
- Cheng, Y.; HongGui, D.; YuXin, F. Effects of faster region-based convolutional neural network on the detection efficiency of rail defects under machine vision. In Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 12–14 June 2020; pp. 1377–1380. [Google Scholar]
- Bai, T.; Yang, J.; Xu, G.; Yao, D. An optimized railway fastener detection method based on modified Faster R-CNN. Measurement 2021, 182, 109742. [Google Scholar] [CrossRef]
- Wang, H.; Li, M.; Wan, Z. Rail surface defect detection based on improved Mask R-CNN. Comput. Electr. Eng. 2022, 102, 108269. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLO9000: Better, faster, stronger. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 7263–7271. [Google Scholar]
- Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. Yolov4: Optimal speed and accuracy of object detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Li, C.; Li, L.; Jiang, H.; Weng, K.; Geng, Y.; Li, L.; Ke, Z.; Li, Q.; Cheng, M.; Nie, W. YOLOv6: A single-stage object detection framework for industrial applications. arXiv 2022, arXiv:2209.02976. [Google Scholar]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 7464–7475. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single shot multibox detector. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Proceedings, Part I 14. Springer: Cham, Switzerland, 2016; pp. 21–37. [Google Scholar]
- Wang, T.; Yang, F.; Tsui, K.-L. Real-time detection of railway track component via one-stage deep learning networks. Sensors 2020, 20, 4325. [Google Scholar] [CrossRef]
- Wang, M.; Li, K.; Zhu, X.; Zhao, Y. Detection of surface defects on railway tracks based on deep learning. IEEE Access 2022, 10, 126451–126465. [Google Scholar] [CrossRef]
- Zhang, C.; Xu, D.; Zhang, L.; Deng, W. Rail Surface Defect Detection Based on Image Enhancement and Improved YOLOX. Electronics 2023, 12, 2672. [Google Scholar] [CrossRef]
- Wang, Y.; Zhang, K.; Wang, L.; Wu, L. An Improved YOLOv8 Algorithm for Rail Surface Defect Detection. IEEE Access 2024, 12, 44984–44997. [Google Scholar] [CrossRef]
- Xin, F.; Jia, Q.; Yang, Y.; Pan, H.; Wang, Z. A high accuracy detection method for coal and gangue with S3DD-YOLOv8. Int. J. Coal Prep. Util. 2023, 1–19. [Google Scholar] [CrossRef]
- Yu, F.; Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv 2015, arXiv:1511.07122. [Google Scholar]
- Tan, M.; Pang, R.; Le, Q.V. Efficientdet: Scalable and efficient object detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 10781–10790. [Google Scholar]
- Ouyang, D.; He, S.; Zhang, G.; Luo, M.; Guo, H.; Zhan, J.; Huang, Z. Efficient multi-scale attention module with cross-spatial learning. In Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 4–10 June 2023; pp. 1–5. [Google Scholar]
Experimental Component | Version |
---|---|
CPU | Intel(R) Xeon(R) CPU E5-2640 v4 |
GPU | NVIDIA Quadro P2200 |
CUDA version | 10.2 |
Python version | 3.8 |
Pytorch version | 2.2.1 |
Model | Parameter (M) | P (%) | R (%) | [email protected] (%) |
---|---|---|---|---|
YOLOv8n | 30.06 | 92.3 | 86.7 | 90.8 |
YOLOv8n+CDConv | 30.11 | 96.0 | 88.6 | 94.1 |
YOLOv8n+BiFPN | 30.22 | 95.6 | 87.3 | 92.3 |
YOLOv8n+EMA | 30.17 | 95.8 | 87.9 | 93.7 |
YOLOv8n+CDConv+BiFPN | 30.28 | 96.2 | 88.5 | 94.8 |
YOLOv8n+BiFPN+EMA | 30.36 | 95.9 | 88.3 | 94.4 |
YOLOv8n+CDConv+BiFPN+EMA | 30.41 | 96.4 | 90.6 | 95.4 |
Model | Position | Numbers | P (%) | R (%) | [email protected] (%) |
---|---|---|---|---|---|
YOLOv8n+CDConv | Backbone | 1 | 95.4 | 88.9 | 94.1 |
YOLOv8n+CDConv | Backbone | 3 | 94.7 | 88.4 | 93.9 |
YOLOv8n+CDConv | Backbone | 5 | 95.2 | 88.6 | 91.4 |
YOLOv8n+CDConv | Head | 1 | 94.3 | 88.6 | 88.8 |
YOLOv8n+CDConv | Head | 2 | 94.4 | 88.5 | 89.4 |
Model | Position | Numbers | P (%) | R (%) | [email protected] (%) |
---|---|---|---|---|---|
YOLOv8n+EMA | Backbone | 1 | 94.2 | 86.5 | 90.9 |
YOLOv8n+EMA | Backbone | 3 | 95.4 | 87.1 | 91.9 |
YOLOv8n+EMA | Backbone | 4 | 95.2 | 87.6 | 92.5 |
YOLOv8n+EMA | Head | 3 | 95.8 | 87.9 | 93.7 |
YOLOv8n+EMA | Head | 4 | 94.6 | 87.5 | 93.2 |
Model | P (%) | R (%) | [email protected] (%) | Times/ms |
---|---|---|---|---|
Faster R-CNN | 71.5 | 72.9 | 72.3 | 156.5 |
SSD | 78.3 | 67.0 | 75.1 | 46.2 |
YOLOv5s | 86.7 | 80.8 | 89.8 | 22.3 |
YOLOv7-tiny | 85.7 | 83.3 | 88.0 | 19.4 |
YOLOv8n | 92.3 | 87.7 | 90.8 | 18.5 |
Ours | 96.4 | 90.6 | 95.4 | 19.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Du, J.; Zhang, R.; Gao, R.; Nan, L.; Bao, Y. RSDNet: A New Multiscale Rail Surface Defect Detection Model. Sensors 2024, 24, 3579. https://doi.org/10.3390/s24113579
Du J, Zhang R, Gao R, Nan L, Bao Y. RSDNet: A New Multiscale Rail Surface Defect Detection Model. Sensors. 2024; 24(11):3579. https://doi.org/10.3390/s24113579
Chicago/Turabian StyleDu, Jingyi, Ruibo Zhang, Rui Gao, Lei Nan, and Yifan Bao. 2024. "RSDNet: A New Multiscale Rail Surface Defect Detection Model" Sensors 24, no. 11: 3579. https://doi.org/10.3390/s24113579
APA StyleDu, J., Zhang, R., Gao, R., Nan, L., & Bao, Y. (2024). RSDNet: A New Multiscale Rail Surface Defect Detection Model. Sensors, 24(11), 3579. https://doi.org/10.3390/s24113579