research-article

Vehicle detection and traffic density estimation using ensemble of deep learning models

Authors:

Priyanka ChawlaAuthors Info & Claims

Multimedia Tools and Applications, Volume 82, Issue 7

Pages 10397 - 10419

https://doi.org/10.1007/s11042-022-13659-5

Published: 27 August 2022 Publication History

Abstract

Traffic density estimation can be used for controlling traffic light signals to provide effective traffic management. It can be done in two steps: vehicle recognition and counting. Deep learning (DL) technologies are being explored more and more as CNN grows in popularity. In this study, initially, data was collected from various open-source libraries that is FLIR, KITTI, and MB7500. Vehicles in the images are labelled in six different classes. To deal with an imbalanced dataset, data augmentation techniques were applied. Then, a model based on an ensemble of the faster region-based convolutional neural networks (Faster R-CNN) and Single-shot detector (SSD) were trained on finally processed datasets. The results of the proposed model were compared with base estimators of the FLIR dataset (Thermal and RGB images separately), MB7500, and KITTI dataset. Experimental results depict that the highest mAP obtained was 94% by the proposed Ensemble on FLIR thermal dataset which was 34% better than SSD and 6% from the Faster R-CNN model. Overall, the proposed ensemble achieves better and more promising results as compared to base estimators. Experimental results also show that detection with thermal images was better than visible images. In addition, three algorithms were compared for estimated density and the proposed model shows significant potential for traffic density estimation.

References

[1]

AlZu’bi S, Jararweh Y (2020) Data fusion in autonomous vehicles research, literature tracing from imaginary idea to smart surrounding community. In: Fifth International Conference on Fog and Mobile Edge Computing (FMEC), pp 306–311.

[2]

AlZu’bi S, Al-Qatawneh S, Alsmirat M (2018) Transferable HMM trained matrices for accelerating statistical segmentation time. In: Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp 172–176.

[3]

AlZu’bi S, Hawashin B, Mujahed M, Jararweh Y, and Gupta BB An efficient employment of internet of multimedia things in smart and future agriculture Multimed Tools Appl 2019 78 20 29581-29605

[4]

AlZu’bi S, Shehab M, Al-Ayyoub M, Jararweh Y, and Gupta B Parallel implementation for 3D medical volume fuzzy segmentation Pattern Recogn Lett 2020 130 312-318

[5]

Al-Zu’bi S, Hawashin B, Mughaid A, and Baker T Efficient 3D medical image segmentation algorithm over a secured multimedia network Multimed Tools Appl 2020 80 11 16887-16905

[6]

Aqel D, Al-Zubi S, Mughaid A, Jararweh Y (2021) Extreme learning machine for plant diseases classification: a sustainable approach for smart agriculture. Clust Comput 25(3:2007–2020.

[7]

Arrospide J and Salgado L Log-Gabor filters for image-based vehicle verification IEEE Trans Image Process 2013 22 6 2286-2295

[8]

Azimi SM (2018) ShuffleDet: real-time vehicle detection network in on-board embedded UAV imagery. In: European Conference on Computer Vision ECCV. Computer vision – ECCV 2018 workshops Lect Notes Comput Sci 2019:88–99.

[9]

Biswas D, Su H, Wang C, Stevanovic A, and Wang W An automatic traffic density estimation using Single Shot Detection (SSD) and MobileNet-SSD Phys Chem Earth, Parts 2019 110 176-184

[10]

Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-nms – improving object detection with one line of code. In: Proceedings of the IEEE international conference on computer vision, pp 5561–5569

[11]

Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162

[12]

Chan YM, Huang SS, Fu LC, Hsiao PY, and Lo MF Vehicle detection and tracking under various lighting IET Intell Transp Syst 2012 6 1-8

[13]

Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. Adv Neural Inf Process Syst 30

[14]

Chowdhury PN, Shivakumara P, Pal U, Lu T, and Blumenstein M A new augmentation-based method for text detection in night and day license plate images Multimed Tools Appl 2020 79 43–44 33303-33330

[15]

Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. Adv Neural Inf Process Syst 29

[16]

Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05). Ieee, vol 1, pp 886–893

[17]

Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6569–6578

[18]

Fidler S, Mottaghi R, Yuille A, Urtasun R (2013) Bottom-up segmentation for top-down detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3294–3301

[19]

Gao Y, Lee HJ (2015) Moving car detection and model recognition based on deep learning. Adv Sci Technol Lett 90 57–61

[20]

Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

[21]

Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

[22]

Harsha SS and Anne KR Gaussian mixture model and deep neural network based vehicle detection and classification. (IJACSA) Int J Adv Comput Sci Appl 2016 7 9 17-25

[23]

He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

[24]

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

[25]

He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, Cham, pp 630–645

[26]

Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

[27]

Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

[28]

Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

[29]

Jagannathan P, Rajkumar S, Frnda J, Divakarachari PB, and Subramani P Moving vehicle detection and classification using Gaussian mixture model and ensemble deep learning technique Wirel Commun Mob Comput 2021 2021 1-15

[30]

Jamiya SS and Esther Rani P LittleYOLO-SPP: A delicate real-time vehicle detection algorithm Optik 2021 225 165818

[31]

John V, Mita S, Zheng L, Qi B (2015) Pedestrian detection in thermal images using adaptive fuzzy C-means clustering and convolutional neural networks. In: 2015 14th IAPR International Conference on Machine Vision Applications (MVA). IEEE, pp 246–249

[32]

Kembhavi A, Harwood D, and Davis LS Vehicle detection using partial least squares IEEE Trans Pattern Anal Mach Intell 2011 33 6 1250-1265 Jun.

[33]

Keserwani P, Dhankhar A, Saini R, and Roy PP Quadbox: quadrilateral bounding box based scene text detection using vector regression IEEE Access 2021 9 36802-36818

[34]

Kleban J, Xie X, Ma W-Y (2008) Spatial pyramid mining for logo detection in natural scenes. In: 2008 IEEE International Conference on Multimedia and Expo. IEEE, pp 1077–1080

[35]

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25

[36]

Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV), pp 734–750

[37]

Lee JT, Chung Y (2017) Deep learning-based vehicle classification using an ensemble of local expert and global networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 47–52.

[38]

Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: Proceedings international conference on image processing. IEEE, vol 1, pp I–I

[39]

Lin T-Y, Dollàr P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection, in: CVPR

[40]

Lin T-Y, Goyal P, Girshick R, He K, Dollàr P (2017) Focal loss for dense object detection, in: ICCV

[41]

Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector, in: ECCV

[42]

Liu W, Luo Z, and Li S Improving deep ensemble vehicle classification by using selected adversarial samples Knowl-Based Syst 2018 160 167-175

[43]

Lowe DG (1999) Object recognition from local scale-invariant features, in: ICCV

[44]

Nam Y, Nam Y-C (2018) Vehicle classification based on images from visible light and thermal cameras. EURASIP J Image Vid Process 2018.

[45]

Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: European conference on computer vision. Springer, Cham, pp 483–499

[46]

Oliveira DC and Wehrmeister MA Using deep learning and low cost RGB and thermal cameras to detect pedestrians in aerial images captured by multirotor UAV Sensors (Basel) 2018 18 7 2244

[47]

Ozkurt C and Camci F Automatic traffic density estimation and vehicle classification for traffic surveillance systems using neural networks Math Comput Appl 2009 14 3 187-196 Dec.

[48]

Ranjeeth Kumar C and Anuradha R Feature selection and classification methods for vehicle tracking and detection J Ambient Intell Humaniz Comput 2020 12 3 4269-4279

[49]

Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger, in: CVPR

[50]

Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection, in: CVPR

[51]

Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks, in: NeurIPS

[52]

Rezaee H, Aghagolzadeh A, Seyedarabi MH, Zu’bi SA (2011) “Tracking and occlusion handling in multi-sensor networks by particle filter,” in IEEE GCC Conference and Exhibition (GCC), pp. 397–400,.

[53]

Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, in: arXiv preprint arXiv:1409.1556

[54]

Suryanto, Kim D-H, Kim H-K, and Ko S-J Spatial color histogram based center voting method for subsequent object tracking and segmentation Image Vis Comput 2011 29 12 850-860

[55]

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions, in: CVPR

[56]

Uijlings JR, Van De Sande KE, Gevers T, and Smeulders AW Selective search for object recognition IJCV 2013 104 154-171

[57]

Wu X, Sahoo D, Hoi SCH (2019) Recent advances in deep learning for object detection. Elsevier

[58]

Xie S, Girshick R, Dollàr P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks, in: CVPR

[59]

Yang Y, Song H, Sun S, Zhang W, Chen Y, Rakal L, and Fang Y A fast and effective video vehicle detection method leveraging feature fusion and proposal temporal link J Real-Time Image Proc 2021 18 4 1261-1274

[60]

Zhang B Reliable classification of vehicle types based on Cascade classifier ensembles IEEE Trans Intell Transp Syst 2013 14 1 322-332

[61]

Zhang S, Wen L, Bian X, Lei Z, Li SZ (2018) Single-shot refinement neural network for object detection, in: CVPR

[62]

Zhou Y, Nejati H, Do TT, Cheung NM, and Cheah L Image-based vehicle analysis using deep neural network: a systematic study. IEEE international conference on digital signal processing (DSP) 2016 Beijing IEEE

[63]

Zhou X, Wang D, Krähenbühl P (2019) Objects as points, in: arXiv preprint arXiv:1904.07850

[64]

Zhu J, Li X, Jin P, Xu Q, Sun Z, and Song X MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance Sensors 2020 21 1 27

Cited By

Sharma DRoul R(2024)Intelligent Traffic Congestion Classification FrameworkSN Computer Science10.1007/s42979-024-03564-z6:1Online publication date: 27-Dec-2024
https://dl.acm.org/doi/10.1007/s42979-024-03564-z
Said YAlassaf YAlsariera YGhodhbani RSaidani TBen Rhaiem OMakhdoum M(2024)Traffic flow management by detecting and estimating vehicles density based on object detection modelNeural Computing and Applications10.1007/s00521-024-09753-436:19(11495-11505)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09753-4

Index Terms

Vehicle detection and traffic density estimation using ensemble of deep learning models
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

EnsembleNet: a hybrid approach for vehicle detection and estimation of traffic density based on faster R-CNN and YOLO models
Abstract
Due to static traffic management regulations on roadways, traffic flow may become congested as it has been growing on roads. Estimating traffic density impacts intelligent transportation systems as it helps build efficient traffic management. ...
Performance optimisation of deep learning models using majority voting algorithm for brain tumour classification
Abstract Background
Although biopsy is the gold standard for tumour grading, being invasive, this procedure also proves fatal to the brain. Thus, non-invasive methods for brain tumour grading are urgently needed. Here, a magnetic resonance imaging (MRI)-...
Highlights
- A computer-aided diagnostic tool is proposed for automated brain tumour classification using MRI.
- Machine and deep learning-based ensemble algorithms are proposed to enhance the performance of many models.
- The proposed method is ...
Deep Learning-based Lightweight Models for Bactrocera Oleae Detection
Abstract
The detection of Bactrocera oleae (BO), an invasive pest that poses a serious threat to olive tree cultivation, is crucial for ensuring the quality and quantity of olive oil production. This study explores the application of deep learning with ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 82, Issue 7

Mar 2023

1554 pages

ISSN:1380-7501

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 27 August 2022

Accepted: 02 August 2022

Revision received: 06 April 2022

Received: 02 March 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sharma DRoul R(2024)Intelligent Traffic Congestion Classification FrameworkSN Computer Science10.1007/s42979-024-03564-z6:1Online publication date: 27-Dec-2024
https://dl.acm.org/doi/10.1007/s42979-024-03564-z
Said YAlassaf YAlsariera YGhodhbani RSaidani TBen Rhaiem OMakhdoum M(2024)Traffic flow management by detecting and estimating vehicles density based on object detection modelNeural Computing and Applications10.1007/s00521-024-09753-436:19(11495-11505)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09753-4

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents