research-article

An effective automatic object detection algorithm for continuous sonar image sequences

Authors:

Liang LuAuthors Info & Claims

Multimedia Tools and Applications, Volume 83, Issue 4

Pages 10233 - 10246

https://doi.org/10.1007/s11042-023-15837-5

Published: 20 June 2023 Publication History

Abstract

Object detection of continuous sonar image sequences has become an efficient way for underwater environment exploration. However, the task always suffers from the influence of th e complex underwater environment. In particular, the existing algorithms mainly focus on image-based detection and can not balance the detection speed and accuracy in continuous sonar image sequences. To solve the problem, this paper proposes a novel automatic detection algorithm based on deep learning for continuous sonar image sequences. Firstly, the convLSTM (convolution Long Short-Term Memory) is improved to fuse sonar features obtained from the cross-detection model, which are from three aspects: 1) The original convolution is replaced by depthwise separable convolution; 2) The original network input is divided into G groups and processed by group convolution; 3) A connection layer between the Bottleneck convolution layer and output is added to further capture feature information between frames. Then, to fully extract sonar features, a cross-detection network is established by fusing two different feature extraction networks MobileNetV3-Large and MobileNetV3-Small. Finally, we combine thecross-detection network with the improved convLSTM to establish the whole model, which can fully extract and utilize temporal information in continuous sonar image sequences. The experimental results show that the proposed model has effectively improved the detection speed in sonar image sequences at 150 FPS, simultaneously keeping an 85.8% mAP.

References

[1]

Bochkovskiy A, Wang C-Y, Mark Liao H-Y (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

[2]

Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

[3]

Dura E, Zhang Y, Liao X, Dobeck GJ, and Carin L Active learning for detection of mine-like objects in side-scan sonar imagery IEEE J Ocean Eng 2005 30 2 360-371

[4]

Fan Z, Xia W, Liu X, and Li H Detection and segmentation of underwater objects from forward-looking sonar based on a modified mask rcnn 2021 Image and Video Processing Signal 1-9

[5]

Feichtenhofer C, Pinz A, Zisserman A (2017) Detect to track and track to detect. In Proceedings of the IEEE International Conference on Computer Vision, pp 3038–3046

[6]

Guo J, Li Y, Lin W, Chen Y, Li J (2018) Network decoupling: From regular to depthwise separable convolutions. arXiv preprintar Xiv:1808.05517

[7]

Hochreiter S and Schmidhuber J Long short-term memory Neural Comput 1997 9 8 1735-1780

[8]

Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

[9]

Howard A, Sandler M (2019) Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1314–1324

[10]

Hurtós N, Palomeras N, Nagappa S, Salvi J (2013) Automatic detection of underwater chain links using a forward-looking sonar. In 2013 MTS/IEEE OCEANS-Bergen, pp 1–7

[11]

Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

[12]

Hu K and Wang Z Graph sequence recurrent neural network for vision-based freezing of gait detection IEEE Trans Image Process 2019 29 1890-1901

[13]

Jiang Z, Liu Y, Yang C (2020) Learning where to focus for efficient video object detection. In European Conference on Computer Vision, pp 18–34

[14]

Kim J, Yu S-C (2016) Convolutional neural network-based real-time rov detection using forward-looking sonar image. In 2016 IEEE/OES Autonomous Underwater Vehicles (AUV), pp 396–400

[15]

Krizhevsky A, Sutskever I, and Hinton GE Imagenet classification with deep convolutional neural networks Adv Neural Inf Process Syst 2012 25 1097-1105

[16]

Liu M, Zhu M (2018) Mobile video object detection with temporally-aware feature maps. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5686–5695

[17]

Maki T and Horimoto H Tracking a sea turtle by an auv with a multibeam imaging sonar: Toward robotic observation of marine life Int J Control Autom Syst 2020 18 3 597-604

[18]

Ma Q, Jiang L, Yu W (2020) Training with noise adversarial network: A generalization method for object detection on sonar image. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp 729–738

[19]

McKay J, Gerg I, Monga V, Raj RG (2017) What’s mine is yours: Pretrained cnns for limited training sonar atr. In OCEANS 2017-Anchorage, pp 1–7

[20]

Myers V and Fawcett J A template matching procedure for automatic target recognition in synthetic aperture sonar imagery IEEE Signal Process Lett 2010 17 7 683-686

[21]

Qin R, Zhao X, and Zhu W Multiple receptive field network (mrf-net) for autonomous underwater vehicle fishing net detection using forward-looking sonar images Sensors 2021 21 6 1933

[22]

Sung M, Kim J, and Lee M Realistic sonar image simulation using deep learning for underwater object detection Int J Control Autom Syst 2020 18 3 523-534

[23]

Tian M, Chen H, and Wang Q Detection and recognition of flower image based on ssd network in video stream J Physics: Conf Ser 2019 1237 032045

[24]

Valdenegro-Toro M (2016) Objectness scoring and detection proposals in forward-looking sonar images with convolutional neural networks. In IAPR workshop on artificial neural networks in pattern recognition, pp 209–219

[25]

Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprintar Xiv:2207.02696

[26]

Wang Y-X, Girshick R, Hebert M, Hariharan B (2018) Low-shot learning from imaginary data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7278–7286

[27]

Wang Z, Zhang S, Huang W, Guo J, and Zeng L Sonar image target detection based on adaptive global feature enhancement network IEEE Sens J 2022 22 2 1509-1530

[28]

Williams DP and Fakiris E Exploiting environmental information for improved underwater target classification in sonar imagery IEEE Trans Geosci Remote Sens 2014 52 10 6284-6297

[29]

Wu S, Liu Y, Li S, and Zhang S Lsh-detr: object detection algorithm for marine benthic organisms based on improved detr J Electron Imaging 2022 31 6 063030

[30]

Zacchini L, Topini A, Franchi M, Secciani N, Manzari V, Bazzarello L, Stifani M, Ridolfi A (2022) Autonomous underwater environment perceiving and modeling: An experimental campaign with feelhippo auv for forward looking sonar-based automatic target recognition and data association. IEEE Journal of Oceanic Engineering, pp 1–20

[31]

Zhou T, Si J, Wang L, Xu C, and Yu X Automatic detection of underwater small targets using forward-looking sonar images IEEE Trans Geosci Remote Sens 2022 60 1-12

[32]

Zhou W, Wang Z (2021) Research on autonomous detection method of underwater unmanned vehicle. In 2021 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp 1–5

[33]

Zhu X, Xiong Y (2017) Deep feature flow for video recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2349–2358

Cited By

Shi PHe QZhu SLi XFan XXin Y(2024)Multi-scale fusion and efficient feature extraction for enhanced sonar image object detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124958256:COnline publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124958

Index Terms

An effective automatic object detection algorithm for continuous sonar image sequences
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
      2. Computer vision tasks
        Scene understanding
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Extraction of 3D information from sonar image sequences

This paper describes a set of methods that make it possible to estimate the position of a feature inside a three-dimensional (3D) space by starting from a sequence of two-dimensional (2D) acoustic images of the seafloor acquired with a sonar system. ...
Detection and tracking of underwater object based on forward-scan sonar
ICIRA'12: Proceedings of the 5th international conference on Intelligent Robotics and Applications - Volume Part I

Underwater object detection is critical in a lot of applications in maintenance, repair of undersea structures, marine sciences, and homeland security. However, because optics camera is subject to the influence of light and turbidity, its visibility is ...
Rotated object detection with forward-looking sonar in underwater applications
Highlights
- A novel data set containing underwater sonar images.
- Two novel deep learning ...
Abstract
Autonomous underwater vehicles (AUVs) are often used to inspect the condition of submerged structures in oil and gas fields. Because the use of global positioning systems to aid AUV navigation is not feasible, object detection is an ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 83, Issue 4

Jan 2024

2884 pages

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 20 June 2023

Accepted: 10 May 2023

Revision received: 19 March 2023

Received: 21 October 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shi PHe QZhu SLi XFan XXin Y(2024)Multi-scale fusion and efficient feature extraction for enhanced sonar image object detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124958256:COnline publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124958

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents