PolSAR Ship Detection Based on a SIFT-like PolSAR Keypoint Detector

Abstract

The detection of ships on the open sea is an important issue for both military and civilian fields. As an active microwave imaging sensor, synthetic aperture radar (SAR) is a useful device in marine supervision. To extract small and weak ships precisely in the marine areas, polarimetric synthetic aperture radar (PolSAR) data have been used more and more widely. We propose a new PolSAR ship detection method which is based on a keypoint detector, referred to as a PolSAR-SIFT keypoint detector, and a patch variation indicator in this paper. The PolSAR-SIFT keypoint detector proposed in this paper is inspired by the SAR-SIFT keypoint detector. We improve the gradient definition in the SAR-SIFT keypoint detector to adapt to the properties of PolSAR data by defining a new gradient based on the distance measurement of polarimetric covariance matrices. We present the application of PolSAR-SIFT keypoint detector to the detection of ship targets in PolSAR data by combining the PolSAR-SIFT keypoint detector with the patch variation indicator we proposed before. The keypoints extracted by the PolSAR-SIFT keypoint detector are usually located in regions with corner structures, which are likely to be ship regions. Then, the patch variation indicator is used to characterize the context information of the extracted keypoints, and the keypoints located on the sea area are filtered out by setting a constant false alarm rate threshold for the patch variation indicator. Finally, a patch centered on each filtered keypoint is selected. Then, the detection statistics in the patch are calculated. The detection statistics are binarized according to the local threshold set by the detection statistic value of the keypoint to complete the ship detection. Experiments on three data sets obtained from the RADARSAT-2 and AIRSAR quad-polarization data demonstrate that the proposed detector is effective for ship detection.

1. Introduction

Ship monitoring plays an important role in maintaining navigation safety, combating illegal fishing and coastal defense early warning missions. As an active microwave imaging system, synthetic aperture radar (SAR) has the ability to observe the Earth in all weather and at all times.

Due to its metal material and complex structure, the backscattering energy of a large ship is often strong, and it appears as a bright area on the SAR imaging plane. At the same time, when the wind speed is low, the backscattering energy of sea clutter is weak, and it appears as a dark area in the SAR image. Therefore, the intensity difference between the ship and sea is a useful feature to distinguish them. However, for a small ship or high wind speed condition, the intensity difference between the ship and sea clutter may decrease.

Since the polarimetric scattering mechanism difference between ship and sea clutter is beneficial to ship detection, ship detection using polarimetric SAR (PolSAR) data has become a promising research area for maritime surveillance. Many ship detectors based on PolSAR data have been proposed [1,2,3,4,5,6,7,8,9,10,11,12]. According to whether spatial information is used, PolSAR ship detection methods can be divided into two categories.

One category is based on the pixel-level feature. The polarimetric whitening filter is an optimal technique for target detection by minimizing the fluctuation of speckle noise [1]. The reflection symmetry (RS) metric is used to detect man-made targets [2] based on the difference of physical behavior between ship and sea clutter. It has been successfully used in PolSAR ship detection. Marino et al. proposed a novel PolSAR ship detector called the polarimetric notch filter (PNF), which enhanced the TCR between ship and sea clutter by minimizing the sea clutter power [3]. Yang et al. proposed a novel polarimetric notch filter which is suitable for a heterogeneous region [4]. It extended the clutter model in PNF from a complex feature vector to a complex feature subspace. Liu et al. proposed a new form of the polarimetric notch filter (NPNF) based on the physical mechanism of targets and clutter which is further developed for partial targets [5]. The polarimetric matched filter (PMF) is another optimal polarimetric enhancement technique which improved the target detection performance by maximizing the target-to-clutter ratio (TCR) [12]. Liu et al. proposed a new optimal technique by combining the PMF and PNF to maximize the TCR and minimize the speckle noise, which was referred to as the polarimetric detection optimization filter (PDOF) [13]. Yang et al. proposed a novel optimal polarimetric enhancement method based on the minimal clutter-to-signal (MCSR) subspace [14]. Chen et al. developed the uniform polarimetric matrix transformation theory [15] and proposed a visualization and characterization tool for PolSAR data investigation called polarimetric correlation pattern [16]. Cui et al. proposed a PolSAR ship detection method based on the polarimetric rotation domain feature [17]. Li et al. investigated the optimal combination of the polarimetric rotation domain feature and proposed a ship detection method based on SVM [18].

The other category is based on the region-level feature which includes the superpixel-level feature and patch-level feature. Wang et al. proposed a PolSAR ship detection method based on the superpixel-level scattering mechanism distribution feature [6]. He et al. proposed a ship detection method by using a local scattering mechanism difference based on regression kernel (SD-LSMDRK) [7]. He et al. also proposed a novel automatic ship detection method based on the superpixel-level local information measurement (SD-SLLIM) [8]. Liu et al. proposed a new neighborhood polarimetric covariance matrix (NPCM) to detect the small ships in PolSAR image [9], which utilized the spatial correlation between neighbor pixels and mapped a low-dimensional complex feature vector into a high-dimensional complex vector. Cui et al. proposed a saliency detector for PolSAR ship detection which focused on the different scattering mechanism between ship and sea clutter in low and medium sea condition [19]. Wang et al. proposed a saliency detector for target detection in PolSAR images based on the pattern recurrence, which is defined by the similarity between PolSAR image patches [20]. We proposed a refined PNF ship detector which introduced region information into traditional PNF by a patch variation indicator [10]. It is validated on the measured PolSAR data. However, calculating the patch variation indicator of each pixel in a PolSAR image is time consuming. Determining potential target regions in advance helps to improve computational efficiency.

The advent of deep learning algorithms allows ship detection to be done automatically in an end-to-end mode. Chen et al. [21] proposed a PolSAR target detection and classification method using a deep convolutional neural network (CNN). Fan et al. [22] proposed a ship detection method for PolSAR data based on a modified faster region-based convolutional neural network (Faster R-CNN). Jin et al. [23] proposed a patch-to-pixel CNN for small ship detection in PolSAR images. However, these methods usually require a large number of training samples, which limits the application of them in practical use [24].

In our previous work [11], we proposed a PolSAR ship detection method which was based on the physical behavior and spatial characteristics of ship and sea clutter. Reflection symmetry is a useful physical behavior for man-made target detection. Man-made targets usually have a reflection asymmetry characteristic and appears as bright blob-like structures on amplitude images of the $C_{12}$ and $C_{23}$ terms in the polarimetric covariance matrix [11], while sea clutter usually has a reflection symmetry characteristic and appears as dark regions on the above images. Thus, ships can be detected by detecting these bright blob-like structures. The scale-invariant feature transform (SIFT) algorithm is widely used in optical remote sensing fields due to its efficiency [25], and includes a keypoint detector and a local descriptor extractor. Wang et al. applied the traditional SIFT keypoint detector in SAR image change detection and proved that the keypoints are useful to target detection [26]. Inspired by this work, we used the SIFT keypoint detector on the amplitude image of $C_{12}$ and $C_{23}$ terms to detect the bright blob-like structure and extract potential ship regions in the $C_{12}$ and $C_{23}$ images. However, the reflection asymmetry characteristics of ships with simple structures or smaller sizes are not significant, so the original SIFT keypoint detector may not be able to detect this type of ship. Moreover, although the SIFT keypoint detection uses region information, the final detection step of the method we proposed before is still implemented at the pixel level, which is not completely a region-based method.

Compared with optical remote sensing data, SAR images have different characteristics. SAR is an active microwave imaging system and has the ability to acquire fine spatial resolution images regardless of weather conditions and solar illumination. Due to the coherent imaging mechanism of the SAR system, speckle noise is an inherent phenomenon in SAR imaging, which increases the difficulty of data processing. To reduce the influence of speckle on SIFT performance, Dellinger et al. proposed a SIFT-like algorithm named SAR-SIFT which is specifically dedicated to SAR images [27]. The algorithm included both the keypoint detector and local descriptor extractor. A new gradient definition robust to speckle noise which is used for the keypoint detection and local descriptor extraction was introduced. In addition to the intensity information in SAR data, PolSAR data also contain abundant polarimetric information. It is a reasonable idea to define gradients according to the polarimetric information of PolSAR data to construct a SAR-SIFT-like algorithm suitable for PolSAR data. Zou et al. proposed a SAR-SIFT-like algorithm for PolSAR image registration by defining a gradient based on the Freeman three-component decomposition [28]. However, the Freeman decomposition method has negative power values for some pixels [29]; thus, the gradient may not be consistent with actual PolSAR data.

To address the above problems, we introduce an improved gradient definition adapted to the characteristics of PolSAR data. It is then used to adapt the keypoint detector of the SAR-SIFT algorithm to PolSAR data. This part of the work is largely inspired by the SAR-SIFT and will be referred to as the PolSAR-SIFT keypoint detector. Then, the keypoint detector is used to locate the potential target regions in PolSAR images and combined with the patch variation indicator [10] for ship detection, and the keypoints located on the sea area are filtered out by setting a constant false alarm rate threshold for the patch variation indicator.

The content of this paper is arranged as follows: Section 1 is the introduction; Section 2 briefly introduces the PolSAR data and the outline of the SAR-SIFT algorithm; Section 3 introduces the proposed ship detection method based on the PolSAR-SIFT keypoint detection algorithm; Section 4 shows the experimental validations with the measured PolSAR data; and Section 5 concludes this paper.

2. PolSAR Data and SAR-SIFT Keypoint Detector

2.1. Polarimetric SAR Data

In polarimetric SAR data, in addition to the difference in backscattering energy between ship and sea clutter, their polarimetric scattering mechanisms are also different. The backscattering of sea clutter is usually dominated by surface scattering. As a man-made target, the scattering mechanism of a large ship is complex, which includes odd-bounce scattering, double-bounce scattering, volume scattering and helix scattering caused by the complex structure. Among these, the dominant scattering mechanism is the double-bounce scattering mechanism. The polarimetric information of ships and sea clutter can be fully represented by the complex scattering matrix S [30]:

$$S=\left[\begin{array}{l}{S}_{\mathrm{HH}} S_{\mathrm{HV}}\\ S_{\mathrm{VH}} S_{\mathrm{VV}}\end{array}\right]$$

(1)

where the first and second subscripts of $S_{\mathrm{HH}}$ denote the receive and transmit polarization and the subscripts H and V denote the horizontal and vertical polarizations, respectively. $S_{\mathrm{HV}}$ is equal to $S_{\mathrm{VH}}$ in the single station case based on the reciprocity theorem [30]. To deal with polarimetric scattering from a distributed and depolarized target in a dynamically changing environment, each pixel in a PolSAR image can be represented by its second-order statistical characteristics, reflected by the polarimetric covariance matrix as follows:

$$C=\left[\begin{array}{l}〈{\left|S_{\mathrm{HH}}\right|}^2〉 \sqrt{2}〈S_{\mathrm{HH}}{S}_{\mathrm{HV}}^{\ast }〉 〈S_{\mathrm{HH}}{S}_{\mathrm{VV}}^{\ast }〉\\ \sqrt{2}〈S_{\mathrm{HV}}{S}_{\mathrm{HH}}^{\ast }〉 2 〈{\left|S_{\mathrm{HV}}\right|}^2〉 \sqrt{2}〈S_{\mathrm{HV}}{S}_{\mathrm{VV}}^{\ast }〉\\ 〈S_{\mathrm{VV}}{S}_{\mathrm{HH}}^{\ast }〉 \sqrt{2}〈S_{\mathrm{VV}}{S}_{\mathrm{HV}}^{\ast }〉 〈{\left|S_{\mathrm{VV}}\right|}^2〉\end{array}\right]$$

(2)

where the superscript * denotes the complex conjugate and $\langle \cdot \rangle$ stands for the ensemble average.

2.2. Original SAR-SIFT Keypoint Detector

The SAR-SIFT algorithm was proposed by Dellinger et al. in 2015 [13] and later applied in SAR image registration, target discrimination and target recognition. The algorithm consists of two operators: a keypoint detection operator and a local feature descriptor. Both the operators rely on the consistent gradient computation method adapted to the characteristics of SAR images.

The ratio of exponentially weighted averages (ROEWA) [27] operator is used to calculate the multiscale gradients for a SAR image, which is suitable to multiplicative noise. For a given pixel (a,b) in a SAR image, the local exponentially weighted averages for the four directions, which include the up, the down, the left and the right side, need to be computed firstly, and can be denoted as $M_{\mathrm{u},\alpha }$, $M_{\mathrm{d},\alpha }$, $M_{\mathrm{l},\alpha }$ and $M_{\mathrm{r},\alpha }$, respectively. For example, the definition of $M_{\mathrm{u},\alpha }$ and $M_{\mathrm{d},\alpha }$ given in [23] is as follows:

$$\begin{array}{l}{M}_{\mathrm{u},\alpha }={\displaystyle {\int }_{x=R}{\displaystyle {\int }_{y=R^{+}}I\left(a+x,b+y\right)\times e^{-\frac{\left|x\right|+\left|y\right|}{\alpha }}}}\\ M_{\mathrm{d},\alpha }={\displaystyle {\int }_{x=R}{\displaystyle {\int }_{y=R^{-}}I\left(a+x,b+y\right)\times e^{-\frac{\left|x\right|+\left|y\right|}{\alpha }}}}\end{array}$$

(3)

where $I\left(\cdot \right)$ represents the image intensity of a pixel and $\alpha$ is the exponential weight parameter that determines the scale of the average. The ratios for the vertical and the horizontal directions are formulated as follows:

$$R_{\mathrm{v},\alpha }=\frac{M_{\mathrm{u},\alpha }}{M_{\mathrm{d},\alpha }},R_{\mathrm{h},\alpha }=\frac{M_{\mathrm{l},\alpha }}{M_{\mathrm{r},\alpha }}$$

(4)

The exponential weight parameter $\alpha$ allows adaptive smoothing for SAR images. Therefore, the two ratios are robust to speckle noise. The gradient by ratio (GR) in the vertical and the horizontal directions are defined as

$$G_{\mathrm{v},\alpha }=\log \left(R_{\mathrm{v},\alpha }\right),G_{\mathrm{h},\alpha }=\log \left(R_{\mathrm{h},\alpha }\right)$$

(5)

Utilizing the GR, the SAR-SIFT algorithm formulated a new multiscale SAR-Harris matrix C_SH and a new multiscale SAR-Harris response function R_SH, shown as follows:

$$\begin{array}{c}{C}_{\mathrm{SH}}\left(x,y,\alpha \right)=g_{\sqrt{2}\alpha }\ast \left[\begin{array}{cc}{\left(G_{\mathrm{h},\alpha }\right)}^2& \left(G_{\mathrm{h},\alpha }\right)\left(G_{\mathrm{v},\alpha }\right)\\ \left(G_{\mathrm{v},\alpha }\right)\left(G_{\mathrm{h},\alpha }\right)& {\left(G_{\mathrm{v},\alpha }\right)}^2\end{array}\right]\\ R_{\mathrm{SH}}\left(x,y,\alpha \right)=\det \left(C_{\mathrm{SH}}\left(x,y,\alpha \right)\right)-d\cdot \mathrm{tr}{\left(C_{\mathrm{SH}}\left(x,y,\alpha \right)\right)}^2\end{array}$$

(6)

where $g_{\sqrt{2}\alpha }$ is a Gaussian kernel with standard deviation $\sqrt{2}\alpha$, $\ast$ is the convolution operator, det(·) and tr(·) represent the determinant and trace of a matrix, respectively, and d is a given parameter.

The detection of keypoints can be achieved by: (1) computing the multiscale SAR-Harris matrix C_SH by setting different scale parameter α and generating the SAR-Harris response function R_SH (Equation (6)), and then constructing the SAR-Harris scale-space by R_SH images with different scales; (2) selecting extrema in space; and (3) applying a threshold T_SH on the selected extrema to get the detected keypoints. Please refer to [27] for more details of the SAR-SIFT keypoint detection method.

3. Ship Detection Method Based on PolSAR-SIFT Keypoint Detector

The proposed PolSAR ship detection method mainly consists of three steps: (1) keypoint detection for locating the potential target area; (2) keypoint filtering for eliminating false alarm keypoints; and (3) ship region extraction for final detection. Figure 1 shows the outline of the proposed algorithm, which is named as a PolSAR-SIFT-based ship detector.

3.1. PolSAR-SIFT Keypoint Detector

To adapt the SIFT algorithm to SAR images, the SAR-SIFT method took the speckle noise of SAR images into account. It developed a new gradient computation method by using the ratio of exponentially weighted averages. The exponentially weighed averages are obtained by computing exponentially weighted local means of a pixel in a SAR image, which is essentially an image filtering operation. The weight coefficient of the filter is determined by the spatial distance between pixels using an exponential kernel.

Inspired by the formulation of the exponentially weighted averages in SAR images, and considering the robustness of ROEWA to speckle noise, we define the exponentially weighted averages of polarimetric covariance matrix for PolSAR data, which are also obtained by computing exponentially weighted local means. For a given pixel (a,b) in a PolSAR image, the exponentially weighted averages of the polarimetric covariance matrix for the four directions can be denoted as $M{C}_{\mathrm{u},\alpha }$, $M{C}_{\mathrm{d},\alpha }$, $M{C}_{\mathrm{l},\alpha }$ and $M{C}_{\mathrm{r},\alpha }$, respectively. For example, the definition of $M{C}_{\mathrm{u},\alpha }$ and $M{C}_{\mathrm{d},\alpha }$ is as follows:

$$\begin{array}{l}M{C}_{\left(i,j\right)}{}_{\mathrm{u},\alpha }={\displaystyle {\int }_{x=R}{\displaystyle {\int }_{y=R^{+}}{C}_{\left(i,j\right)}\left(a+x,b+y\right)\times e^{-\frac{\left|x\right|+\left|y\right|}{\alpha }}}}\\ M{C}_{\left(i,j\right)}{}_{\mathrm{d},\alpha }={\displaystyle {\int }_{x=R}{\displaystyle {\int }_{y=R^{-}}{C}_{\left(i,j\right)}\left(a+x,b+y\right)\times e^{-\frac{\left|x\right|+\left|y\right|}{\alpha }}}}\end{array}$$

(7)

where $C_{\left(i,j\right)}$ represents the (ith,jth) term in the polarimetric covariance matrix of a pixel.

The ratio of the exponentially weighted means in ROEWA is essentially a difference measurement for the weighted pixels. For PolSAR data, there are many difference measurements based on polarimetric covariance matrix. For example, Wishart distribution-based distance [30], Riemannian metric distance [31] and symmetrized Kullback–Leibler divergence [32]. Among them, the Wishart distribution-based distance and the symmetrized Kullback–Leibler divergence are both defined according to statistical distribution. The weighting operation on the covariance matrix changes the distribution characteristics of the covariance matrix, so the distance defined based on the statistical distribution cannot be directly applied to the weighted matrix. Therefore, we define the difference measurement for exponentially weighted averages of the polarimetric covariance matrix based on the Riemannian metric distance in this paper, and this method is referred to as the Riemannian distance of exponentially weighted averages (RDEWA). The Riemannian distances for the vertical and the horizontal directions are formulated as follows:

$$\begin{array}{l}R{D}_{\mathrm{v},\alpha }=\sqrt{\mathrm{tr}\left(M{C}_{\mathrm{u},\alpha }\right)+\mathrm{tr}\left(M{C}_{\mathrm{d},\alpha }\right)-2\mathrm{tr}\left(M{C}_{\mathrm{u},\alpha }^{1/2}M{C}_{\mathrm{d},\alpha }^{1/2}\right)}\\ R{D}_{\mathrm{h},\alpha }=\sqrt{\mathrm{tr}\left(M{C}_{\mathrm{l},\alpha }\right)+\mathrm{tr}\left(M{C}_{\mathrm{r},\alpha }\right)-2\mathrm{tr}\left(M{C}_{\mathrm{l},\alpha }^{1/2}M{C}_{\mathrm{r},\alpha }^{1/2}\right)}\end{array}$$

(8)

where tr(·) represents the trace of a matrix. The gradients of the vertical and the horizontal directions are computed by $R{D}_{\mathrm{v},\alpha }$ and $R{D}_{\mathrm{h},\alpha }$ in the same way as SAR-SIFT, i.e.,

$$G_{\mathrm{RDv},\alpha }=\log \left(R{D}_{\mathrm{v},\alpha }\right),G_{\mathrm{RDh},\alpha }=\log \left(R{D}_{\mathrm{h},\alpha }\right)$$

(9)

The multiscale SAR-Harris matrix and function are defined in (4). Inspired by this definition and the gradient by Riemannian distance (GRD), we propose the new multiscale PolSAR-Harris matrix C_PSH and PolSAR-Harris response function R_PSH, respectively, as

$$\begin{array}{c}{C}_{\mathrm{PSH}}\left(x,y,\alpha \right)=g_{\sqrt{2}\alpha }\ast \left[\begin{array}{cc}{\left(G_{\mathrm{RDh},\alpha }\right)}^2& \left(G_{\mathrm{RDh},\alpha }\right)\left(G_{\mathrm{RDv},\alpha }\right)\\ \left(G_{\mathrm{RDv},\alpha }\right)\left(G_{\mathrm{RDh},\alpha }\right)& {\left(G_{\mathrm{RDv},\alpha }\right)}^2\end{array}\right]\\ R_{\mathrm{PSH}}\left(x,y,\alpha \right)=\det \left(C_{\mathrm{PSH}}\left(x,y,\alpha \right)\right)-d\cdot \mathrm{tr}{\left(C_{\mathrm{PSH}}\left(x,y,\alpha \right)\right)}^2\end{array}$$

(10)

where d is an arbitrary parameter whose default value is properly set to 0.015 in experiments and the gradients $G_{\mathrm{RDv},\alpha }$ and $G_{\mathrm{RDh},\alpha }$ are computed using (9). The keypoint detection step of the proposed PolSAR ship detection method can be achieved by: (1) computing the multiscale PolSAR-Harris matrix C_PSH by setting different scale parameter α and generating the PolSAR-Harris response function R_PSH (Equation (10)), and then constructing the PolSAR-Harris scale-space by R_PSH images with different scales; (2) selecting extrema in PolSAR-Harris scale-space; and (3) applying a threshold T_PSH on the selected extrema, which could suppress low-contrast and edge points. The top part of Figure 1 summarizes the above steps.

We use the GRD which is obtained by computing the RDEWA of the polarimetric covariance matrix to adapt the keypoint detector to PolSAR data. The same as for the SAR-Harris function, the threshold T_PSH on the PolSAR-Harris response function value could remove the edge points and low-contrast points. In the original SAR-SIFT algorithm, the steps after detecting the keypoints are orientation assignment and descriptor extraction, which both rely on the histogram of the gradient computed by the GR. For PolSAR data, the orientation assignment and descriptor extraction could be obtained by the GRD, which is not the focus of this paper. In the following section, we introduce the PolSAR ship detection method based on this keypoint detector.

3.2. Ship Detection Based on the Keypoint and Patch Variation Indicator

The effectiveness of the SAR-SIFT keypoint detection operator in detecting corner structures in SAR images has been confirmed [27]. In PolSAR images, compared with sea clutter, ships have strong backscattering and complex polarimetric scattering characteristics. Therefore, in both the image intensity domain and the polarimetric scattering mechanism domain, ships will have corner shape structures. The PolSAR-SIFT keypoint detector based on the GRD, which combines intensity information and polarimetric scattering information, could be used to detect corner points in PolSAR images. Then, ships in PolSAR images could be extracted using these corner points.

Considering weak and small ships, we need to set a lower $T_{\mathrm{PSH}}$ for R_PSH to detect weak ships. A lower threshold means false alarms, i.e., keypoints located in the sea area will be also extracted. To eliminate these kinds of false alarm keypoints, the local context information should be taken into account. Therefore, the patch variation indicator [10] of each keypoint is computed. The expression of $P_{\mathrm{vi}}$ is as follows [10]:

$$P_{\mathrm{vi}}\left(x_i\right)=\frac{\mathrm{std}\left[I_s\left(p_i\right)\right]}{\mathrm{mean}\left[I_s\left(p_i\right)\right]}$$

(11)

where $p_i$ is a patch centered on a keypoint and $I_s\left(p_i\right)$ is the similarity image obtained by computing the similarity metric between the center pixel and other pixels in $p_i$. Moreover, std[·] and mean[·] represent the standard deviation and mean of the elements in the similarity image $I_s\left(p_i\right)$, respectively. The similarity metric between pixels is defined based on the symmetrized Kullback–Leibler divergence [32] of polarimetric covariance matrixes as introduced in [10], after comparing the detection performance of $P_{\mathrm{vi}}$ computed using different distance metrics on synthetic data. The expression of the symmetrized Kullback–Leibler divergence is as follows [18]:

$$\mathrm{KL}\left(C_1,C_2\right)=1/2\cdot \mathrm{tr}\left(C_1^{-1}{C}_2+C_2^{-1}{C}_1\right)-dim$$

(12)

where dim is the dimension of the polarimetric covariance matrix $C_1$ and $C_2$. Then, the similarity based on Equation (12) can be formulated as

$$s_{\mathrm{KL}}\left(C_1,C_2\right)=\mathit{exp}\left(-\mathrm{KL}\left(C_1,C_2\right)/h\right)$$

(13)

where h is a tuning parameter whose default value is properly set to 5 in experiments.

Generally, keypoints located on the ship are quite different from other points in the context. However, keypoints located on the sea area are less different from other points in the context. Thus, the $P_{\mathrm{vi}}$ of the keypoint on the ship is larger, and the $P_{\mathrm{vi}}$ of the keypoint on the sea area is smaller. Therefore, by comparing the $P_{\mathrm{vi}}$ of the keypoint with a threshold, the keypoints located on the sea area can be filtered out. The threshold is set under a constant false alarm rate (CFAR) criterion in this paper.

To realize the CFAR detection, it is necessary to establish an accurate statistical model for $P_{\mathrm{vi}}$ first. In general, statistical modeling and probability density function estimation methodologies have two main categories: parametric and nonparametric. Parametric methodologies rely on a certain a priori parametric model assumption for data distribution. Nonparametric methodologies can model arbitrary probability density functions with general assumptions and a certain approximation error [33]. As a nonparametric method, kernel density estimation (KDE) can produce smooth continuous differentiable functions and ensure a good probability density function approximation for a set of given data [34,35]. KDE has been used in many applications. We also use the KDE method to develop an effective model for $P_{\mathrm{vi}}$. In the KDE method, a kernel function is assigned to each sample $\left\{X_i,i=1,\dots ,N\right\}$, and the probability density function that represents these data is approximated by

$$\psi \left(X\right)=\frac{1}{N\sigma }{\displaystyle \sum }_{i=1}^N\kappa \left(\frac{X-X_i}{\sigma }\right)$$

(14)

where $\kappa \left(\cdot \right)$ is the kernel function and σ is the kernel bandwidth. The kernel function and bandwidth are two main factors influencing the model efficiency and local smoothness. Most studies prefer to apply Gaussian kernel due to its good properties. Therefore, we also chose the Gauss kernel in this paper. A classical asymptotic mean integrated squared error (AMISE) bandwidth estimation [33] is as follows:

$$\sigma =\frac{1.06}{N^{1/5}}{\sigma }_{\mathrm{data}}$$

(15)

where ${\sigma }_{\mathrm{data}}$ is the stand deviation of the given data $\left\{X_i,i=1,\dots ,N\right\}$. We performed KDE estimation on a set of sea clutter data, and set a CFAR threshold for keypoint filtering based on the estimated PDF and CDF. This part of the work is summarized in the middle part of Figure 1.

In our previous work [10], we needed to calculate the $P_{\mathrm{vi}}$ of each pixel in the image and use $P_{\mathrm{vi}}$ to identify whether the pixel is a ship or sea clutter, which is time-consuming, and the statistical distribution of $P_{\mathrm{vi}}$ was not analyzed. In this paper, the location of potential targets is obtained by extracting the keypoints in advance. The keypoints located on the sea are eliminated through the $P_{\mathrm{vi}}$ of the keypoints, and the number of keypoints is much smaller than the total number of pixels in the image, which improves the calculation efficiency. Moreover, we analyze the statistical distribution of $P_{\mathrm{vi}}$ to set the threshold by the CFAR criterion.

Ship detection can be accomplished in the regions around the filtered keypoints. We can select a square region centered on a keypoint and apply a ship detector on this region. A ship detector is usually implemented by calculating the detection statistics of all pixels in the image, and then setting a threshold on the defined detection statistics, and the pixels larger than the threshold are considered as ship targets. In this paper, the detection statistic is represented by the power of the double-bounce scattering mechanism which is mainly contained in ship targets. According to the Pauli decomposition theorem [30], HH + VV, HH − VV and HV channels correspond to the surface reflection and double-bounce reflection which is caused by the angular reflectors with direction angle of 0° and 45°. Therefore, the HH − VV channel and HV channel can be used to complete ship detection. It should be noticed that the ensemble average of |HH − VV|·|HV| is also a term in the polarimetric coherence matrix, which also represents the reflection asymmetry component [30]. Therefore, we use the ensemble average of |HH − VV|·|HV| as the detection statistic in this paper. Finally, by combining all the ship regions extracted through the keypoints, the final ship detection result can be obtained as shown in the bottom part of Figure 1.

4. Ship Detection Performance Validation

Figure 2 shows the two sets of C-band quad-polarization RADARSAT-2 data and one set of L-band quad-polarization AIRSAR data used for validation in this section. The details of the data are listed in

Table 1. Description of the three data sets.

	Dataset 1	Dataset 2	Dataset 3
Location	Yokohama Port, Japan	Tanggu Port, China	Kojimawan Bay, Japan
Date	4 August 2010	23 June 2011	4 October 2000
Resolution (range × azimuth)	12 m × 8 m	12 m × 8 m	3.33 m × 4.63 m
Incidence angle	35°	30°	-

.

4.1. Validation of Each Part of the Proposed Method

4.1.1. Keypoint Detection Test

First, we generated a set of synthetic data using the ship targets in the R1 region and the randomly selected sea clutter pixels in other regions, and the generation method is consistent with that in [6]. The Pauli pseudo-color image and the ground truth image of the synthetic data are shown in Figure 3a,b, respectively. The keypoint detection results of the proposed PolSAR-SIFT keypoint detector and those of the SAR-SIFT keypoint detector are shown in Figure 3c,d. The keypoint detection results of the SAR-SIFT keypoint detector are obtained by applying the SAR-SIFT detector on the SPAN image of PolSAR data. The result of Figure 3c is different from Figure 1(1) due to the different thresholds for the PolSAR-Harris response values.

The scale parameters in the two keypoint detectors are both set to α = 2.51. Although the application background of the SAR-SIFT algorithm and the PolSAR-SIFT keypoint detector proposed in this paper is different, we still give the detection results of the SAR-SIFT keypoint detector on the synthetic data to demonstrate the effect of introducing polarimetric information into keypoint extraction. It can be seen from Figure 3c,d that the proposed keypoint detector that introduces polarimetric information is more suitable for the ship detection. It extracts more keypoints on ship targets in the synthetic data, and has no false alarm points on the sea area in the synthetic data. The keypoint detection result of the SAR-SIFT keypoint detector also has no false alarms on the sea area. However, it extracts fewer keypoints on the large ship. The Harris response images of the two keypoint detectors are shown in Figure 3e,f. Since the intensity difference is small inside the large ship in the SPAN image, the GR is small inside the large ship. Since the magnitude of the SAR-Harris function response value is related to the magnitude of the gradient, the SAR-Harris response inside the ship region is small. In this condition, the SAR-SIFT keypoint detector cannot extract keypoints inside the ship region. While the polarimetric scattering difference is significant inside the large ship, as shown in Figure 3e, the PolSAR-Harris response function still has a high response value inside the large ship.

Although PolSAR-SIFT keypoint detection can be achieved by constructing multi-scale PolSAR-Harris function images, we found in experiments that the keypoints extracted from a single scale PolSAR-Harris function image are sufficient to complete the ship detection mission. Figure 4 presents the keypoint extraction results on the R2 region for a set of keypoint detectors with the same threshold and different scale parameters ${\alpha }_m={\alpha }_0\cdot c^m$, with $m\in \left[0,\dots ,7\right]$, ${\alpha }_0=1$ and $c=2^{1/3}$. Figure 4 shows that many false alarm keypoints occur at small scale parameters as mentioned in [13]. When the scale parameter of the keypoint detector is large, there are fewer false alarms detected. However, since the larger scale parameter has a more significant smoothing effect on the PolSAR image, it also has missed detections on small and weak targets. According to the keypoint detection results in Figure 4, we chose an appropriate scale parameter α = 2.51 for the following experiments.

Large ships tend to have some strong scattering points, and the PolSAR-Harris response value of keypoints extracted near such scattering points will also be large. It is not difficult to extract keypoints located on large ships by setting a suitable threshold on the PolSAR-Harris response value. However, for a weak and small ship target, its PolSAR-Harris response may be very close to that of sea clutter, so it is difficult for us to choose an accurate threshold. To detect more weak and small objects, we set a lower threshold. The threshold T_PSH has been set to the maximum value of the PolSAR-Harris response function multiplied by 10⁻⁵ in the following experiments. This parameter does not need to be finely adjusted for different data, as long as it is small enough to detect all small targets. The resulting false alarm keypoints will be removed by the patch variation indicator in the subsequent step.

4.1.2. Patch Variation Indicator

In order to filter out the keypoints located on the sea clutter, we introduced the patch variation indicator we proposed before. Moreover, a filter operation based on the CFAR threshold is given for the patch variation indicator in this paper.

Figure 5 shows similarity images of three ship pixels and three sea clutter pixels, which are generated using our previous method [10], and

Table 2. P_vi of the six pixels in Figure 5.

Pixel	a	b	c	d	e	f
$P_{\mathrm{vi}}$	4.2882	3.9423	2.3387	0.1951	0.4723	0.2875

shows their $P_{\mathrm{vi}}$ values. It can be seen in Figure 5 that the patterns of similarity images of the ship target and sea clutter are different. The ship target pixels are different from the background, while the sea clutter pixels are similar to the background. The patch variation indicator can reflect the contextual information of a pixel, which is used to represent the patch pattern of the ship target pixel and sea clutter pixel.

The patch size used in the patch variation indicator is set to 21 pixels in this paper according to the size of the large ships. Figure 6 shows the histogram of the $P_{\mathrm{vi}}$ of the sea regions in the synthetic data under different patch sizes. Figure 7 shows the probability density function and the cumulative distribution function of the $P_{\mathrm{vi}}$ estimated by the KDE method. It can be seen that the PDF and CDF curves of the $P_{\mathrm{vi}}$ of sea clutter under different window lengths are very close. This means that the distribution of the $P_{\mathrm{vi}}$ of sea clutter is not sensitive to changes with patch size. The threshold of $P_{\mathrm{vi}}$ is chosen from 1.2 to 1.4, and the false alarm rate is around 0.0063 according to the CDF of the $P_{\mathrm{vi}}$ of sea clutter.

4.1.3. Detection Statistic

As mentioned before, the backscattering energy of the ship is dominated by the energy of the double-bounce scattering. Figure 8 and Figure 9 show several amplitude images of a small ship and a large ship on different channels. As can be seen in Figure 9, the contrast of a small target to sea clutter is higher in the HH − VV channel than in the other channels and the contrast of a large target to sea clutter is higher in the HV channel than in the other channels.

Table 3. Peak signal-to-clutter ratios of the two ships in different image channels (SCR/dB).

Channel	HH	VV	HV	HH − VV	HH + VV
Large ship	9.9	9.3	26.8	19.6	9.7
Small ship	−1.3	−5.2	5.8	8.4	−6.6

gives the peak signal-to-clutter ratios of the ship and sea clutter in different channels. From these results, it can be seen that it is reasonable to set the detection statistic to the ensemble average of |HH − VV|·|HV|.

For a keypoint detection statistic patch, the detection threshold is set according to the value of the center pixel, namely the detection statistic value of the keypoint pixel. In this paper, the threshold is set as the detection statistic value of the center pixel multiplied by 0.55 according to the experiment.

4.2. Results of Different Ship Detection Methods

In this section, R2, R3, R4 and R5 regions in Figure 2 are used for the performance evaluation of different PolSAR ship detection methods. By observing the Pauli pseudo-color image, we manually marked 34, 46, 21 and 21 potential ship targets in Figure 10a, Figure 11a, Figure 12a and Figure 13a, respectively. The strong ship targets are marked with rectangles and the comparatively weak ship targets are marked with ovals.

We used the quality factor figure of merit (FoM) [5] to evaluate the ship detection performance of the proposed method and other methods. The FoM considers both the detection rate (PD) and false alarm rate (PFA) of the detection result. A higher FoM means a better detection performance. The FoM is defined as follows:

$$\mathrm{FoM}=\frac{N_{\mathrm{td}}}{N_{\mathrm{gt}}+N_{\mathrm{fa}}}$$

(16)

where $N_{\mathrm{td}}$ is the number of detected targets, $N_{gt}$ is the number of ground-truth targets and $N_{\mathrm{fa}}$ is the number of total false alarms. The FoM of different methods on the R2 and R3 regions are shown in

Table 4. Quantitative comparisons of different methods on R2/R3/R4/R5.

Data	Method	${\mathit{N}}_{\mathbf{gt}}$	${\mathit{N}}_{\mathbf{td}}$	${\mathit{N}}_{\mathbf{fa}}$	FoM
R2	Proposed	34	33	0	0.97
	PNF		33	>20	<0.61
	PWF		34	>18	<0.65
	RS		30	>30	<0.47
	SD-LSMDRK		33	3	0.89
	SD-SLLIM		34	4	0.89
R3	Proposed	46	45	3	0.92
	PNF		43	>30	<0.57
	PWF		45	>30	<0.59
	RS		42	>20	<0.63
	SD-LSMDRK		45	5	0.88
	SD-SLLIM		46	3	0.94
R4	Proposed	21	21	3	0.88
	PNF		21	>15	<0.58
	PWF		21	>30	<0.41
	RS		21	>40	<0.34
	SD-LSMDRK		18	4	0.72
	SD-SLLIM		21	12	0.58
R5	Proposed	21	21	5	0.81
	PNF		21	>30	<0.41
	PWF		21	>30	<0.41
	RS		19	>40	<0.31
	SD-LSMDRK		18	2	0.78
	SD-SLLIM		15	9	0.50

.

The proposed PolSAR-SIFT ship detection method is compared with the PWF [1], the RS [2], the PNF [3] and two state-of-the-art methods, namely the SD-LSMDRK [7] and the SD-SLLIM [8]. As revealed by our previous work [6], the ship detection comparison of these methods mainly lies in the ability to detect ships with a low ship–sea contrast, rather than the ships with a high ship–sea contrast. Therefore, we chose appropriate detection thresholds for the above methods, except for the SD-SLLIM, which implemented ship target detection through a linear SVM classifier [8], to detect as many ships as possible. These thresholds were carefully chosen to reach a trade-off between few false alarms (FAs) and more detected ships.

The scenes delineated by R2, R3, R4 and R5 in Figure 2 are displayed in Figure 10a, Figure 11a, Figure 12a and Figure 13a, which are the Pauli pseudo-color images of the R2, R3, R4 and R5 regions, respectively. The keypoint extraction results of the four regions are shown in Figure 10b, Figure 11b, Figure 12b, Figure 13b respectively. The detection results of different methods on the R2, R3, R4 and R5 regions are displayed in Figure 10c–h, Figure 11c–h, Figure 12c–h, Figure 13c–h respectively, and the main FAs are marked by red ovals. Before ship detection, sea and land segmentation is usually performed first, and then the land area is masked. Therefore, this paper does not consider the land part when counting the number of false alarm targets. There may be isolated strong scattering points in the sea clutter region that can cause false alarms in target detection. This kind of false alarm may affect the evaluation of detection performance in the object level. Therefore, morphological filtering has often been used to remove these isolated strong points to reduce false alarms in many previous papers. However, considering that the small ship may only be composed of a few pixels, this paper does not perform morphological filtering on the binary image of the ship detection, which may cause the missed detection of the small ship.

Figure 10 shows the detection results of different methods in the R2 region. The proposed method has only 1 missed detection and no false alarms with the highest FoM of 0.97, while other methods have more false alarms and lower FoMs, which verifies the performance advantages of the proposed method. Figure 11 shows the detection results of different methods in the R3 region. The proposed method has 1 missed detection and 3 false alarms with an FoM of 0.92, while the SD-SLLIM has a better performance with the highest FoM of 0.94. Figure 12 shows the detection results of different methods in the R4 region. The proposed method has no miss detection and three false alarms with the highest FoM of 0.88. Figure 13 shows the detection results of different methods in the R5 region. The proposed method has no miss detection and five false alarms with the highest FoM of 0.81.

Compared with the traditional methods, such as the PWF, PNF and RS, the proposed method uses the semantic information of the corner shape structure on the ship in the PolSAR image, and extracts the corner points in the image through the PolSAR-SIFT keypoint detector. This method eliminates the sea clutter regions that do not contain corner shape structures and locates the potential ship target regions. The traditional methods present a few more FAs. The proposed method achieves comparable performance with the supervised method SD-SLLIM in the R3 region and better performance in the R2, R4 and R5 regions.

Compared with the detection performance on the RADARSAT-2 dataset, the detection performance of the SD-SLLIM method decreased on the AIRSAR dataset, as shown in Table 4. We think that there are two main reasons for this performance difference. One reason is that, as a supervised detection method based on the SVM classifier, the selection of training samples has a great impact on the detection performance, and it is hard to find the most representative training samples. Another reason is that the SD-SLLIM method performs energy normalization on the polarimetric covariance matrix when extracting features. The original intention of this is to use the difference in the polarimetric scattering mechanism between weak targets and sea clutter to improve the detection performance of weak targets. When the polarimetric scattering mechanism of the clutter in the scene is complex, this energy normalization operation may make the clutter that can be filtered out by the energy difference indistinguishable.

To sum up, the proposed method had a robust detection performance on all three data sets, two of which are spaceborne RADARSAT-2 PolSAR data and one of which is airborne AIRSAR PolSAR data. Moreover, the sea condition of the AIRSAR data is relatively more complex than that of the RADARSAT-2 data. Compared with the existing detection methods, the method proposed in this paper shows a close detection performance to the supervised detection method SD-SLLIM in the R3 region, and shows the best performance in the R2/R4/R5 regions.

5. Conclusions

Ship detection, especially for small ships, using PolSAR images is a difficult and interesting research field at present. Based on the experience of previous work, we find that the SIFT keypoint detector can be used for ship detection in PolSAR data. However, since the traditional SIFT keypoint detector is mainly for processing optical data, it is not competent for the detection tasks in PolSAR data, such as the detection of small targets. Therefore, inspired by the SAR-SIFT keypoint detection algorithm, we proposed a keypoint detection algorithm suitable for ship detection in PolSAR data, and named it the PolSAR-SIFT keypoint detector. To adapt to PolSAR data, we introduced an improved gradient based on the Riemannian distance of exponentially weighted averages of the polarimetric covariance matrix. The gradients by RDEWA are utilized to construct the PolSAR-Harris function. Keypoint detection can be achieved using the PolSAR-Harris response function image. Then, the keypoints located in the sea area are eliminated by setting a CFAR threshold on the patch variation indicator of each keypoint, and the potential target regions are selected according to the filtered keypoints. Finally, we obtained binarized ship detection results with detection statistics constructed based on the double-bouncing scattering components.

The performance of the proposed method was evaluated by two RADARSAT-2 datasets and one AIRSAR dataset. Compared with some existing methods, the proposed method had much less false alarms and could achieve a better detection performance on weak targets.

However, there are still many deficiencies in this work, and further investigation work should be carried out in the following aspects:

(1)

All the targets in the ground truth are manually marked based on the visual judgment of the Pauli pseudo-color images and analysis of scattering patterns. In order to more accurately verify the detection performance of the proposed method, in the future, we will try to obtain more accurate ground truth through AIS verification, optical picture-assisted verification, etc.

(2)

The two data sets used in this paper are both under low- and middle-sea conditions. The method should be validated on more data with complex sea conditions in the future, with a view to identifying deficiencies and improving them.