JP5310027B2 - Lane recognition device and lane recognition method - Google Patents

Description

  The present invention relates to a lane recognition device and a lane recognition method.

  The brightness change part in the image including the lane (lane mark) is extracted as an edge point, the direction angle of the extracted edge point is calculated, and the brightness change part whose filing frequency of the direction angle is a predetermined value or more is estimated as the lane. There was a lane recognition method to perform (see Patent Document 1).

Japanese Patent No. 4016735

By the way, double white lines, tire slip marks, water films, and the like may exist on the road surface. That is, the edge points extracted from the image data may include edge points other than the traffic line (white line). Therefore, if the lane is recognized based only on the edge points detected in error, the lane recognition accuracy is improved. Will drop.
An object of the present invention is to improve the accuracy of lane recognition.

The lane recognition device according to the present invention detects an edge point whose density change is equal to or greater than a threshold in an image obtained by imaging a traveling road, generates a road model of the traveling road, and generates the detected edge point. The effective edge point is selected by comparing with, and the road model is updated based on the selected effective edge point. The selection of the effective edge point is performed based on at least the angular deviation among the coordinate deviation and the angular deviation between the edge point and the road model. When selecting an effective edge point based on the angle deviation, a straight line passing through the edge point and orthogonal to the road model is drawn, and the tangent angle at the intersection of this line and the road model, and the angle deviation of the vector angle at the edge point Is smaller than the angle threshold, the edge point is selected as an effective edge point.

  According to the lane recognition device of the present invention, the effective edge point is selected based on the comparison between the detected edge point and the road model, that is, based on at least one of the coordinate deviation and the angle deviation between the edge point and the road model. By updating the road model based on the effective edge points, the accuracy of lane recognition can be improved.

It is a schematic structure of a lane recognition device. Indicates the camera position. It is a road model. It is a flowchart of an image processing program. The coefficient of Sobel operator used for edge point detection is shown. Indicates the angle of the edge point. A screen coordinate system and a road coordinate system. The outline of an edge filtering process is shown. It is a schematic structure of the lane recognition apparatus which shows 2nd embodiment. It is a flowchart of the image processing program which shows 2nd embodiment. It is a schematic structure of the lane recognition apparatus which shows 3rd embodiment. It is a flowchart of the image processing program which shows 3rd embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<< First Embodiment >>
"Constitution"
FIG. 1 is a schematic configuration of a lane recognition device.
The lane recognition device 102 is a parameter of the road model 108 that represents the road shape based on the image memory 103 that stores the road image ahead of the vehicle acquired by the camera 100 mounted on the vehicle, the steering angle detection means 104, and the vehicle speed detection means. For example, a processor 106 using a microprocessor.

  The processor 106 processes the road image stored in the image memory 103 to detect the coordinates and angle of the white line shading edge point, and detects the coordinates and angle of the detected edge point. A coordinate conversion unit 110 that converts to a coordinate system on the road with the vehicle traveling direction as one axis (road coordinate system), and the coordinates and angles of the shaded edge points that have been coordinate-converted are compared with the road model 108, and an effective travel lane An edge filtering unit 111 that selects only the edge points of the vehicle dividing line, and a road model update unit 112 that updates the road model 108 based on the selected effective edge points. Furthermore, a road model generation unit 107 that generates an initial road model from the rudder angle sensor 104 and the vehicle speed sensor 105 at the start of lane recognition is provided. The processor 106 implements the above-described units by a program.

FIG. 2 shows the camera position.
The vehicle 200 is provided with a camera 100 that captures a road surface ahead of the vehicle, a steering angle sensor 104, a vehicle speed sensor 105, and a lane recognition device 102 that recognizes the lane based on them. The camera 100 is preferably installed at a position as high as possible in the center of the vehicle, and is installed slightly downward from the horizontal so that a horizontal line and a wide range of traffic dividing lines can be photographed.

FIG. 3 shows a road model.
The road model is a mathematical expression of the lane shape, and includes at least the radius of curvature of the road, the width of the lane, and the yaw angle of the vehicle with respect to the lane centerline as its parameters. It is an equation.
The vehicle 200 faces a direction rotated by φ around the Yw axis with respect to the center line 1201 of the lane. At this time, the right traffic dividing line is expressed by the following equation.

一方、左側の通行区分線は、次式で表される。

On the other hand, the left traffic dividing line is expressed by the following equation.

Here, R is the curvature radius of the road, φ is the vehicle yaw angle with respect to the lane extending direction, and W is the lane width. In each of the above equations, it is assumed that the road gradient is horizontal, and Yw = 0.
FIG. 4 is a flowchart of the image processing program.
First, in step S1, the rudder angle is read.
In subsequent step S2, the vehicle speed is read.
In the subsequent step S3, an initial road model is generated by calculating a trajectory from which the vehicle will travel in accordance with the steering angle and the vehicle speed.
In a succeeding step S4, an image captured by the camera 100 is read into the image memory 103.
In subsequent step S5, the Sobel operator detects an edge point where the density change is equal to or greater than a threshold in the captured image.

FIG. 5 shows the coefficients of the Sobel operator used for edge point detection.
In the detection of edge points, differential images in the horizontal and vertical directions are obtained by Sobel operators 400 and 401 with respect to the captured image. In addition, even if it is not a Sobel operator, it may replace with arbitrary processes, if the density change of a horizontal and vertical direction can be extracted.
First, the horizontal and vertical differential values of the pixel of interest (x, y) are Gx and Gy, respectively, and the edge strength G is calculated according to the following equation. When the edge strength G exceeds a predetermined threshold value greater than at least 0, the pixel is determined to be an edge point.
G = √ (Gx ² + Gy ² )

FIG. 6 shows the angle of the edge point.
The traffic dividing line 500 is written in white or the like having a luminance value larger than that of the road surface 501, and the angle θ of the edge point 502 is counterclockwise with respect to the downward axis of the image, and its value range is 0 rad or more and less than 2π rad. To do. The angle θ of the edge point is defined by the following equation according to the magnitudes of Gx and Gy. Note that when Gx = 0 and Gy = 0, the edge strength G is less than the threshold and is not determined as an edge point, so the edge angle θ is not calculated.
When Gx> 0 θ = π−tan ⁻¹ (Gy / Gx)
When Gx <0 and Gy ≧ 0, θ = tan ⁻¹ (−Gy / Gx)
When Gx <0 and Gy <0, θ = tan ⁻¹ (−Gy / Gx) + 2π
When Gx = 0 and Gy> 0, θ = π / 2
When Gx = 0 and Gy <0, θ = 3π / 2
The angles of the edge points calculated in this way are opposite to each other at the left and right edges of the traffic dividing line as shown in FIG.

In the subsequent step S6, the coordinates and angles of the edge points are converted into a road coordinate system.
FIG. 7 shows a screen coordinate system and a road coordinate system.
In the screen coordinate system, as shown in FIG. 7A, the right direction of the imaging surface 600 is the Xs axis, the downward direction is the Ys axis, and the upper left corner of the imaging surface is the origin. The horizontal and vertical resolutions of the camera are Sx and Sv, respectively, and the center of the camera lens is the center of the imaging surface (Sx / 2, Sy / 2). In the road coordinate system, as shown in FIG. 7B, the lens center of the camera 100 attached to the vehicle 200 is the origin, the vehicle traveling direction is the Zw axis, the vehicle left direction is the Xw axis, and the vertical direction is the Yw axis. . Assuming that the road surface is horizontal, the Yw coordinate component of the edge point in the road coordinate system is always zero. Therefore, the coordinates of the edge point in the screen coordinate system can be converted into the road coordinate system according to the following formula.

Here, α, β, and γ are the pitch angle, yaw angle, and roll angle of the camera, and are rotation angles around the Xw axis, the Yw axis, and the Zw axis, respectively. tx, ty, and tz are coordinates of the lens center of the camera in the road coordinate system. fh and fv are parameters for perspective transformation, and are constants determined by the characteristics of the camera.
In subsequent step S7, by comparing the coordinates and angles of the edge points in the road coordinate system with the road model described in the road coordinate system, the edge points deviating from the road model are excluded as invalid edge points, and Only edge points are selected as effective edge points.

FIG. 8 shows an outline of the edge filtering process.
Here, two methods for edge filtering will be described.
First, when a straight line that passes through the edge point 700 and is orthogonal to the road model 701 is drawn, a distance d from the edge point 700 to the intersection point P with the road model 701 is obtained. Is a method of excluding this edge point 700 as an invalid edge point. When the distance d is smaller than the coordinate threshold value Dth, the edge point 700 is selected as an effective edge point.

  Second, an angle θd formed between the tangent vector 703 of the road model at the intersection P and the vector 702 of the edge point 700 is obtained. If this angle θd is equal to or larger than the angle threshold θth, this edge point In this method, 700 is excluded as an invalid edge point. When the angle θd is smaller than the angle threshold, the edge point 700 is selected as an effective edge point.

  Note that the tangent vector 703 of the road model representing the boundary on the left side of the traveling lane is Zw in order to extract the edge on the lane side in which the vehicle is traveling among the left and right edges of the traffic dividing line having a certain width. The tangent vector 704 of the road model representing the right boundary of the traveling lane in the negative direction of the axis is in the positive direction of the Zw axis. By the above edge filtering processing, the edge points that remain without being excluded become effective edge points.

In the subsequent step S8, the effective edge point group is fitted to the aforementioned circle equation by the least square method, and the equation is updated as a new road model.
In the subsequent step S9, an end determination is made. When the process is continued, the process returns to step S4, and when the process is ended, the process returns to a predetermined main program.
Note that the processes in steps S4 to S9 are repeatedly executed in accordance with the imaging cycle of the camera 100.

<Action>
The camera 100 captures the traveling road ahead of the vehicle, detects an edge point where the density change is greater than or equal to the threshold value in the captured image (step S5), and compares this with the road model to determine the lane (traffic division line) ) And the road model is updated (step S8).
By the way, double white lines, tire slip marks, water films, and the like may exist on the road surface. That is, the edge points extracted from the image data may include edge points other than the traffic lane marking lines, and if the road model is updated based on the edge points detected in error, the accuracy of lane recognition Will drop.

In this embodiment, when comparing the detected edge point with the road model, only the effective edge point is selected (step S7), and the road model is updated based on the selected effective edge point. Thereby, the precision of lane recognition can be improved.
The selection of the effective edge point is performed based on one of the coordinate deviation and the angle deviation between the detected edge point and the road model (FIG. 8).

First, a straight line that passes through the edge point and is orthogonal to the road model is drawn to obtain a distance d from the edge point to the intersection point P with the road model, and a distance d smaller than the coordinate threshold Dth is selected as an effective edge point. . Alternatively, the angle θd formed by the tangent vector of the road model at the intersection P and the vector of the edge point is obtained, and the one having the angle θd smaller than the angle threshold θth is selected as an effective edge point. That is, by excluding edge points that deviate from the road model as noise (invalid edge points), only effective edge points can be accurately filtered.
On the other hand, the road model is generated based on the steering angle and vehicle speed of the host vehicle (step S3). Thereby, a highly accurate road model can be easily generated.

《Application example》
In this embodiment, the effective edge point is selected based on one of the coordinate deviation and the angle deviation between the edge point and the road model. However, the present invention is not limited to this, and both the coordinate deviation and the angle deviation are selected. Edge filtering may be performed based on the above.

"effect"
As described above, the camera 100 corresponds to the “imaging unit”, the edge coordinate / angle detection unit 109 and the process in step S5 correspond to the “detection unit”, and the road model generation unit 107 and the processes in steps S1 to S3 are performed. Corresponding to “generating means”, the edge filtering processing unit 111 and the process in step S7 correspond to “selecting means”, and the road model updating unit 112 and the process in step S8 correspond to “updating means”.

(1) An imaging unit that images a traveling road, a detection unit that detects an edge point where a density change is equal to or greater than a threshold in an image captured by the imaging unit, a generation unit that generates a road model of the traveling road, A selection unit that selects an effective edge point by comparing an edge point detected by the detection unit with a road model generated by the generation unit, and updates the road model based on the effective edge point selected by the selection unit. Updating means, and the sorting means sorts the effective edge points based on at least one of a coordinate deviation and an angle deviation between the edge point detected by the detecting means and the road model generated by the generating means. .
As described above, the effective edge point is selected by comparing the detected edge point with the road model, and the road model is updated based on the selected effective edge point, thereby improving the accuracy of the lane recognition. Also, only effective edge points can be accurately filtered.

(2) The selecting means draws a straight line passing through the edge point and orthogonal to the road model, and when the distance from the intersection of the straight line and the road model to the edge point is smaller than a coordinate threshold value, The edge point is selected as the effective edge point.
As a result, edge points that deviate from the road model can be excluded as noise, and only effective edge points can be accurately filtered.

(3) The selecting means draws a straight line that passes through the edge point and is orthogonal to the road model, and the angle deviation of the tangent angle at the intersection of the straight line and the road model, and the vector angle at the edge point is used for the angle. When the value is smaller than the threshold value, the edge point is selected as the effective edge point.
As a result, edge points that deviate from the road model can be excluded as noise, and only effective edge points can be accurately filtered.

(4) The generation unit generates the road model based on a steering angle and a vehicle speed of the host vehicle.
Thereby, a highly accurate road model can be easily generated.
(5) It is effective to detect an edge point whose density change is greater than or equal to a threshold value in an image obtained by capturing a road and to generate a road model of the road and compare it with the road model that generated the detected edge point. Edge points are selected, and the road model is updated based on the selected effective edge points.
As described above, the effective edge point is selected by comparing the detected edge point with the road model, and the road model is updated based on the selected effective edge point, thereby improving the accuracy of the lane recognition.

<< Second Embodiment >>
"Constitution"
In the second embodiment, an initial road model is generated based on the current position of the host vehicle and road map information, and the above-described coordinate threshold value and angle threshold value are set according to the resolution of the camera 100. .
FIG. 9 is a schematic configuration of the lane recognition device.
Instead of omitting the steering angle sensor 104 and the vehicle speed sensor 105, the lane recognition device 102 includes a GPS 114 that acquires a current position and road map information 115 that stores a road shape. The road model generation unit 807 acquires the road shape ahead of the current position obtained from the GPS 114 from the road map information 115, and generates an initial road model from the road shape of the travel path.

FIG. 10 is a flowchart of the image processing program.
In the present embodiment, the first step described above is performed except that the process of new steps S21 to S23 is executed instead of the steps S1 to S3, and the process of new step S27 is executed instead of the step S7. The same processing as in the embodiment is executed.
In step S21, the current position is acquired from the GPS 114.
In step S22, the road shape in front of the current position is acquired from the road map information 115.

In step S23, the above-described circle equation is fitted to the road shape by the least square method, and the equation is generated as a road model.
In step S27, a coordinate threshold and an angle threshold are set according to the resolution of the camera, and edge points are filtered.
The farther the edge point is, the less the resolution of the camera becomes, and an error occurs in the coordinate value. This error is proportional to the distance of the edge point seen from the vehicle, that is, the Zw coordinate component of the road coordinate system. Therefore, if the edge point is filtered with a uniform threshold, the filtering accuracy is lowered.

Therefore, the coordinate threshold value Dth is set according to the following equation so as to be proportional to the Zw coordinate component of the road coordinate system. Here, h is a proportionality coefficient, D0 is the minimum value of the threshold value Dth, and is a constant determined in advance.
Dth = h × | Zw | + D0
Similarly, the angle threshold θth is set according to the following equation. Here, k is a proportional coefficient, θ0 is the minimum value of the angle threshold θth, and is a constant determined in advance.
θth = k × | Zw | + θ0
As described above, the same process as step S7 is executed except that the coordinate threshold value Dth and the angle threshold value θth are set.

<Action>
In this embodiment, the current position is detected by the GPS 114 (step S21), and the road shape of the traveling road is acquired by referring to the road map information 115 ahead of the vehicle at the current position (step S22). A road model is generated (step S23). Thereby, an appropriate road model can be generated from the initial stage of lane recognition.
Further, the farther the edge point, the less the resolution of the camera, and the error occurs in the coordinate value. This error is proportional to the distance of the edge point seen from the vehicle, that is, the Zw coordinate component of the road coordinate system. Therefore, if the edge point is filtered with a uniform threshold, the filtering accuracy is lowered.

  Therefore, the coordinate threshold Dth and the angle threshold θth are set according to the Zw coordinate component of the road coordinate system. That is, the larger the Zw coordinate component, that is, the farther the edge point, the larger the coordinate threshold value Dth and the angle threshold value θth (step S27). As a result, the far edge points are less likely to be excluded as invalid edge points, so that erroneous edge filtering due to the resolution of the camera 100 can be prevented.

"effect"
From the above, the processing of steps S21 to S23 corresponds to “generating means”, and the processing of step S27 corresponds to “selecting means”.
(1) The selection unit sets the threshold according to the resolution of the imaging unit.
Thereby, an appropriate road model can be generated from the initial stage of lane recognition.
(2) The generation unit generates the road model based on a current position of the host vehicle and road map information.
As a result, erroneous sorting due to the resolution of the imaging means can be prevented.

<< Third embodiment >>
"Constitution"
In the third embodiment, a coordinate threshold value and an angle threshold value are set according to a change in the posture of the vehicle, it is determined whether or not the road model is appropriate, and the road model is updated as appropriate.
FIG. 11 is a schematic configuration of a lane recognition device.
The lane recognition device 102 includes an acceleration sensor 116 that detects a change in the attitude of the vehicle, and a road model verification unit 117 that determines whether or not the current road model is valid.
FIG. 12 is a flowchart of the image processing program.
The present embodiment is the same as the first embodiment described above except that the processes of new steps S31 and S32 are executed instead of step S7, and the newly added processes of steps S33 and S34 are executed. Execute the process.

In step S31, at least the acceleration in the vertical direction (including the pitching direction and the bound direction) detected by the acceleration sensor 116 is read.
In step S32, when the acceleration G is equal to or greater than the predetermined value Gth, the coordinate threshold Dth and the angle threshold θth are set larger by a predetermined amount. In this way, the same process as in step S7 is executed except that the coordinate threshold value Dth and the angle threshold value θth are set.

In a succeeding step S33, it is determined whether or not the number of valid edge points is less than a predetermined value. If the number of valid edge points is greater than the predetermined value, it is determined that the current road model is valid, and the process proceeds to step S8. On the other hand, when the number of valid edge points is less than the predetermined value, it is determined that the current road model is not valid, and the process proceeds to step S34.
In step S34, the current road model is initialized, and then the process returns to step S4.

<Action>
In the present embodiment, the acceleration sensor 116 detects the acceleration G in the vertical direction (step S31), and if the acceleration G is equal to or greater than the predetermined value Gth, it is determined that the vehicle attitude change is large, and the coordinate threshold value Dth or The angle threshold value θth is set large (step S32). Thereby, since it becomes difficult to be excluded as an invalid edge point, it is possible to prevent erroneous edge filtering due to a change in the attitude of the vehicle.

  Also, if most of the edges are excluded as invalid edge points as a result of edge filtering, the road model may not be appropriate. Therefore, the number of valid edge points is counted, and when the quantity is less than the predetermined value, it is determined that the road model is not valid, and the road model is initialized (reset) and the road model is reconstructed. (Steps S33, S34). Thereby, it can prevent that the precision of lane recognition falls.

"effect"
As described above, the processes in steps S31 and S32 correspond to “selecting means”, and the processes in steps S33 and S34 correspond to “update means”.
(1) The selection means sets the threshold according to a change in the posture of the vehicle.
As a result, it is possible to prevent erroneous sorting due to a change in the attitude of the vehicle.
(2) The updating means determines whether or not the road model is appropriate according to the number of effective edge points selected by the selecting means. initialize.
Thereby, it can prevent that the precision of lane recognition falls.

DESCRIPTION OF SYMBOLS 100 Camera 104 Rudder angle sensor 105 Vehicle speed sensor 107 Road model production | generation part 109 Edge coordinate and angle detection part 111 Edge filtering process part 112 Road model update part

Claims (8)

Image pickup means for picking up an image of the road, detection means for detecting an edge point whose density change is greater than or equal to a threshold in an image picked up by the image pickup means, generation means for generating a road model of the road, and the detection means Selecting means for selecting effective edge points by comparing the edge points detected by the generating means with the road model generated by the generating means; and updating means for updating the road model based on the effective edge points selected by the selecting means; With
The selecting means is for selecting the effective edge point based on at least an angular deviation among a coordinate deviation and an angular deviation between the edge point detected by the detecting means and the road model generated by the generating means, When selecting the effective edge point based on the angle deviation, a straight line passing through the edge point and orthogonal to the road model is drawn, a tangent angle at the intersection of the straight line and the road model, and the edge point A lane recognition device, wherein an edge point is selected as the effective edge point when an angle deviation of a vector angle is smaller than an angle threshold value.   The selecting means draws a straight line passing through the edge point and orthogonal to the road model, and when the distance from the intersection of the straight line and the road model to the edge point is smaller than a coordinate threshold value, the edge point The lane recognition device according to claim 1, wherein the lane recognition device is selected as the effective edge point. It said sorting means, according to the resolution of the imaging means, the lane recognition device according to claim 1 or 2, characterized in that setting the threshold value. The lane recognition device according to any one of claims 1 to 3 , wherein the selection unit sets the threshold according to a change in posture of the vehicle. It said generating means, the lane recognition device according to any one of claim 1 to 4, characterized in that to generate the road model based on the steering angle and the speed of the vehicle. It said generating means, the lane recognition device according to any one of claim 1 to 5, characterized in that to generate the road model based on the current position and the road map information of the vehicle. The updating means determines whether or not the road model is appropriate according to the number of effective edge points selected by the selecting means, and initializes the road model when determining that the road model is not appropriate. The lane recognition device according to any one of claims 1 to 6 , wherein: In the image of the travel road, an edge point whose density change is equal to or greater than a threshold value is detected, a road model of the travel road is generated, and a coordinate deviation and an angle deviation between the detected edge point and the generated road model are detected. Among them, the effective edge point is selected based on at least the angular deviation, and the road model is updated based on the selected effective edge point.
When selecting the effective edge point based on the angular deviation, a straight line passing through the edge point and orthogonal to the road model is drawn, a tangent angle at the intersection of the straight line and the road model, and the edge point A lane recognition method comprising: selecting an edge point as the effective edge point when an angle deviation of a vector angle at is smaller than an angle threshold value.

Images