JP2012247847A - Information transmission control device for vehicle and information transmission control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for preventing driving behavior from being hindered as a driver is attracted to unnecessary information in the driving behavior.SOLUTION: An on-vehicle camera picks up an image of a drive visual field in front of a local vehicle (S101), and a highly visually attractive area in the image of the drive visual field which attracts driver's attention strongly is estimated (S102). Then it is determined whether the estimated highly visually attractive area is information necessary for drive behavior (S103). A necessity/unnecessity is made based on the position relation between the highly visually attractive area in the image of the drive visual field and a caution area recognized by a caution area recognition part 30. When an area which is not necessary for the drive behavior and highly visually attractive is found in a drive assist image as a result of the necessity/unnecessity determination (S104: YES), a mask image which lowers the visibility of the drive visual field is displayed in a corresponding area at a mask display part which corresponds to an object area of information presentation filtering specified from the drive visual field image (S105).

Description

  The present invention relates to a technique for controlling information transmission to a vehicle occupant.

  Conventionally, for example, a vehicle information providing apparatus described in Patent Literature 1 is known as a technique for providing information related to the surrounding situation to a vehicle occupant. In this vehicle information providing apparatus, the method and intensity of information presentation are changed according to the visibility and attractiveness of an object to be watched by the driver.

  Specifically, when it is estimated that the visibility and attractiveness of the target object are low, it is presumed that information indicating the position and attributes of the target object is actively presented and the visibility and attractiveness of the target object are high. In such a case, information presentation performed by the vehicle information providing apparatus is partially suppressed. By doing so, the driver is prevented from feeling bothered when providing information to the driver.

JP 2004-30212 A

  In the above prior art, the method and intensity of information presentation are changed according to the degree of visibility and attractiveness of the object, and although it can reduce the troublesomeness of information presentation, it is essentially for the driver. There is no change in driving assistance by increasing the information transmitted. If the information transmitted to the driver increases, the load on the information processing capability of the driver will increase accordingly. In recent driving support systems, a lot of information is presented to the driver, and there is a concern that the burden on the information processing ability of the driver becomes excessive.

Therefore, it is desired to develop a technology that reduces the burden on the information processing ability of the driver by suppressing transmission of information unnecessary for the driving behavior and supports the driving behavior of the driver.
The present invention has been made to solve the above-described problem, and an object of the present invention is to provide a technique for preventing a driver from being obstructed by information unnecessary for driving behavior.

  The vehicle information transmission control device according to claim 1, which has been made to achieve the above object, is estimated to be highly attractive to the driver and unnecessary for driving behavior from image information obtained by imaging the driving field of view of the vehicle. By applying a filter that reduces the visibility from the driver to the visual information transmitted to the driver from the corresponding area of the driving field of view corresponding to the highly attractive area The transmission of the visual information to the driver is suppressed.

  According to a second aspect of the present invention, there is provided a vehicle information transmission control device comprising an attractive area specifying unit, a necessity determining unit, and a filtering unit. The attractive area specifying means specifies a highly attractive area that is estimated to have high attractiveness to the driver from image information obtained by imaging the driving field of view of the vehicle. The necessity determination means determines whether or not the visual information included in the highly attractive area specified by the attractive area specifying means is necessary for the driving action. As a result of the determination by the necessity determination unit, the filtering unit performs driving on the visual information transmitted to the driver from the corresponding region of the driving field of view corresponding to the highly attractive region determined to be unnecessary for the driving action. Apply a filter that reduces visibility from the user.

  According to the vehicle information transmission control device of the present invention, the visibility of visual information judged to be unnecessary for driving behavior and highly attractive among various visual information brought to the driver from the outside is reduced. By doing so, transmission of such visual information can be suppressed. Accordingly, it is possible to prevent the driver from being attracted to an object to which attention should not be paid while driving, and to concentrate the driver on driving behavior. Further, the present invention suppresses transmission of unnecessary information from the outside, in other words, unnecessary information, unlike the conventional case where driving assistance is provided by actively providing information to the driver. The burden on the information processing ability of the driver can be reduced by thinning out the.

  By the way, as a method for identifying a region having high attractiveness from image information obtained by imaging the driving field of view of a vehicle, image saliency is derived based on at least one of luminance, color, gradient, and motion features in the image information. It is conceivable to specify a highly attractive region based on the saliency (claim 3).

  In human vision, attention is drawn to areas where spatiotemporal changes in various features such as the color and intensity of light, the rate of change from the surroundings (gradient), and temporal change (motion) are noticeable (attracted) ) It is known to have work. Therefore, from the image information obtained by capturing the driving field of view, based on the feature amount such as the brightness, color, gradient, movement, etc. of the image, by specifying the portion with the high saliency of the spatiotemporal change as the highly attractive region, It is possible to accurately identify an object that is easily attracted by the driver.

  Next, in the invention according to claim 4, the visual attraction included in the high attraction area according to the positional relationship between the attention area to be noted by the driver in the image information of the driving field of view and the high attraction area. It is characterized by determining whether information is required for driving behavior.

  When driving a vehicle, it is necessary to determine what it is and pay attention to everything that is visible. It is important to pay attention to the object. From such a viewpoint, according to the positional relationship between the region that the driver should be aware of in the image information obtained by capturing the driving field of view and the highly attractive region, the visual information included in the highly attractive region is the driving behavior. It is good to judge whether it is necessary.

  The region to be noted by the driver here includes, for example, a predetermined range such as in front of the traveling direction of the vehicle, a range in which an object detected as an object to be noted exists, and the like. Then, when the position of the highly attractive area detected from the image information corresponds to the position of the attention area, it is determined that the highly attractive area is necessary for the driving action, and the position of the attention area is set. If not applicable, it may be determined that the highly attractive region is unnecessary for driving behavior.

  Furthermore, the invention according to claim 5 includes a recognizing means for recognizing a cautionary object including a traffic participant or a specific road structure existing in the driving field of view of the host vehicle, and the necessity determining means is a recognizing means. The region in the image information in which the attention object recognized by is used as the attention region, and the highly attractive region in the range excluding the attention region is determined to be unnecessary for the driving action.

  By doing so, the range in which traffic structures such as traffic signals such as pedestrians and other vehicles, traffic signs, traffic signs, road markings, road shapes, etc. that should be noticed by the driver are detected. In such driving behavior, it is possible to reduce the transmission of visual information that is easily attracted to other objects while keeping the visual information of the object to be noted by the driver. This makes it easier for the driver to concentrate attention on traffic participants and road structures that should be noted in driving behavior.

  By the way, as a filtering means that reduces the visibility of visual information that is unnecessary for driving behavior and has high attractiveness, in a display means that includes a display area that is provided over the driving field of view visible to the driver, the filtering target It can be considered that a mask image for reducing the transmittance is displayed in the display area corresponding to the position of the visual information.

  Examples of the display means used for such filtering means include a transparent windshield display that is attached to or used in the vicinity of a vehicle windshield, or a head-mounted display that is mounted on the driver's head. It is done. When such a display device is used, the filtering of visual information is realized by converting the position of visual information to be filtered into the position of the driving field of view as viewed from the driver and presenting the mask at the position.

  Further, it is conceivable to devise a mask image display method by filtering means. Specifically, the filtering means displays the mask image with the transmittance continuously increased from the center of the display range in which the mask image is to be displayed toward the boundary between the display range and the non-display range ( Claim 7). By doing so, it is possible to reduce the difference in transmittance between the display range of the mask image and the non-display range where the mask image is not displayed, thereby making the boundary inconspicuous. By doing so, it is possible to prevent the driver from being attracted to the boundary between the display range and the non-display range when the mask image is displayed, and to reduce the uncomfortable feeling given to the driver by displaying the mask image.

  Furthermore, as a device for the display method of the mask image, the level of contrast on the image information of a predetermined area to be noted for driving behavior and the level of contrast on the image information of visual information to be filtered Accordingly, a configuration may be adopted in which the degree of transmittance of the mask image is set, and the mask image is displayed with the set transmittance (claim 8).

  For example, if the transmittance is set so that the visual information filtered by the mask image is relatively inconspicuous than the predetermined area to be noted for driving behavior, the information on the area unnecessary for driving behavior is attracted. It is thought that it becomes easy to recognize the information of the area to be careful for driving behavior without being done.

  For this purpose, a state is assumed in which the transmittance is lowered by filtering with a mask image based on the contrast between the contrast of a predetermined area to be noted for driving behavior and the contrast of visual information to be filtered. What is necessary is just to set the transmittance | permeability of a mask image so that the contrast of the said visual information may become lower than the contrast of the predetermined area | region which should be careful for driving action.

  Alternatively, the filtering unit may be configured to set the degree of transmittance of the mask image according to the degree of speed of the host vehicle and display the mask image with the set transmittance. (Claim 9).

  For example, when the vehicle is traveling at a low speed, there is a time margin for the driver to recognize the visual information and reflect it in the driving behavior (the burden on the driver is small). Therefore, even if you are attracted to various objects, there is no big problem because the driver's judgment for driving behavior is in time. Therefore, during low speed traveling, for example, it is conceivable to improve the visibility of the entire driving field of view by setting the transmittance of the mask image used for filtering to be relatively high. Further, at the time of stopping, the transmittance may be maximized (no filtering effect).

  On the other hand, when the vehicle is traveling at a high speed, the time that can be spent for judgment for driving behavior is shortened (the burden on the driver is large). Therefore, there is a risk that the judgment necessary for the driving action may not be in time even with a little looking aside. Therefore, during high-speed driving, for example, it is conceivable that the transmittance of the mask image used for filtering is set to a relatively low level so as to avoid being attracted as much as possible outside the area necessary for driving behavior.

  In this way, by setting the level of the transmittance of the mask image in accordance with the level of the speed of the host vehicle, the information processing capability of the driver can be improved in a mode corresponding to the level of the driving burden due to the speed of the vehicle. The burden can be reduced.

  Next, an information transmission control method according to claim 10 includes an attractiveness area specifying procedure for specifying a highly attractive area that is estimated to have high attractiveness to the driver from image information obtained by imaging the driving field of view of the vehicle. The necessity determination procedure for determining whether or not the visual information included in the highly attractive region identified in the attractive region identification procedure is necessary for the driving behavior, and the result of the determination in the necessity determination procedure, the driving behavior A filtering procedure for applying a filter that reduces visibility from the driver to visual information transmitted to the driver from the corresponding region of the driving field of view that is determined to be unnecessary for the driver. It is characterized by having. According to the information transmission control method having such a procedure, the effects described above for the vehicle information transmission control device of the present invention can be obtained.

It is a block diagram which shows the system configuration | structure of the vehicle information transmission control apparatus of embodiment. It is explanatory drawing which shows the outline | summary of the saliency map for estimating attractiveness. It is explanatory drawing which shows the outline | summary of the necessity determination of a highly attractive region. It is explanatory drawing which shows the outline | summary of information presentation filtering. It is explanatory drawing which shows the outline | summary of the display method of a mask image. It is a flowchart which shows the procedure of a main process. It is a flowchart which shows the procedure of an attractiveness estimation process. It is explanatory drawing which shows the outline | summary of a resolution pyramid (Gaussian pyramid). It is a flowchart which shows the procedure of the necessity determination process. It is a flowchart which shows the procedure of an attention area estimation process. It is a flowchart which shows the procedure of an information presentation filtering process. It is a flowchart which shows the procedure of exception processing. It is a flowchart which shows the procedure of the transmittance | permeability setting process. It is explanatory drawing which shows the analysis result of the gaze distribution at the time of a driving | operation. It is explanatory drawing which shows the verification result of the effect by information presentation filtering. It is explanatory drawing which shows the verification result of the effect by information presentation filtering. It is explanatory drawing which shows the verification result of the effect by information presentation filtering. It is explanatory drawing which shows balance with information presentation filtering, a room mirror, and a side mirror.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment at all, and can be implemented in various aspects.
[Description of system configuration of vehicle information transmission control device]
A system configuration of a vehicle information transmission control device to which the present invention is applied will be described with reference to FIG. The vehicle information transmission control device of the present embodiment is a driving support system mounted on a vehicle, and as shown in FIG. 1, an image acquisition unit 10, an attractiveness estimation unit 20, a caution area recognition unit 30, The necessity determination part 40, the transmittance | permeability change switch 50, and the filtering part 60 are provided.

  The image acquisition unit 10 is configured by imaging means such as an in-vehicle camera that images the front of the host vehicle. This in-vehicle camera is a well-known camera constructed using a CCD image sensor, a CMOS image sensor, and a lens for condensing light from a subject on the image sensor, and the driving field of view in front of the vehicle seen from the driver. It arrange | positions toward the front of the own vehicle so that the whole may be contained in an imaging range.

  The attractiveness estimation unit 20, the attention area recognition unit 30, and the necessity determination unit 40 are configured as a known information processing apparatus (for example, a microcomputer) including a CPU, a ROM, a RAM, an I / O, and the like.

  Among these, the attractiveness estimation unit 20 has a function of analyzing the captured image input from the image acquisition unit 10 by image processing and specifying a highly attractive region that is estimated to have high attractiveness to the driver. In order to estimate attractiveness, a “saliency map” (Laurent Itti, Christof Koch, Ernst Niebur, “A model of saliency-based visual attention for rapid scene” was devised as a mechanism for finding objects that attract human attention. analysis, "IEEE Trans. Pattern Analysis and Machine Intelligence, vol.20, no.11, pp.1254-1259, 1998.).

  Here, an outline of the saliency map will be described with reference to FIG. As shown in FIG. 2, the saliency map is a characteristic of each element that is intrigued by humans, such as the colors (Colors), luminance (Intensity), gradients (Orientations), and motion (Motion). A feature map (Feature Map) is created for each of them, and they are overlapped so that a place where attention is drawn on the image can be specified. The feature map used to generate the saliency map is called Center-Surround Operation, which uses a pair of different resolutions to obtain the difference from the image group (Gaussian pyramid) obtained by repeating the image resolution reduction process step by step. Created by technique.

  Returning to the description of FIG. The attention area recognizing unit 30 selects information necessary for driving behavior from the captured image input from the image acquisition section 10, that is, a warning area in which an object and a range to which the driver should be careful during driving behavior are shown. recognize. In this embodiment, traffic objects such as pedestrians, bicycles, two-wheeled vehicles, automobiles, and traffic structures such as traffic signals, traffic signs, road markings, road shapes, and white lines are recognized as objects and ranges to be recognized as attention areas. Is assumed. In order to recognize these attention areas, in addition to image recognition from the captured image by the image acquisition unit 10, object recognition using a millimeter wave radar or laser radar (not shown), a vehicle such as a steering angle of a steering wheel, etc. Traveling path recognition (traveling direction determination) using information may be used.

  The necessity determination unit 40 determines whether the highly attractive region is necessary for driving behavior from the positional relationship between the highly attractive region estimated by the attractive region estimating unit 20 and the attention region recognized by the attention region recognition unit 30. It is determined whether or not.

  Here, an outline of necessity determination executed in the necessity determination unit 40 will be described with reference to FIG. In the necessity determination, first, as shown in the procedure of FIGS. 3 (1) to (4), the attention area recognition unit 30 recognizes the attention area on the captured image of the driving field of view, and then the determination of FIG. In the procedure, it is determined from the positional relationship between the attention area and the highly attractive area whether or not the highly attractive area is information necessary for driving behavior.

  Fig. 3 (1) Pedestrian recognition: For safe driving, it is necessary to know the presence and location of pedestrians. Therefore, a pedestrian present in the captured image of the driving field of view is detected and recognized as a caution area necessary for driving behavior.

  Fig. 3 (2) Vehicle recognition: For safe driving, it is necessary to know the existence and position of other vehicles (including motorcycles). Therefore, the vehicle existing in the captured image of the driving field of view is detected and recognized as a caution area necessary for driving behavior.

  FIG. 3 (3) Signal / Signal Recognition: Traffic signals and traffic signs are indispensable information for selecting the driving method of the vehicle. Therefore, a traffic signal or a traffic sign existing in the captured image of the driving field of view is detected and recognized as a caution area necessary for driving behavior.

  FIG. 3 (4) White line / travel road recognition: The white line and road markings provided on the travel path, or the shape of the travel path are essential information for determining the speed and steering angle of the host vehicle. Therefore, when a white line, a road marking, or a travel path is detected in the captured image of the driving field of view, the area including them is recognized as the attention area necessary for the driving action.

  FIG. 3 (5) Necessity determination of a region with high attractiveness: The attention region recognized in the above (1) to (4) has a high necessity for driving behavior, while other regions have a necessity for driving behavior. Relatively low. Therefore, the necessity of the high attractiveness region is determined based on the estimation result of the high attractiveness region by the attractiveness estimation unit 20 and the recognition result by the attention region recognition unit 30. As a result, a region that is unnecessary for driving behavior other than the attention region and that corresponds to the highly attractive region (region surrounded by a dotted line in FIG. 3 (5)) is extracted, and this region is the target of information presentation filtering. To do.

  Upon receiving the result of necessity determination by the necessity determination unit 40, the filtering unit 60 displays a mask image for reducing visibility in a corresponding region of the driving field of view corresponding to the region to be subjected to information presentation filtering, and the region. The transmission of visual information from the driver to the driver is suppressed (FIG. 3 (6): filtering).

  Returning to the description of FIG. The transmittance change switch 50 is an operation switch for the user to set the strength of information presentation filtering. By operating this transmittance change switch 50, the setting of “transmittance: high” can be validated. If the “Transmittance: High” setting is enabled, the transmittance of the mask image displayed in the range subject to information presentation filtering can be made higher than usual, which allows information to be displayed as needed. The strength of the presentation filtering can be intentionally reduced.

  The filtering unit 60 includes a windshield display (WSD) provided along the front window (windshield) of the vehicle, and a mask image display unit including a head mounted display (HMD) mounted on the driver's head, The display device includes a control unit that controls the operation of the display unit.

  FIG. 4A is an example of a mask display unit, and shows an example using a WSD provided along the front window. The WSD is an image presentation device for providing visual information support to the driver, and can display information in a wide range on the front window. This is a see-through information presentation means that can display a CG image superimposed on the actual driving field of view through the front window. In WSD, a real scene is directly viewed through a screen, and there is an advantage that a virtual image can be superimposed on the real scene.

  The filtering unit 60 of the present embodiment may be configured to share one WSD for general information presentation and display of a mask image for information presentation filtering, or a general display WSD. In addition, a configuration may be provided in which a WSD dedicated to mask image display is provided separately.

  FIG. 4B is a display example of a mask image on a display unit such as WSD or HMD. The position of the filtering target area specified on the captured image of the driving field of view is converted into a display range in WSD, HMD, or the like, and a mask image is displayed in the corresponding area. The mask image does not completely block the transmission of visual information, but uses, for example, a black mesh image that lowers the light transmittance and partially suppresses the transmission of visual information. Further, the transparency of the mask image is changed according to the contrast of the highly attractive area and other external circumstances.

  Next, a specific method for displaying a mask image will be described with reference to FIG. The display of the mask image is performed through each procedure of local area extraction shown in FIG. 5A, labeling shown in FIG. 5B, and mask area setting shown in FIG. 5C. The contents of each procedure are as follows.

  (A) Extraction of a local area: An area that is unnecessary for driving behavior other than the attention area and that has a degree of saliency in the saliency map generated by the attractiveness estimation unit 20 that is greater than a preset threshold is extracted. .

  (B) Labeling: A labeling process is performed on the local region group extracted by the procedure (a). Here, the same identification number is assigned to each successive local region, and a local region group having the same identification number is regarded as one region. Further, the center of gravity position and the vertical and horizontal sizes of each region obtained by the labeling process are extracted.

  (C) Mask area setting: The transmittance distribution of the mask image based on the two-dimensional Gaussian distribution is set in accordance with the barycentric position and the vertical and horizontal sizes of each area obtained in the procedure of (b) above. In the model of the two-dimensional Gaussian distribution, the probability is higher as it is closer to the center (= the center of gravity position of the region), and the probability is lower as the peripheral region is far from the center. That is, the higher the probability in the two-dimensional Gaussian distribution model, the lower the transmittance of the mask image, and the lower the probability, the higher the transmittance of the mask image. In this way, by continuously increasing the transmittance from the center to the periphery of the mask image and making the transmittance of the boundary portion substantially the same as the periphery, the driver pays attention to the boundary of the mask image. It can be prevented from being drawn.

[Description of main processing]
Next, the procedure of main processing executed in each part of the vehicle information transmission control device of the embodiment will be described with reference to the flowchart of FIG. This process is repeatedly executed every predetermined time while the vehicle is traveling.

  First, the image acquisition unit 10 acquires a driving field image in front of the host vehicle (S101). FIG. 6A is an example of a driving view image. Further, detection information from radar sensors for detecting traffic participants such as pedestrians and other vehicles is also acquired.

  Next, the attractiveness estimation part 20 performs the attractiveness estimation process which estimates the high attractiveness area | region which draws a driver | operator's attention strongly on the driving | operation visual field image acquired by S101 (S102). The attractiveness is estimated by image-processing the driving view image and creating a saliency map. FIG. 6B is an example of the saliency map created from the driving view image. In this saliency map, the degree of attractiveness is expressed as the shading of the image. Details of the attractiveness estimation process will be described later.

  Next, the necessity determination part 40 performs the necessity determination process which determines whether the highly attractive area | region in the driving | operation visual field image estimated by S102 is information required for driving action (S103). The necessity determination is performed based on the positional relationship between the highly attractive area on the driving field image identified from the saliency map of the driving field image and the attention area recognized by the attention area recognition unit 30. FIG. 6C is an example of the necessity determination result. Details of the necessity determination process will be described later.

  Then, in the determination step of S104, it is determined from the result of the necessity determination in S103 whether or not there is a region that is unnecessary for driving action and has high attractiveness on the driving support image. If there is a region that is unnecessary for driving behavior and has high attractiveness (S104: YES), the process proceeds to S105, and if it does not exist (S104: NO), this process ends.

  In S105, which proceeds when an affirmative determination is made in S104, the filtering unit 60 executes the information presentation filtering process in response to the result of the necessity determination process (S103), and then ends the present process. In this information presentation filtering process, a mask image for reducing the visibility of the driving view is displayed in a corresponding region of the mask display unit corresponding to the region to be subjected to the information presentation filtering specified from the driving view image. FIG. 6D is an example in which a mask image is displayed superimposed on a partial region of the driving field of view that is estimated to be unnecessary for driving behavior and highly attractive. In this example, mask images that reduce the light transmittance are displayed in the upper left and upper right portions of the driving field of view. This suppresses transmission of visual information that is unnecessary for driving behavior such as lighting of surrounding buildings, neon signs, and electric bulletin boards, and has high attractiveness. Details of the information presentation filtering process will be described later.

[Description of attractiveness estimation processing]
The procedure of the attractiveness estimation process executed in S102 of the main process will be described with reference to the flowchart of FIG.

  The attractiveness estimation unit 20 first processes the captured image of the driving field of view acquired by the image acquisition unit 10 to generate a resolution pyramid composed of a plurality of images having different resolutions (S201). Here, as shown in FIG. 8, an image group having a resolution reduced to 1/2, 1/4, 1/8... Is generated by repeating the reduction process on the original image. When such images are superimposed in the order of resolution, a hierarchical structure like a pyramid is formed, and such an image group is called a resolution pyramid.

  Next, the luminance information of each point of each image is extracted from the image of each layer of the resolution pyramid generated in S201 (S202). Then, a luminance feature map indicating the characteristic regarding the luminance of the image is generated from the luminance information of each point of each extracted image (S203). Here, using a plurality of pairs of images having different resolutions in the resolution pyramid, a difference in brightness at each point of the image is obtained, and a brightness feature map can be obtained based on the result.

  Next, color (red, green, blue, or saturation, hue) information of each point of each image is extracted from the image of each layer of the resolution pyramid generated in S201 (S204). Then, from the extracted color information of each point of each image, a color feature map indicating the characteristics relating to the color of the image is generated (S205). Here, using a plurality of pairs of images with different resolutions in the resolution pyramid, a color difference at each point of the image is obtained, and a color feature map can be obtained based on the result.

  Next, gradient information of each point of each image is extracted from the image of each layer of the resolution pyramid generated in S201 (S206). The image gradient indicates the degree of change of each pixel value. And the gradient feature map which shows the characteristic regarding the gradient of an image is produced | generated from the gradient information of each point of each extracted image (S207). Here, using a plurality of pairs of images having different resolutions in the resolution pyramid, a difference in gradient at each point of the image is obtained, and a gradient feature map can be obtained based on the result.

  Next, a temporal and spatial change (motion information) of each point of each image is extracted from the images of each layer in the plurality of time-series resolution pyramids generated from the plurality of time-series captured images in S201 (S208). Then, from the extracted motion information of each point of each image, a motion feature map indicating the feature relating to the motion of the image is generated (S209). Here, based on the time series of the resolution pyramid, the direction and velocity components are calculated at each point of each hierarchical image, and a motion feature map can be obtained based on the result.

  Next, a saliency map relating to the driving view image is created by superimposing and integrating the feature maps generated in steps S203, S205, S207, and S209 (S210). With this saliency map, a result of estimating the degree of ease of being attracted to the driving view image is obtained. Then, an image area corresponding to a location showing a value higher than a predetermined threshold in the saliency map is specified as a highly attractive area (S211), and this process ends.

[Description of necessity determination processing]
The procedure of the necessity determination process executed in S103 of the main process will be described with reference to the flowchart of FIG.

  In the necessity determination processing, first, attention region estimation processing is executed by the attention region recognition unit 30 (S301). The attention area estimation process will be described with reference to the flowchart of FIG.

  As shown in the flowchart of FIG. 10, the attention area recognition unit 30 first acquires vehicle information for estimating the traveling direction of the host vehicle, such as the current steering angle (S401). Next, the current traveling direction of the host vehicle is estimated based on the vehicle information acquired in S401 (S402). Then, in the driving view image acquired by the image acquisition unit 10, a region corresponding to the front in the traveling direction of the host vehicle is set as a caution region (S403).

  Next, road structures such as traffic signals, traffic signs, road markings, white lines, and shapes of traveling roads are detected from the driving view image by image recognition (S404). Then, the region of the driving view image corresponding to the road structure detected in S404 is set as the attention region (S405).

  Next, traffic participants such as vehicles, two-wheeled vehicles, and pedestrians are detected from image recognition in the driving view image and detection information by the radar sensors (S406). Then, the region of the driving view image corresponding to the traffic participant detected in S406 is set as a caution region (S407), and this process ends.

  Returning to the flowchart of FIG. In S302, the positional relationship between each highly attractive area estimated in the above-described attractiveness estimation process (see FIG. 7) and each attention area estimated in the attention area estimation process in S301 is compared, and each high attractiveness is compared. It is determined whether or not the sex region is included in the attention region. When the high attraction area is included in the attention area (S302: YES), the high attraction area is specified as information necessary for driving behavior (S303). On the other hand, when the highly attractive area is not included in the attention area (S302: NO), the highly attractive area in that portion is specified as information unnecessary for driving behavior (S304).

[Description of information presentation filtering processing]
The procedure of the information presentation filtering process executed in S105 of the main process will be described with reference to the flowchart of FIG.

  The filtering unit 60 first receives the processing result from the necessity determination unit 40 and converts the position of the highly attractive region determined as unnecessary to the position in the mask display unit (S501). Here, the range in the mask display unit is estimated so that the position of the highly attractive region specified on the driving field image is the same as the position of the driving field viewed from the driver.

  Next, exception processing for determining whether or not to apply an exception condition to the transmittance of the mask image to be displayed is executed (S502). This exception process will be described with reference to the flowchart of FIG.

  As shown in the flowchart of FIG. 12, it is determined whether or not a route guidance destination is set based on navigation information acquired from an in-vehicle navigation device (not shown) that provides a route guidance to the driver. Determination is made (S601). If the destination is set (S601: YES), the process proceeds to S602. If the destination is not set (S601: NO), the process proceeds to S603.

  In step S602, which proceeds when the destination is set, it is determined whether or not the host vehicle has arrived near the destination based on the navigation information. If it has arrived near the destination (S602: YES), the process proceeds to S607. If it has not arrived near the destination (S602: NO), the process proceeds to S603.

  In step S607, which is performed when the host vehicle has arrived near the destination, the setting for increasing the transmittance of the mask image displayed for information presentation filtering is enabled (≈1). In this way, when the destination is set in the in-vehicle navigation device and the own vehicle has arrived near the destination, the driver intentionally uses the surrounding scenery to search for the destination on an unfamiliar road. It is expected that the situation is paying attention to. In such a situation, it is possible to prevent the information that the driver wants to see from being suppressed by setting the transmittance of the mask image to a value that is substantially close to the total transmission.

  On the other hand, in S603, which proceeds when a negative determination is made in S601 or S602, since the driver started driving based on driving support information acquired from another driving support device (not shown) that manages the driving time. It is determined whether or not the elapsed time has exceeded N hours (N is a preset time). When the elapsed time from the start of operation is N hours or more (S603: YES), the process proceeds to S604, and when the elapsed time from the start of operation is less than N hours (S603: NO), the process proceeds to S605.

  In S604, which proceeds when the elapsed time from the start of driving is N hours or more, it is determined whether or not guidance for prompting a break has been implemented in the driving support device that manages the driving time. If guidance for prompting a break has been implemented (S604: YES), the process proceeds to S607. If guidance for prompting a break has not yet been performed (S604: NO), the process proceeds to S605.

  In step S <b> 607, which is performed when guidance for prompting breaks has been implemented, a setting for increasing the transmittance of the mask image displayed for information presentation filtering (≈1) is validated. In this way, when a long time has passed since the start of driving and guidance that encourages a break has been implemented, the driver intentionally pays attention to the surrounding scenery in order to find a facility for taking a break. The situation is expected to be. In such a situation, it is possible to prevent the information that the driver wants to see from being suppressed by setting the transmittance of the mask image to a value that is substantially close to the total transmission.

  On the other hand, in S605, which proceeds when a negative determination is made in S603 or S604, it is determined whether or not the remaining amount of fuel is very small based on the information acquired from the fuel gauge. If the remaining amount of fuel is very small (S605: YES), the process proceeds to S607. If the remaining amount of fuel is not small (S605: NO), the process proceeds to S606.

  In step S607, which proceeds when the remaining amount of fuel is very low, the setting for increasing the transmittance of the mask image displayed for information presentation filtering is made valid (≈1). Thus, when fuel is scarce, it is expected that the driver is deliberately paying attention to the surrounding scenery in order to find a fueling facility or a charging facility that can be refueled. In such a situation, it is possible to prevent the information that the driver wants to see from being suppressed by setting the transmittance of the mask image to a value that is substantially close to the total transmission.

  On the other hand, in S606 which proceeds when a negative determination is made in S605, it is determined whether or not the transmittance change switch 50 is operated, that is, whether the setting of “transmittance: high” is enabled. If the setting of “transmissivity: high” is enabled by the operation of the transmissivity change switch 50 (S605: YES), the process proceeds to S607 to increase the transmissivity of the mask image displayed for information presentation filtering. Enable the setting to (≈1). In this way, by operating the transmittance changing switch 50 by the user, the transmittance of the mask image can be intentionally set to a value almost close to the total transmittance.

  On the other hand, when the “transmission: high” setting is disabled in the transmittance change switch 50 (S605: NO), the process proceeds to S608, and the transmittance of the mask image displayed for information presentation filtering is applied. The setting to change automatically (no exceptions apply).

  Returning to the flowchart of FIG. In next step S503, it is determined whether or not the setting for adaptively changing the transmittance is enabled in the result of the exception processing in the previous step S502. When the setting for adaptively changing the transmittance is enabled (S503: YES), the process proceeds to S504, and when the setting for increasing the transmittance is not enabled (S503: NO), the process proceeds to S505. Proceed to processing.

  In S504, which proceeds when the setting for adaptively changing the transmittance is enabled, a transmittance setting process for determining the transmittance of the mask image for each mask target region (highly attractive region determined to be unnecessary). After that, the process proceeds to S506. The transmittance setting process will be described with reference to the flowchart of FIG.

As shown in the flowchart of FIG. 13, first, the luminance contrast C ₁ of the attention area on the driving view image is calculated (S601). Next, the luminance contrast C ₂ of the mask target region (highly attractive region determined to be unnecessary) on the driving view image is calculated (S602).

Next, the brightness contrast C ₂ ′ when it is assumed that the mask target region has been filtered by the mask image from the values of the brightness contrasts C ₁ and C ₂ calculated in S 601 and S 602 is expressed by the following formula (1). The transmittance that satisfies the condition is calculated (S603).
C ₂ ′ = αC ₁ (where α is a predetermined number satisfying 0 ≦ α <1) (1)
In this way, the brightness contrast of the mask target area when the mask image is displayed can be made lower than the brightness contrast of the attention area necessary for driving behavior, and the driver can pay attention to the attention area. Easy to point. In this way, the transmittance of the mask image can be determined based on the contrast of the luminance contrast between the mask target area and the attention area.

  In addition, the transmittance may be changed according to the current speed of the host vehicle. For example, when stopping or slowing down, even if attention is paid to areas unnecessary for driving behavior, safety is ensured, so the transmittance is set to 1 (no masking effect), while during normal driving As shown in the flowchart of FIG. 13, the transmittance is adaptively set based on the luminance contrast of the driving view image.

  Returning to the flowchart of FIG. On the other hand, in S505 that proceeds when a negative determination is made in S503, the transmittance of the mask image to be displayed is set to be higher (≈1), and then the process proceeds to S506.

  Next, in S506, according to the transmittance set in S504 or S505, a mask image is displayed at a corresponding position on the mask display surface, and the transmittance of the driving field of view visible to the driver is changed. Specifically, as already described with reference to FIG. 5, the mask image is displayed through procedures such as (a) extraction of a local region, (b) labeling, and (c) setting of a mask region and a transmittance distribution.

[Verification of effects by information presentation filtering]
The effect of information presentation filtering is verified from the measurement result of the driver's gaze distribution during driving. Here, when there is a highly attractive area that is not necessary for driving behavior, the gaze distribution spreads over a wide range of driving vision, but by applying information presentation filtering, the gaze is concentrated at the position where attention should be directed in driving behavior This is confirmed from the experimental results.

  First, a preliminary experiment using an eye mark recorder that measures the line of sight confirmed the position where the driver should pay attention during driving. FIG. 14A shows the analysis result of the driver's line-of-sight distribution when there is no region with high attractiveness in the driving field of view. As shown in this figure, the line-of-sight distribution when there is no highly attractive area in the driving field of view may be 22% near the vanishing point, 43% on the white line on the road surface, and about 35% in front of the host vehicle. confirmed.

In addition to the experimental results shown in FIG. 14 (a), research results such as the following references have been reported as the gaze distribution during driving. According to this, a gaze tends to gather on the center line on the right curve regardless of day and night (see FIG. 14B), and a gaze tends to gather on the shoulder line on the left curve (FIG. 14C). reference).
<References> Koji Ishiguro and Yuji Yamada, “Development of Control Specifications for the World's First Intelligent AFS”, Denso Technical Review, vol.9, no.2, pp.137-141,2004.
From the analysis results shown in FIGS. 14 (a) to 14 (c), it can be seen that when attention is concentrated on a region with high attractiveness and attention is concentrated, the line of sight tends to gather in the traveling direction of the host vehicle.

  Next, the results of a comparison experiment in which the time transition of the gaze position of the subject was measured in the same driving scene with and without information presentation filtering will be described with reference to the three experimental results shown in FIGS. To do.

  In this experiment, a moving image captured by an in-vehicle camera is used in a downtown area at night, and the subject's gaze position is measured with the same moving image captured by the camera (no filtering), and the other is neon. We compared the results of measuring the subject's gaze position with an image (with filtering) overlaid with a mask image that reduces transmittance in areas that are highly attractive and unnecessary for driving behavior, such as signs and electronic bulletin boards. The experiment was performed on three types of driving view images DATA # 1 to # 3.

  FIG. 15 is an explanatory diagram showing the experimental results of DATA # 1. Among these, (a) is an experimental result without filtering, (b) is an experimental result with filtering, respectively, a driving view image example, a time transition graph of a gaze position on the driving view image, a time transition, respectively. A table of graph statistical results (coordinate average value μ, standard deviation σ) is described (the same applies to FIGS. 16 and 17). In the time transition graph of the line-of-sight position, the horizontal axis indicates time, and the vertical axis indicates XY coordinate values (unit: pixel).

  Comparing the measurement results of FIGS. 15A and 15B, the average value μ of the line-of-sight position is near the front of the vehicle in the traveling direction, but the standard deviation σ indicating the variation is not filtered. Is about several times larger. From this result, in the case of no filtering, the subject's line-of-sight position varies widely as shown by the dashed-line ellipse added to the driving view image, and in the case of filtering, the subject's line-of-sight position is added to the driving view image It can be confirmed that the vehicle is concentrated in the forward direction of the vehicle, that is, at the position where attention should be paid in the driving action.

  FIG. 16 is an explanatory diagram showing the experimental results of DATA # 2. Among these, (a) is an experimental result without filtering, and (b) is an experimental result with filtering. Comparing these measurement results, the average value μ of the line-of-sight position is near the front in the vehicle traveling direction, but the standard deviation σ indicating the variation is about twice as large without filtering. . From this result, in the case of no filtering, the subject's line-of-sight position varies widely as shown by the dashed-line ellipse added to the driving view image, and in the case of filtering, the subject's line-of-sight position is added to the driving view image It can be confirmed that the vehicle is concentrated in the forward direction of the vehicle, that is, at the position where attention should be paid in the driving action.

  FIG. 17 is an explanatory diagram showing the experimental result of DATA # 3. Among these, (a) is an experimental result without filtering, and (b) is an experimental result with filtering. Comparing these measurement results, the average value μ of the line-of-sight position is near the front in the traveling direction of the vehicle, but the value of the standard deviation σ indicating the variation is more than doubled without filtering. . From this result, in the case of no filtering, the subject's line-of-sight position varies widely as shown by the dashed-line ellipse added to the driving view image, and in the case of filtering, the subject's line-of-sight position is added to the driving view image It can be confirmed that the vehicle is concentrated in the forward direction of the vehicle, that is, at the position where attention should be paid in the driving action.

  As described above, from the experimental results shown in FIGS. 15 to 17, by suppressing the transmission of information to the driver by performing information presentation filtering, it becomes difficult for the driver to be attracted to the highly attractive area unnecessary for the driving action, and the driving action It was confirmed that the line of sight concentrates at the position where attention should be directed.

[Correspondence with configuration described in claims]
The correspondence between the configuration of the vehicle information transmission control device of the embodiment and the configuration described in the claims is as follows. The attractiveness estimation unit 20 in the vehicle information transmission control device of the embodiment corresponds to attractiveness region specifying means in the claims. The attention area recognition unit 30 corresponds to a recognition unit. The necessity determination unit 40 corresponds to necessity determination means. The filtering unit 60 corresponds to filtering means.

[Reference example]
As shown in FIG. 18, a rearview mirror such as a rearview mirror or side mirror provided in or near the driving field of view is visually recognized by the information transmission filtering of the present invention unless the antiglare function is working. There is a possibility that the luminance contrast is relatively higher than that of the region where the property is lowered. Since humans tend to be attracted to areas with high luminance contrast, it is presumed that information transmission filtering will increase the frequency with which the driver views the rearview mirror.

  On the other hand, there is a possibility that a back monitor that displays an image of the rear of the vehicle on a display device such as a liquid crystal display will become widespread in the future as an alternative to the conventional rear-view mirror configured with a normal mirror. Therefore, the information transmission filtering function of the present invention may be mounted on the back monitor display. Information transmission filtering performed on the front window and information transmission filtering performed on the display of the back monitor may be used in combination.

  DESCRIPTION OF SYMBOLS 10 ... Image acquisition part, 20 ... Attractiveness estimation part, 30 ... Caution area | region recognition part, 40 ... Necessity determination part, 50 ... Transmittance change switch, 60 ... Filtering part.

Claims (10)

  From the image information obtained by imaging the driving field of view of the vehicle, the region of high visibility that is highly attracted to the driver and is estimated to be unnecessary for driving behavior is identified, and the corresponding region of the driving field of view corresponding to the highly attractive region Information transmission for vehicles characterized in that transmission of visual information to the driver is suppressed by applying a filter that reduces visibility from the driver to the visual information transmitted from the vehicle to the driver Control device. From the image information obtained by imaging the driving field of view of the vehicle, an attractive area specifying means for specifying a highly attractive area that is estimated to be highly attractive to the driver;
Necessity determining means for determining whether the visual information included in the highly attractive area specified by the attractive area specifying means is necessary for the driving action;
As a result of the determination by the necessity determination means, the driver for the visual information transmitted to the driver from the corresponding region of the driving field corresponding to the highly attractive region determined to be unnecessary for driving behavior A vehicle information transmission control device comprising: filtering means for applying a filter for reducing visibility from the vehicle. The vehicle information transmission control device according to claim 2,
The attraction area specifying means derives the saliency of the image based on at least one of the feature values of luminance, color, gradient, and motion in the image information obtained by imaging the driving field of view of the vehicle, and the high attraction based on the saliency. A vehicle information transmission control device characterized by specifying a sex region. In the vehicle information transmission control device according to claim 2 or 3,
The necessity determination means determines whether the visual information included in the high attraction area is in accordance with the positional relationship between the attention area to be noted by the driver in the image information of the driving field of view and the high attraction area. A vehicle information transmission control device characterized by determining whether or not it is necessary for action. The vehicle information transmission control device according to claim 4,
It further comprises a recognition means for recognizing a cautionary object including a traffic participant and a specific road structure existing in the driving field of the own vehicle,
The necessity determination means uses the area in the image information where the attention object recognized by the recognition means exists as the attention area, and does not require the highly attractive area in a range excluding the attention area for driving action. A vehicle information transmission control device characterized in that it is determined that there is a certain vehicle. The vehicle information transmission control device according to any one of claims 2 to 5,
In the display means having a display area provided to overlap the driving field of view visible to the driver, the filtering means provides a mask image for reducing the transmittance in a display area corresponding to the position of the visual information to be filtered. A vehicle information transmission control device characterized by displaying. The vehicle information transmission control device according to claim 6,
The filtering means displays the mask image with the transmittance continuously increased from the center of the display range where the mask image is to be displayed toward the boundary between the display range and the non-display range. An information transmission control device for a vehicle. In the vehicle information transmission control device according to claim 6 or 7,
The filtering means, according to the level of contrast on the image information of a predetermined area to be noted for driving behavior, and the level of contrast on the image information of the visual information to be filtered, A vehicular information transmission control device, wherein the mask image is set with a degree of transmittance of the mask image and the mask image is displayed with the set transmittance. The vehicle information transmission control device according to any one of claims 6 to 7,
The filtering means sets the level of the transmittance of the mask image in accordance with the level of the speed of the host vehicle, and displays the mask image with the set transmittance. Control device. From the image information obtained by imaging the driving field of view of the vehicle, an eye-catching area identification procedure for identifying a highly eye-catching area that is presumed to be highly attracting to the driver
A necessity determination procedure for determining whether the visual information included in the highly attractive region specified in the attractive region specifying procedure is necessary for the driving action;
As a result of the determination in the necessity determination procedure, the driver for the visual information transmitted to the driver from the corresponding region of the driving field of view corresponding to the highly attractive region determined to be unnecessary for driving behavior And a filtering procedure for applying a filter for reducing visibility from the information transmission control method.