JP6440303B2 - Object recognition device, object recognition method, and program - Google Patents

Description

  The present invention relates to an object recognition device, an object recognition method, and a program.

  Conventionally, there is a technique for recognizing a target included in an image by performing image recognition on a captured image. In this object recognition technique, in recent years, an object is recognized with high accuracy by using a deep learning technique in which a neural network is multilayered. This deep learning technique generally recognizes an object with high accuracy by layering a plurality of layers in an intermediate layer between an input layer and an output layer of a neural network. In this deep learning technology, in particular, a convolutional neural network (CNN) has attracted attention because it has higher performance than a conventional method for recognizing an object based on image feature amounts.

  According to the convolutional neural network, a recognition target image to which a label is attached is learned, and a main target included in the recognition target image is recognized. For this reason, when there are multiple main objects in the learning image, the convolutional neural network manually designates the area rectangle that contains the main object and assigns a label to the image corresponding to the designated area. And learning.

In this learning process, it is necessary to add a label to the designated area. In the subsequent learning process, an image corresponding to the designated region is input to the convolutional neural network, and feature vectors output from the convolutional neural network are learned.
In addition, the conventional object recognition process extracts a candidate area based on local features by performing edge extraction processing or the like on the recognition target image, and inputs the candidate area into a convolutional neural network. Classification was performed after extracting the vectors, and the classified candidate areas with the highest certainty were obtained as recognition results. The certainty factor is an amount indicating how much the similarity degree of the subject of the image learned together with a certain image region and label is relatively higher than the similarity degree of the other class.

“Rich feature hierarchies for accurate object detection and semantic segmentation”, Tech report, Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, UC Berkeley (2014)

  However, the conventional object recognition technique using the convolutional neural network cannot be automatically narrowed down to one desired object when there are a plurality of objects in the learning image, and thus is included in the image as described above. It was necessary to manually specify multiple areas that contain the target one by one.

  That is, the learning process in the conventional object recognition technology using the convolutional neural network needs to create learning data by assigning one label to one recognition target image, and therefore, there is a burden for area designation. large. Further, it is necessary to create and store an image (hereinafter referred to as a teacher image) that is a processing target of a learning process for improving the recognition accuracy of the object, dedicated to the processing of the convolutional neural network.

  In addition, the recognition process in the object recognition technology using the conventional convolutional neural network often has a clear outline of the object to be recognized, and the teacher image often has a clear outline of the object. For this reason, good recognition accuracy could not be exhibited for images in which it is difficult to determine a specific object. For example, images containing many objects such as Akihabara and Asakusa landscapes, images that make it difficult to identify a specific object, and objects that are short-range objects and long-distance objects are mixed. Images with a large size or objects with greatly different sizes cannot be processed with high recognition accuracy.

  Furthermore, there are similar problems not only in the conventional object recognition technology using the convolutional neural network but also in other existing object recognition technologies.

  Therefore, the present invention has been made in view of the above-described problems, and an object recognition device, an object recognition method, and a program that can determine a main object in an image and the position of the object with higher recognition accuracy. The purpose is to provide.

(1) According to one aspect of the present invention, an image conversion unit that sets a plurality of candidate areas of an arbitrary area in a recognition target image, and a recognition process for image information included in each candidate area set by the image conversion unit A recognition calculation unit that calculates a certainty factor for each candidate candidate in each candidate region, and a target candidate included in the recognition target image for a plurality of candidate regions that overlap at each position of the recognition target image A map generation unit that calculates an evaluation value obtained by summing the certainty factors for each class ID, and generates a distribution of the evaluation values at each position of the recognition target image as a map for each class ID ; A position calculation unit that specifies a position of a target based on a map created by the map generation unit.

(2) According to one aspect of the present invention, an image conversion unit that sets a plurality of candidate areas having an arbitrary area in a recognition target image, and a recognition process for image information included in each candidate area set by the image conversion unit A recognition calculation unit that calculates a certainty factor for each target candidate in each candidate area, and each class that represents a target candidate included in the recognition target image at each position of the recognition target image. For each ID, the confidence factor and a coefficient corresponding to the area of the candidate region are used as parameters, and an evaluation value that increases as the area increases is calculated. For each class ID, the recognition target image It has a map generating unit for generating a distribution of the evaluation value at each position as a map, and a position calculating unit for specifying a position of the object on the basis of the map created by the map generation unit, pairs It is a recognition device.

(3) According to one aspect of the present invention, an image conversion unit that sets a plurality of candidate areas of an arbitrary area in a recognition target image, and a recognition process for image information included in each candidate area set by the image conversion unit A recognition calculation unit that calculates a certainty factor for each candidate candidate in each candidate region, and a target candidate included in the recognition target image for a plurality of candidate regions that overlap at each position of the recognition target image For each class ID that represents, using as a parameter the total value of the certainty factor and a coefficient according to the area of the plurality of overlapping candidate regions, calculating an evaluation value that increases as the area increases, for each class ID, the map generation unit for generating a distribution of the evaluation value in each position of the recognition target image as a map, the position of the object based on the maps created by the map generation unit Having a position calculating unit for specifying a target recognition apparatus.

  (4) One aspect of the present invention is the above-described target recognition device, wherein the image conversion unit extracts a feature amount of the recognition target image, and based on the extracted feature amount, A candidate area corresponding to the size of the feature included in the image is set, and a candidate area having a smaller area than a predetermined area is deleted from the plurality of candidate areas set in the recognition target image.

  (5) One aspect of the present invention is the above-described target recognition device, including a learning processing unit that learns a target based on a learning image, wherein the learning processing unit is an entire learning image supplied from the outside. The recognition calculation unit calculates the certainty factor of the target candidate using the image of the candidate area set by the image conversion unit and the learning result.

(6) According to one aspect of the present invention, a step of setting a plurality of candidate areas having an arbitrary area in a recognition target image, a recognition process is performed on the image information included in each candidate area, and the candidate candidates Calculating the certainty factor for each, and at each position of the recognition target image, the certainty factor for each class ID representing a target candidate included in the recognition target image for a plurality of candidate regions overlapping at the position For each class ID, generating a distribution of the evaluation values at each position of the recognition target image as a map , and identifying a target position based on the map ; The object recognition method which has these.

(7) According to one aspect of the present invention, a computer is configured to set a plurality of candidate areas having an arbitrary area in a recognition target image, perform recognition processing on the image information included in each candidate area, and target each candidate area. The certainty factor for each candidate is calculated, and the certainty factor is calculated for each class ID representing a candidate candidate included in the recognition target image for a plurality of candidate regions overlapping at the position at each position of the recognition target image. A program for calculating a total evaluation value, generating a distribution of the evaluation value at each position of the recognition target image as a map for each class ID, and specifying a target position based on the map is there.

  According to one embodiment of the present invention, it is possible to determine the main object and the position of the object in the image with higher recognition accuracy.

It is a block diagram which shows the functional structure of the object recognition apparatus 1 of embodiment to which this invention is applied. It is a block diagram showing an example of 1 composition of object recognition device 1 of an embodiment to which the present invention is applied. It is a figure which shows the learning image P10 which imaged the tower-shaped building from short distance and the downward direction. It is a figure which shows the learning image P11 which imaged the building from a long distance and upper direction. It is a figure which shows the learning image P12 which imaged the building at night. It is a flowchart which shows the procedure of the recognition process in the object recognition apparatus 1 shown as embodiment of this invention. It is a figure which shows an example of the candidate area | region set to the recognition target image input into the target recognition apparatus 1 to which this invention is applied, and the certainty factor for every candidate area | region. It is a figure which shows the reliability and the area, and the relationship between a candidate area | region in the target recognition apparatus 1 to which this invention is applied. In the object recognition device 1 to which the present invention is applied, it is a diagram illustrating a map created by a map generation unit 16. It is a figure which shows an example of the candidate area | region set to the recognition target image P1 as a comparative example. It is a figure which shows an example of the other candidate area | region set to the recognition target image P1 as a comparative example.

Hereinafter, an object recognition apparatus, an object recognition method, and a program to which the present invention is applied will be described with reference to the drawings.
First, a functional description of the object recognition device 1 according to an embodiment to which the present invention is applied will be given.
FIG. 1 is a block diagram showing a functional configuration of an object recognition device 1 according to an embodiment to which the present invention is applied. The target recognition device 1 recognizes a main target among the targets included in the recognition target image, and outputs an attribute (name, type) and position of the recognized target.
The object recognition device 1 includes an image conversion unit 12, a recognition calculation unit 14, a map generation unit 16, and a position calculation unit 18.

  The image conversion unit 12 is supplied with a recognition target image from the outside. The image conversion unit 12 may set a plurality of candidate regions in the recognition target image by an arbitrary method, and the plurality of candidate regions may overlap each other. The image conversion unit 12 will be described assuming that a candidate area is set based on the characteristics of the recognition target image. However, the candidate region setting method is not limited to this, and a method of shifting frames of several sizes may be employed.

The recognition calculation unit 14 performs a recognition process on the image information included in each candidate area set by the image conversion unit 12. The recognition calculation unit 14 refers to the learning result stored in the learning result database 140 and increases the certainty as the image information included in each candidate region is similar to the target candidate. Thereby, the recognition calculation unit 14 calculates the certainty factor for each target candidate in each candidate region. The target candidate is a target that can be recognized by a person who sees an image such as an object or a landscape included in the recognition target image, and is a candidate that may be included in the recognition target image. The certainty is the probability of the target candidate, and how much the subject's similarity in the image learned with an arbitrary image and label is relatively higher than the similarity of other classes (target candidates) It is the quantity which shows.
The recognition calculation unit 14 obtains a certainty factor for each target candidate when a plurality of target candidates are calculated for the image included in the candidate region. Thereby, the recognition calculation unit 14 obtains one or a plurality of target candidates and the certainty factor of each target candidate for each candidate region.

The map generation unit 16 calculates an evaluation value for each position of the recognition target image based on the class ID and the certainty factor. For example, the map generation unit 16 calculates an evaluation value for each pixel of the recognition target image or for each representative position where several pixel sets are regarded as one position. The map generation unit 16 performs predetermined statistical processing on the certainty factor and calculates an evaluation value. The evaluation value becomes higher as the evaluation that the main target included in the recognition target image is a target corresponding to the class ID is higher.
In the statistical processing, for example, for each position of the recognition target image, the reliability is calculated for each class ID to calculate an evaluation value. According to this statistical processing, a higher evaluation value is calculated for a position where candidate areas for which the same class ID is calculated overlap.
Further, the statistical processing may use a coefficient according to the area of the candidate region and the certainty factor, and increase the evaluation value as the area of the candidate region increases.
Further, in the statistical processing, the class ID and the certainty factor may be totaled and the evaluation value may be calculated using a coefficient corresponding to the area of the candidate region. According to this conversion process, a candidate area where the same class ID is calculated overlaps, and a higher evaluation value is calculated as the area of the candidate area is larger.
Note that the statistical conversion process only needs to be able to calculate an evaluation value that increases the evaluation that the main target included in the recognition target image is a target corresponding to the class ID, and other processes may be performed.
Thereby, the map production | generation part 16 can produce the map containing the evaluation value for every class ID for every position of the recognition object image. That is, a map is generated in which higher evaluation values are distributed as the possibility that an object corresponding to the class ID exists is higher.

The position calculation unit 18 refers to the map created by the map generation unit 16 and calculates the position of the target (class ID) mainly included in the recognition target image. The position calculation unit 18 determines that a target corresponding to the class ID exists in an area where the evaluation value of the class ID is high in the recognition target image.
For example, the position calculation unit 18 may extract a region of the recognition target image whose evaluation value exceeds a predetermined threshold, and determine that a target corresponding to the class ID exists in the extracted region. In addition, the position calculation unit 18 performs edge extraction processing on the map, extracts a region surrounded by the edge line whose evaluation value is increased stepwise, and an object corresponding to the class ID is included in the extracted region. You may determine that it exists.
Thereby, the position calculation unit 18 can obtain the class ID and target position information corresponding to the class ID. This position information may be the contour position of the target or the center position of the target.

Hereinafter, a specific configuration and operation of the object recognition apparatus 1 shown in FIG. 1 will be described.
FIG. 2 is a block diagram showing a configuration example of the object recognition apparatus 1 according to the embodiment to which the present invention is applied. The object recognition device 1 is mounted on, for example, a portable communication terminal. The object recognition device 1 is connected to the image processing device 200 and the image display unit 300.

The image processing apparatus 200 includes an image generation unit 202 and an AR (Augmented Reality) processing unit 204.
The image generation unit 202 generates a recognition target image supplied to the target recognition device 1. The image generation unit 202 includes, for example, a camera device, a communication unit that receives image data from the outside, or a storage unit that stores image data. The image processing apparatus 200 supplies the recognition target image generated by the image generation unit 202 to the target recognition apparatus 1 when recognizing a target included in the recognition target image.

The AR processing unit 204 performs AR processing by performing image processing on the recognition target image generated by the image generation unit 202. This AR processing includes processing for adding, deleting, enhancing, or attenuating information in the environment represented by the recognition target image. The AR process includes, for example, a process of adding a description image of a target included in the recognition target image to the recognition target image captured by the camera device. The AR processing unit 204 causes the target recognition device 1 to output a recognition target image when the target recognition device 1 identifies a target included in the recognition target image.
In the present embodiment, the target recognition apparatus 1 is described as performing an AR process of recognizing a landmark image included in the recognition target image and adding a description of the landmark image to the recognition target image.

In addition, in this embodiment, it demonstrates using the result (landmark) by which the object recognition apparatus 1 recognized the object for AR processing, However, The result by which the object recognition apparatus 1 recognized the object is other uses. Of course, it may be used.
For example, the image processing apparatus 200 may cause the target recognition apparatus 1 to identify a target included in an image displayed by the SNS application in response to a request output from an SNS (social networking service) application. The SNS application deletes or attenuates a target from the recognition target image when a target set in advance is included in the recognition target image based on the result of recognition of the target by the target recognition device 1 (class ID and position). Image processing such as adding other information is performed.

The object recognition device 1 includes an image conversion unit 12, a recognition calculation unit 14, a map generation unit 16, a position calculation unit 18, an interface unit 20, a learning image storage unit 22, and a learning processing unit 24.
The object recognition device 1 is a computer having a control device that executes software and a storage unit. The object recognition device 1 is a software function unit that functions when a processor such as a CPU (Central Processing Unit) executes a program stored in a storage unit (not shown). The object recognition device 1 may be a hardware function unit such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).
The storage unit in the object recognition device 1 is realized by, for example, an HDD (Hard Disc Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). Is done. The storage unit stores various programs such as firmware and application programs, processing results by various functional units, information acquired from the outside by the interface unit 20, and the like.

The interface unit 20 exchanges information with the image processing apparatus 200 mounted on the portable communication terminal, and also exchanges information with each unit in the object recognition apparatus 1.
In response to the supply of the recognition target image from the image processing apparatus 200, the interface unit 20 outputs the recognition target image to the image conversion unit 12 as a processing target. The interface unit 20 is supplied with the class ID and the position from the position calculation unit 18 as the processing result of the object for the recognition target image, and outputs the class ID and the position to the image processing apparatus 200. The class ID and position are information indicating the classification of the landmark image included in the recognition target image. The class ID is information for specifying a main target included in the recognition target image, and examples thereof include a building name, a city name, and a landscape name.

The learning image storage unit 22 has a function of storing a learning image to which a class ID is added. The learning image storage unit 22 includes an image conversion unit 220 and an image data database 222.
The learning image storage unit 22 downloads a learning image from a server device (not shown) that stores an image as a learning material. For example, the learning image storage unit 22 downloads a learning image in accordance with an AR application command executed in the object recognition device 1.
A label is added to the learning image. The attribute information of the learning image is written on this label. The attribute information includes classification of landmarks included in the learning image. The attribute information is added as a class ID.
In the present embodiment, the learning image storage unit 22 downloads an image material including a landmark as a learning image. The landmark image included in the learning image is preferably an image that can be recognized as a landmark by human eyes. Further, the landmark image in the learning image may have an arbitrary area with respect to the image area, and the position in the image may also be an arbitrary position.

The image conversion unit 220 is supplied with a learning image used for performing learning processing from the outside. The image conversion unit 220 preprocesses the learning image in accordance with the processing method of the subsequent learning process. The image conversion unit 220 converts, for example, a learning image area change process and an aspect ratio adjustment into a format in which the learning process can be performed. The image conversion unit 220 is preferably learned by the learning processing unit 24 using the entire learning image, but the learning processing unit 24 may perform learning by dividing or segmenting the learning image.
The image data database 222 stores the learning image before conversion supplied from the outside and the learning image converted by the image conversion unit 220. The learning data converted by the image conversion unit 220 is stored in the image data database 222 as a batch file that can be learned by the learning processing unit 24. The image data database 222 stores a learning image before conversion and a learning image after conversion for each landmark classified by the label.

The learning processing unit 24 performs learning based on an image with a label (class ID) stored in the image data database 222. The learning process executed by the learning processing unit 24 is performed by converting image data into numerical data such as vectors. The learning result obtained by this learning process is stored in the learning result database 140 in the recognition calculation unit 14.
The learning result obtained by the learning processing unit 24 is referred to by the recognition calculating unit 14.

  The image conversion unit 12, the recognition calculation unit 14, the map generation unit 16, and the position calculation unit 18 operate as described with reference to FIG. As a result, the target recognition apparatus 1 outputs the class ID and the position to the interface unit 20 in response to the recognition target image being supplied.

  Note that the learning method by the learning processing unit 24 may create a convolutional neural network or other machine learning method (SVM or the like), and the learning result database 140 stores learning results corresponding to the learning method. The The recognition calculation unit 14 obtains the class ID and the certainty factor corresponding to the main object included in the recognition target image based on the learning result stored in the learning result database 140.

When the convolutional neural network is used for calculating the class ID and the certainty factor, the convolutional neural network is learned to classify the recognition target images into an arbitrary number of classes. The learning processing unit 24 learns the filter coefficient and the bias value in the convolution layer so that the learning result of the class ID: 1 is obtained in response to the learning image of the class ID: 1 being supplied. Specifically, the convolutional neural networks learned by classifying the classes into “Coast”, “Akihabara”, “Shibuya”, “Tokyo Tower”, and “Tokyo Metropolitan Government” are “Coast”, “Akihabara”, “Shibuya”, “ According to the supply of recognition target images that include “Tokyo Tower” and “Tokyo Metropolitan Government” as main targets, classify as “Coast”, “Akihabara”, “Shibuya”, “Tokyo Tower”, and “Tokyo Metropolitan Government”. Can do.
In this learning process, the convolutional neural network includes, for example, those shown in FIGS. 3 to 5 as learning images. A learning image P10 shown in FIG. 3 is an image of a tower-like building (landmark) taken from a short distance and from below, and a learning image P11 shown in FIG. 4 is an image of a tower-like building from a long distance and from above. The learning image P12 shown in FIG. 5 is an image obtained by capturing a tower-like building at night.
In this learning process, the target recognition apparatus 1 performs the learning process on the entire learning image without extracting a specific object and cutting out candidate areas.

  Next, object recognition processing in the object recognition device 1 configured as described above will be described. FIG. 6 is a flowchart showing the procedure of recognition processing in the object recognition apparatus 1 shown as the embodiment of the present invention.

  First, the image conversion unit 12 stands by until a recognition target image is input (step S200). In response to the input of the recognition target image (step S200: YES), a region having an arbitrary area is added to the recognition target image. A plurality are set (step S202). At this time, the image conversion unit 12 extracts a feature amount of the recognition target image, and based on the extracted feature amount, a region (candidate region) having an arbitrary area according to the size of the feature included in the recognition target image. Set more than one. The image conversion unit 12 supplies an image for each candidate area to the recognition calculation unit 14.

  At this time, the image conversion unit 12 may delete a candidate area having a smaller area than a predetermined area from a plurality of candidate areas set in the recognition target image. The predetermined area area is desirably set to a ratio in which the main target is included in the recognition target image. For example, the size of an object to be noticed by a person viewing the image among the recognition target images may be sufficient. The predetermined area may be set in advance, or may be set based on the areas of a plurality of candidate areas set for the recognition target image.

FIG. 7 is a diagram illustrating an example of candidate regions set in a recognition target image input to the target recognition device 1 to which the present invention is applied and a certainty factor for each candidate region.
As shown in FIG. 7, when the tower-like building is supplied as an image of a candidate area included in a substantially central position with a large image area, the image conversion unit 12 includes a plurality of candidate areas including the tower-like building A and A plurality of candidate areas set for the building B other than the tower-shaped building and a plurality of candidate areas set for the building C are set.
As shown in FIG. 7, the image conversion unit 12 can use any of the existing methods for setting candidate regions as long as it can set a plurality of candidate regions for an object included in the recognition target image with little leakage. Any technique can be used.

The recognition calculation unit 14 receives a region candidate from the image conversion unit 12, and uses the learning result data to calculate a class ID and a certainty factor for the region image passed from the image conversion unit 12 (step S204). The recognition calculation unit 14 selects the next candidate region (step S208) and calculates the class ID and the certainty factor (step S204) until the calculation of the certainty factor is completed for all candidate regions set by the image conversion unit 12. ) Is repeated.
The recognition calculation unit 14 calculates an evaluation value for each position of the recognition target image by the map generation unit 16 in response to the completion of the calculation of the certainty factor for all candidate regions (step S206: YES). Is generated (step S210). Then, based on the map generated by the map generation unit 16, the position calculation unit 18 outputs class IDs and positions representing main objects included in the recognition target image to the interface unit 20 (step S212).

In the example shown in FIG. 7, a state in which the certainty factor for each class ID is calculated for each of the plurality of candidate regions is shown. FIG. 8 is a diagram illustrating the relationship between the certainty factor and the area for each candidate region used when the map generation unit in the object recognition device 1 to which the present invention is applied calculates the evaluation value. In FIG. 8, the area counts candidate areas exceeding one million pixels. As for the tower-shaped building A, as shown in FIG. 8A, there are many candidate regions with high certainty factor and large area. As for the building B, as shown in FIG. 8B, the certainty factor is high, but there are many candidate regions having a small area. As for the building C, as shown in FIG. 8C, there are many candidate regions having a small certainty factor and a small area.
Using the characteristics as shown in FIG. 8, the object recognition apparatus 1 statistically processes the certainty of the position of the recognition target image for each class ID, and the main object is a region having a high evaluation value obtained as a result of the statistical processing. It may be an existing area.

FIG. 9 is a diagram for explaining the relationship between the position represented by the map created by the map generation unit 16 and the evaluation value in the target recognition device 1 to which the present invention is applied, by the recognition calculation unit 14 recognizing the recognition target image P. This map showing an example of visualization is created for Ta (tower type building), Tb (mountain), Tc, Te, and Tf for each class ID. In the example shown in FIG. 9, the map for each class ID is created so that the higher the brightness of the position of the evaluation value at which the statistical processing result is higher. Although the maps differ as shown in (a) to (d) due to the difference in statistical processing for calculating the evaluation value, any of the maps (a) to (d) shown in FIG. The position may be calculated.
The position calculation unit 18 performs edge extraction processing on each class ID map, extracts a region surrounded by edge lines whose evaluation values are increased stepwise, and corresponds to the class ID in the extracted region. It may be determined that there is a target. According to FIG. 9, Ta (tower type building) existing at a substantially central position of the recognition target image is extracted by performing edge extraction processing on a map whose class ID is Ta (tower type building). be able to.
Thereby, the object recognition apparatus 1 can output the processing result including the class ID (Ta) representing the tower type building and the extracted position of the tower type building from the interface unit 20.

In the description shown in FIG. 8, the map of the evaluation value is created for each class ID by the map generation unit 16, but the evaluation value map is not created for the class ID that is unlikely to be included in the recognition target image. May be. When the evaluation value calculated by statistically processing the certainty factor is low, there is a high possibility that a main object is not extracted from the evaluation value map, and the map generation unit 16 calculates the class ID from which such an evaluation value is calculated. Is determined not to create a map.
The map generation unit 16 calculates an evaluation index that can be calculated with a smaller amount of calculation than the map generation process before the map is generated. For example, the map generation unit 16 obtains the total value of the evaluation function using the area of the candidate region and the certainty as parameters for each class ID, compares the total value between the class IDs, and creates a map of the evaluation value The ID and the class ID that does not create the evaluation value map are determined. Thereby, the object recognition apparatus 1 can suppress the processing amount about creation of a map, suppressing the fall of the recognition accuracy of the main object.

  As described above, according to the target recognition device 1 to which the present invention is applied, a plurality of regions having an arbitrary area are set in the recognition target image, the certainty factor of the target candidate included in each region is calculated, Since a map for each target candidate is generated based on the certainty factor and the position of the target candidate, and the position of the target is specified from the map, a target included in an area with a high certainty factor is simply included in the recognition target image. It is possible to suppress misrecognition of being a target, and to determine a main target in a recognition target image with higher recognition accuracy.

  For example, even if a landscape of an urban area that includes a large number of buildings or a landscape that includes a large number of buildings but includes a single landmark is input as a recognition target image, it is clearly displayed in the recognition target image. It can suppress outputting the classification result that the single target included is the main target in the recognition target image. More specifically, according to the object recognition device 1 to which the present invention is applied, it is possible to recognize a symbolic townscape name as a main object, and further to recognize a landmark that occupies a large area as a main object. .

FIG. 10 shows an example of candidate areas set in the recognition target image P1 as a comparative example. FIG. 11 shows an example of another candidate region set in the recognition target image P1 as a comparative example.
As a comparative example, when the landmarks output from the convolutional neural network are obtained as classification results based only on the certainty obtained by inputting the images of the candidate areas A1, B1, and C1 shown in FIG. 10 into the convolutional neural network. Of the candidate areas A1, B1, and C1 set for the recognition target image P1, the certainty factor of the candidate area B1 is the highest. However, the main target included in the recognition target image P1 is the building T1, and is included in the image of the candidate area A1.

  Similarly, as a comparative example, the landmarks output from the convolutional neural network are classified based on only the certainty factor obtained by inputting the images of the candidate areas A2, B2, and C2 shown in FIG. 11 to the convolutional neural network. As a result, among the candidate areas A2, B2, and C2 set for the recognition target image P2, the certainty factor of the candidate area B2 is the highest. However, the main target included in the recognition target image P2 is the building T2, and is included in the image of the candidate area A2.

As described above, when a plurality of targets are included in the recognition target image as in the comparative example, erroneous recognition may occur when the main target included in the recognition target image is recognized based only on the certainty factor. .
On the other hand, according to the object recognition device 1 to which the present invention is applied, the class ID and the certainty factor are calculated for each candidate area, and the certainty factor is statistically processed for each class ID. As described above, it is possible to increase the probability that an object included in a candidate area having a large area among candidate areas can be recognized as a main object.

  Furthermore, according to the object recognition device 1 to which the present invention is applied, learning is performed using the entire learning image supplied from the outside by the learning processing unit 24, so that the object included in the entire recognition target image is recognized. The main object in the recognition target image can be determined with higher recognition accuracy.

  That is, according to the object recognition device 1 to which the present invention is applied, there is no need to cut out a correct object in the learning process, so that no matter what position of the learning image the correct object is included in the learning image Learning processing can be performed on the entire image, and it is possible to determine the main target in the recognition target image with high recognition accuracy, regardless of the form of the main target included in the entire image in the recognition processing. it can.

  Specifically, when teacher data is created so that a specific object can be recognized as in the existing technology and features are matched, landscapes (for example, urban areas, main streets, etc.) are difficult to distinguish object regions. Although recognition performance cannot be exhibited for images and large buildings that differ greatly in appearance from near view to distant view, according to the object recognition device 1 to which the present invention is applied, learning processing is performed on the entire image such as landscape. Thus, even an image such as a landscape can recognize a main object with high recognition accuracy.

  Furthermore, according to the object recognition apparatus 1 to which the present invention is applied, since it is not necessary to cut out an image including an object that is a correct answer for the learning process, the processing amount of the pre-processing of the learning process can be suppressed. A general image used can be used as a learning image, and the efficiency of the learning process can be increased. Further, according to the object recognition device 1 to which the present invention is applied, it is not necessary to edit the learning image so that one learning image becomes one label, and therefore a dedicated teacher image is used for preprocessing of the learning processing. There is no need to prepare, and the processing amount can be further suppressed.

  Furthermore, according to the object recognition device 1 to which the present invention is applied, since a plurality of certainty levels for a plurality of overlapping areas among the plurality of areas set by the image conversion unit 12 are statistically processed, a high certainty level is obtained. As the number of overlapping regions increases, it is possible to suppress misrecognition of being a main target included in the recognition target image, and to determine the main target in the recognition target image with higher recognition accuracy.

  Furthermore, according to the object recognition device 1 to which the present invention is applied, based on the area of the candidate region in addition to the plurality of certainty factors regarding the plurality of overlapping regions among the plurality of regions set by the image conversion unit 12. Since statistical processing is performed, a candidate area with a high certainty factor and a large area can suppress misrecognition that it is a main target included in the recognition target image, and can detect a main target in the recognition target image with higher recognition accuracy. Can be determined.

  Furthermore, according to the object recognition device 1 to which the present invention is applied, the area having a smaller area than the predetermined area is deleted from the plurality of areas set in the recognition target image. By identifying the target included in the recognition target image, the main target in the recognition target image can be determined with high recognition accuracy even if the processing load is reduced.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

Further, a computer program for realizing the function of the object recognition device 1 described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Good. Here, the “computer system” may include an OS and hardware such as peripheral devices.
“Computer-readable recording medium” refers to a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a DVD (Digital Versatile Disk), and a built-in computer system. A storage device such as a hard disk.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Object recognition apparatus, 12 ... Image conversion part, 14 ... Recognition calculation part, 16 ... Map generation part, 18 ... Position calculation part, 20 ... Interface part, 22 ... Learning image storage part, 200 ... Image processing apparatus, 202 ... Image generation unit, 204... AR processing unit

Claims (7)

An image conversion unit that sets a plurality of candidate areas of arbitrary area in the recognition target image;
A recognition calculation unit that performs recognition processing on image information included in each candidate region set by the image conversion unit, and calculates a certainty factor for each target candidate in each candidate region;
At each position of the recognition target image, for each of the plurality of candidate regions overlapping at the position, for each class ID representing the target candidate included in the recognition target image, calculate an evaluation value that sums the certainty factor, For each class ID, a map generation unit that generates a distribution of the evaluation values at each position of the recognition target image as a map;
A position calculation unit that identifies a target position based on the map created by the map generation unit;
An object recognition device. An image conversion unit that sets a plurality of candidate areas of arbitrary area in the recognition target image;
A recognition calculation unit that performs recognition processing on image information included in each candidate region set by the image conversion unit, and calculates a certainty factor for each target candidate in each candidate region;
At each position of the recognition target image, for each class ID representing the target candidate included in the recognition target image, the certainty factor and a coefficient corresponding to the area of the candidate region are used as parameters for the candidate region at the position. A map generation unit that calculates an evaluation value that increases as the area increases, and generates a distribution of the evaluation value at each position of the recognition target image as a map for each class ID ;
A position calculation unit that identifies a target position based on the map created by the map generation unit;
An object recognition device. An image conversion unit that sets a plurality of candidate areas of arbitrary area in the recognition target image;
A recognition calculation unit that performs recognition processing on image information included in each candidate region set by the image conversion unit, and calculates a certainty factor for each target candidate in each candidate region;
At each position of the recognition target image, for each of the plurality of candidate areas overlapping at the position, for each class ID representing the target candidate included in the recognition target image, the total value of the certainty factor and the plurality of overlapping multiple A coefficient corresponding to the area of the candidate region is used as a parameter, an evaluation value that increases as the area increases is calculated, and the distribution of the evaluation value at each position of the recognition target image is mapped for each class ID. A map generator to generate as
A position calculation unit that identifies a target position based on the map created by the map generation unit;
An object recognition device. The image conversion unit extracts a feature amount of the recognition target image, sets a candidate region according to the size of the feature included in the recognition target image based on the extracted feature amount, and the recognition target Deleting a candidate area having a smaller area than a predetermined area from a plurality of candidate areas set in the image;
The object recognition device according to any one of claims 1 to 3. A learning processing unit for learning a target based on a learning image;
The learning processing unit generates a learning result using the entire learning image supplied from the outside,
The recognition calculation unit calculates the certainty factor of the target candidate using the image of the candidate area set by the image conversion unit and the learning result.
The object recognition device according to any one of claims 1 to 4. Setting a plurality of candidate areas of arbitrary area in the recognition target image;
Recognizing the image information included in each candidate area, calculating a certainty factor for each target candidate in each candidate area;
At each position of the recognition target image, for each of the plurality of candidate regions overlapping at the position, for each class ID representing the target candidate included in the recognition target image, calculate an evaluation value that sums the certainty factor, Generating a distribution of the evaluation values at each position of the recognition target image as a map for each class ID ;
Identifying a target location based on the map ;
An object recognition method comprising: On the computer,
Set multiple candidate areas of arbitrary area in the recognition target image,
Recognizing the image information included in each candidate area, causing each candidate area to calculate a certainty factor for each target candidate,
At each position of the recognition target image, for each of the plurality of candidate regions overlapping at the position, for each class ID representing the target candidate included in the recognition target image, calculate an evaluation value that sums the certainty factor, For each class ID, the distribution of the evaluation values at each position of the recognition target image is generated as a map ,
A program for specifying a position of an object based on the map .