KR101257438B1 - System for guiding a indoor waking-path of situation-based using a color-code and method thereof - Google Patents

Abstract

PURPOSE: A context awareness-based indoor walking guide system using a color code and a method thereof are provided to supply a proper walking guide service in any indoor situation using a color code detected up to 2.5m. CONSTITUTION: An input/output unit(100) receives a final destination input signal of a walker, and includes an input module(110) receiving a current situation image of the walker through a camera and an output module(120) outputting environment information. A context awareness unit(200) receives a current context image of the walker from the input module to extract features, and is aware of a current context through a similarity analysis of the current context image. A code processing unit(300) hierarchically detects a color code proper for the current context, and analyzes the color code in order to generate the environment information. [Reference numerals] (100) Input/output unit; (110) Input module; (120) Output module; (200) Context awareness unit; (210) DB module; (220) Image receiving module; (230) Context awareness module; (300) Code processing unit; (310) Code detecting module; (320) Code interpretation module;

Description

System for Guiding A Indoor Waking-Path Of Situation-Based Using A Color-code And Method Thereof}

The present invention relates to a situation-based indoor walking guidance system and method using a color code, and more specifically, to detect a color code corresponding to the situation hierarchically by recognizing a current situation of a pedestrian received through a camera. The present invention relates to a situation-based indoor walking guidance system and method using a color code that provides accurate walking guidance service to a destination by outputting environmental information most suitable for the current situation of a pedestrian.

In general, a blind person does not recognize an object and does not recognize his or her actual location, so it is difficult to secure a secure application where he / she wants, and is exposed to many dangers when going out. Many visually impaired people rely on braille to indicate the location of objects, but this also has a problem of being very limited in behavior.

In order to solve these problems, some visually impaired people rely on guide dogs for walking and safety. However, there is a problem in that safety walking is limited because the method of using guide dogs is not possible to actually perceive the position of pedestrians.

Conventionally, a method of receiving a walking guide using a GPS device is provided, but this is not only a problem that cannot be guided indoors. In addition, the GPS device uses a GPS satellite, and thus the error range of distance recognition is very large. There were a lot of problems because it could lead to the wrong location.

In order to solve this problem, Korean Laid-Open Patent Publication No. 10-2011-0127457 (name of the invention: a visually impaired pedestrian guide system using GPS and RFID) installs a corresponding RFID tag at a position requiring pedestrian guidance using RFID means. When the RFID reader carried by the visually impaired recognizes the RFID tag, the visually impaired pedestrian guidance system using RFID and GPS that induces the safe walking of the visually impaired by outputting a voice signal corresponding to the recognized tag information. Providing. However, this invention also has a problem as an indoor walking guidance system for the visually impaired because it did not solve the problem of RFID that can be used only in close proximity.

The present invention has been made in view of the above problems, by using a color code that can be detected at a maximum of 2.5m, indoor walking based on the situation using a color code that can be provided with the appropriate walking guidance service in any indoor situation To provide a guide system and method.

Then, by first recognizing the situation, by hierarchically detecting and interpreting the color code most suitable for the situation, the indoor pedestrian guidance system based on the situational awareness using the color code that can be provided with a smooth and accurate indoor pedestrian guidance service. And providing a method.

The present invention for achieving the technical problem, the input and output unit including an input module for receiving the final destination input signal of the pedestrian, the input module for receiving the current status image of the pedestrian through the camera and the output module for outputting environmental information; A situation recognition unit which receives a current situation image of a pedestrian from the input module, extracts a feature, and recognizes the current situation by analyzing a similarity with the stored basic situation image; And a code processor for hierarchically detecting a color code suitable for the current situation recognized by the situation recognition unit, and analyzing the detected color code to generate environment information. .

In addition, the input and output unit receiving the final destination of the pedestrian and the current status image of the pedestrian; Extracting the SURF feature from the current situation image of the pedestrian received by the situation recognition unit, calculating a similarity with the basic situation image, and recognizing the high similarity image as the current situation of the pedestrian; If the code processor recognizes the current pedestrian situation, hierarchically detecting a color code suitable for the recognized current situation; Interpreting, by the code processor, environment information stored in color codes detected hierarchically; And outputting, by the input / output unit, the analyzed environmental information to a pedestrian. .

According to the present invention as described above, by using a color code that can be detected at a maximum of 2.5m, it is possible to provide an appropriate walking guidance service in any indoor situation.

In addition, by first aware of the situation, by hierarchically detecting and interpreting the color code most suitable for the situation, it is possible to provide a smooth and accurate indoor walking guidance service.

1 is a block diagram of a context-based indoor walking guidance system using a color code according to an embodiment of the present invention.
2 is a diagram illustrating a feature of a situation using a feature extraction method of a SURF descriptor according to an embodiment of the present invention;
3 is a view showing the configuration of a color code according to an embodiment of the present invention, Figure 3a is a color bar code, Figure 3b is a pictogram, Figure 3c is a view showing a direction code.
4 is a flowchart illustrating a hierarchical color code detection method according to an embodiment of the present invention.
5 is a flowchart according to a cascade method according to an embodiment of the present invention.
Figure 6 is a flow diagram for a situation-based indoor walking guidance method using a color code according to an embodiment of the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

The present invention relates to a situation-based indoor walking guidance system and method using color codes, which will be described below with reference to FIGS. 1 to 6.

1 is a block diagram of a context-based indoor walking guide system using a color code according to an embodiment of the present invention, and the system is an input / output unit 100, a situation recognition unit 200, and a code processing unit 300. It includes.

The input / output unit 100 includes an input module 110 and an output module 120. The input module 110 receives an input signal of a final destination of a pedestrian and receives a current situation image of the pedestrian through a camera. It performs the function of receiving all information and signals coming into the system. The situation recognition unit 200 to be described later includes a camera for collecting an image of the current situation and the input means for receiving information on the final destination of the pedestrian so as to recognize the current situation of the pedestrian. Here, the input means may include a text input means such as a keyboard and a keypad, and a voice recognition means for recognizing a person's voice and identifying a destination. It also includes visual input means for identifying the destination by selecting the destination using a local map or the like.

The output module 120 is a part for outputting environmental information to pedestrians, as well as visual output using a display device, as well as acoustic output using a speaker device. Here, by using the TTS interface that converts the text into voice and outputs it, the recognized result such as the position or direction of a given destination is output to the pedestrian.

At this time, the environmental information includes location information and direction information on the current user's situation.

In particular, when the input module 110 and the output module 120 communicate with each other, and the final destination information of the input module 110 and the environmental information of the output module is inconsistent, the situation recognition unit 200 Send the current image of the pedestrians to the system to repeat the system.

The situation recognition unit 200 receives the current situation image of the pedestrian from the input module, extracts the feature, and recognizes the current situation by analyzing the similarity with the stored basic situation image, the DB module 210 and the image receiving module. 220 and context awareness module 230.

Recognition of what is happening is based on comparison with basic sample data, and it is impossible to define and recognize all the various situations in the world. Therefore, the system assumes a total of four cases of door situation, corridor situation, hall situation, and junction situation in order to organize and recognize the situation more simply. define.

The DB module 210 is an offline step of the system, and indexes and stores the basic situation image using the SURF feature extraction method for the four situations defined above. Although there may be many situations, each situation has characteristics that only the situation has, so it can be extracted and classified into the four most similar situations. For example, the situation where feature A is extracted can be indexed, which can be defined as situation A. The indexed information is called a basic image and is stored in the DB module 210.

In more detail, a SURF descriptor is extracted from basic situation data collected from context images, and a vocabulary tree is generated by applying Hierarchical K-means clustering to the extracted SURF feature. In the context-aware module 230 to be described later, the basic data is indexed in the form of a frequency vector * weight vector using the generated Vocabulary tree to be compared with a current situation image to a basic situation image. Save it.

2 is a diagram illustrating a feature of a situation using a feature extraction method of a SURF descriptor according to an embodiment of the present invention. When the feature extraction method of the SURF descriptor is used for basic data as shown in (a) and (b) of FIG. 2, feature points are extracted as shown in (c), (d), and (e), and each frequency vector is obtained. ) * When the weight vector is calculated, it is indexed into a door situation, a corridor situation, a hall situation, and a junction situation according to the characteristics of each situation and stored as a basic situation image. .

The above-described SURF descriptor and K-means clustering are all known in the art and are well known to those skilled in the art, and thus detailed descriptions thereof will be omitted for clarity.

Meanwhile, the form of the frequency vector * weight vector is based on [Equation 1].

Where t is the basic situation image, m is the frequency vector representing the number of times the SURF features extracted from the image appear in the nodes of the vocabulary tree, and w is the noise node. Weight vector used to reduce reflectance.

Meanwhile, in this system, the number distribution of leaf nodes is used, and the number distribution of inner nodes is not used.

The image receiving module 220 receives the current status image received from the input module. Since the situation recognition unit 200 takes a relatively long time to load, instead of receiving an image directly from the camera, the situation recognition unit 200 periodically receives the image input from the input module 110 through the image receiving module 220.

The context awareness module 230 extracts the SURF feature from the current situation image received from the image receiving module 220, calculates the similarity with the basic situation image of the DB module, and displays a high similarity image of the pedestrian. Recognize.

More specifically, the situation recognition module 230 extracts the SURF feature from the current situation image received from the image receiving module 220, and uses the frequency vector (Vocabulary tree) generated by the DB module 210. Frequency vector) * Calculate the weight vector.

Here, if the current situation image is q, the frequency vector * weight vector of q is expressed by Equation 2 below.

Where q is the current state image, n is the frequency vector representing the number of times the SURF features extracted from the image appear in the nodes of the Vocabulary tree in the form of a vector, and w is the noise node. Weight vector used to reduce reflectance.

Meanwhile, in this system, the number distribution of leaf nodes is used, and the number distribution of inner nodes is not used.

When the values of t and q are calculated through Equation 1 and Equation 2, the vector value of the basic situation image t and the current situation image q is compared to obtain the image having the highest similarity s. Recognize.

That is, if the analysis result of the current situation image determines that the value of the vector is the highest similarity with the door situation vector of the basic situation image, the current situation is recognized as the door situation, and similarly, the current situation As a result of analyzing the image, if the value of the vector is determined as the highest similarity with the corridor situation vector of the basic situation image, the current situation is recognized as a corridor situation.

The similarity s is based on [Equation 3], and the smaller the value of s, the higher the similarity is determined.

The code processor 300 hierarchically detects a color code suitable for the current situation recognized by the situation recognition unit 200, analyzes the detected color code, and generates environmental information, and generates a code detection module 310 and a code. Analysis module 320 is included.

The code detection module 310 hierarchically detects a suitable color code according to the current situation recognized by the situation recognition unit 200.

The code detected by the code detection module 310 is a color code, and the color code is composed of a location code indicating position information and a direction code indicating direction information. At this time, the location code is composed of a color bar code and a pictogram.

Figure 3 is a view showing the configuration of the color code according to an embodiment of the present invention, it can be seen clearly about the color code.

The color bar code (FIG. 3A) displays location information using color and order, and the pictogram (FIG. 3B) selects ten pictograms with the highest frequency of use.

The direction code (FIG. 3C) is indicated by the combination of the location code and the arrow, and the color bar code, pictogram and direction code can be classified by their size. Exactly the size is expressed as a value that is equal to the product of width and height, as shown in Table 1 below.

The method of detecting code hierarchically by the code detection module 310 is composed of four steps.

If the image appears too bright due to a sudden change of lighting in the indoor environment, or the whole image appears dark due to shadows (narrow image contrast), preprocessing to reduce the detection error due to environmental changes by spreading the image contrast Through (S410).

After the preprocessing of step S410, in order to detect a color code, a binarization step of dividing each pixel of the image into 'green' or 'non-green' pixels is performed (S420).

The system uses color-coded quiet zones in common, so the image is divided into 'green' or 'non-green' pixels. How to classify is shown in [Equation 4].

However, the above-described formula is not limited to the design of only one embodiment of the case of using the green quiet zone as in the present invention.

After step S420, Row-by-Row labeling is performed on color code candidate areas detected as being green areas in the image, and Area, Circularity, Width-height ratio of each area is removed to remove noise areas included in the image. The noise regions are removed by comparing with the predefined threshold value (S430).

The hierarchical detection means that the most suitable color code is detected first according to the current situation. The hierarchical detection enables the system to be more efficiently performed. 4 is a flowchart illustrating a hierarchical color code detection method according to an embodiment of the present invention, and will be described based on this.

In fact, location information is more important than direction information in a door situation. In this case, it is more efficient to detect a color bar code or a pictogram first rather than direction code first, and in other situations, direction information is more important. Therefore, it is more efficient to detect the direction code first, then the pictogram and then the code hierarchically in the order of detecting the color bar code.

The criteria for detecting a color code hierarchically in a specific situation, that is, whether to detect a direction code first or a pictogram first is based on [Table 1]. That is, the color code most suitable for the situation is detected by determining which color code is detected first using the width ㅧ height (ratio).

After the step S430, in order to prevent the color code from being detected separately, a process of connecting adjacently detected areas is performed to undergo a post-processing step to be detected as one color code (S440). Through the step S440 it is possible to detect the color code more clearly.

The code interpretation module 320 interprets the corresponding information stored for the code detected by the code detection module 310. For quick calculation, each recognition module commonly performs the extraction of the middle line of the color code. The method of analysis differs depending on the color code type, and the location code color bar code and pictogram can obtain the current location information through analysis, and the direction code can obtain the current location information through analysis. . The location information and the direction information thus obtained are called environmental information.

The color bar code is interpreted as follows. The color bar code consists of eight color patches, which are performed by extracting the middle horizontal lines for quick calculation. First, a Gaussian filter and a Prewitt edge detector are applied to determine the boundary between color patches. The color patch portion except the detected boundary portion is mapped to one color using the Cascade method, and is selected as the representative color as the color having the largest occupancy among the colors of the pixels constituting the color patch. Decode the color barcode by applying the decimal system to the numeric value corresponding to the color of each patch. Color bar code can be used to interpret numerical information of the location.

Although the Cascade method is described in FIG. 5, the description thereof is a general method, which is obvious to those skilled in the art, and a description thereof will be omitted.

To interpret the pictogram, the horizontal and vertical center lines of the pictogram are extracted, normalized to the template size, and color histograms for the red, green, blue, and white channels are calculated for the extracted center lines. The calculated histograms are recognized as the most similar pictograms using template pictograms and Hamming distances. It performs a function of providing in detail the location information that can not be provided by the color bar code through a pictogram. Therefore, the pedestrian can obtain location information on the current situation through the color bar code and the pictogram.

On the other hand, the interpretation method of the direction code is as follows. The direction code consists of a location code and an arrow pictogram, which are first divided into a location code and an arrow pictogram, the horizontal center line is extracted, and then divided in half to calculate the occupancy rate where white pixels appear in each part. Recognizing in the direction of the pictogram, the location code portion is recognized by using the above color bar code decoder, pictogram recognizer to combine the two results, to obtain the direction information.

Finally, the code analysis module 320 aggregates the environmental information obtained from each color code and transmits it to the input / output unit 100.

6 is a flowchart illustrating a situation-based indoor walking guidance method using color codes according to an embodiment of the present invention.

The input module 110 of the input / output unit 100 receives the final destination image of the pedestrian and the current situation image of the pedestrian (S610).

The situation recognition unit 200 extracts the SURF feature from the current situation image of the pedestrian received in step S610, calculates the similarity with the basic situation image of the DB module, and recognizes the high similarity image as the current situation of the pedestrian. (S620).

At this time, in order to generate the basic situation image, the SURF descriptor is extracted from the basic situation data collected from the situation images before step S620, and the vocal tree is generated by applying Hierarchical K-means clustering to the extracted SURF feature. By using the generated Vocabulary tree, indexing is performed in the form of a frequency vector * weight vector and stored as a basic situation image (S620 ').

If the current situation of the pedestrian is recognized from the step S620, the code detection module 310 of the code processing unit 300 hierarchically detects a suitable color code according to the current situation recognized in the step S620 (S630). Thereafter, the code analysis module 320 interprets the corresponding information stored for the color codes detected hierarchically in step S630 (S640).

The information interpreted in step S640 is called environmental information, and the environmental information is transmitted to the output module 120 of the input / output unit 100 and output to the pedestrian (S650).

After the step S650, while the input module 110 and the output module 120 communicate with each other, it determines whether the final destination information received in the step S610 and the environmental information transmitted in the step S650 (S660).

If the final destination information input in step S610 and the environmental information transmitted in step S650 is inconsistent, the input module 110 transmits the current situation image of the pedestrian to the situation recognition unit 200 again, and performs the procedure in step S620. Return to (S670). In this case, it is not necessary to go through step S620 'again.

As a result of the determination in step S660, when the final destination information input in step S610 and the environment information transmitted in step S650 match, the output module 120 outputs a destination arrival signal and the system is terminated (S680). That is, the system is repeatedly performed until the current environment information of the pedestrian matches the destination information.

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications may be made without departing from the invention. And all such modifications and changes as fall within the scope of the present invention are therefore to be regarded as being within the scope of the present invention.

100; Input and output unit 110; Input module
120; Output module 200; Situation awareness
210; DB module 220; Image receiving module
230; Context aware module 300; Code processing unit
310; Code detection module 320; Code Analysis Module

Claims (10)

An input / output unit 100 including an input module 110 for receiving an input signal of the pedestrian's final destination and receiving an image of the current situation of the pedestrian through a camera and an output module 120 for outputting environmental information;
A situation recognition unit 200 which receives a current situation image of a pedestrian from the input module 110, extracts a feature, and recognizes the current situation by analyzing a similarity with the stored basic situation image; And
A code processor 300 for hierarchically detecting a color code suitable for the current situation recognized by the situation recognition unit 200 and analyzing the detected color code to generate environment information; Situational awareness based indoor walking guidance system using a color code comprising a.
The method of claim 1,
The input and output unit 100,
When the input module 110 and the output module 120 are in communication with each other, the final destination information of the input module 110 and the environmental information of the output module 120 is inconsistent, the situation recognition unit 200 Context-aware indoor walking guidance system using color codes to transmit the current situation image of pedestrians.
The method of claim 1,
The situation recognition unit 200
DB module 210 for storing the basic situation image using the SURF feature extraction method;
An image receiving module 220 for receiving a current status image received from the input module 110; And
A situation recognition module for extracting SURF features from the current situation image received from the image receiving module 220, calculating similarity with the basic situation image of the DB module, and recognizing an image having high similarity as the current situation of a pedestrian ( 230); Situational awareness based indoor walking guidance system using a color code comprising a.
The method of claim 3, wherein
The situation awareness module 230
The similarity between the current situation image and the basic situation image is calculated using the following equation, but the image having a small result of s is regarded as a high similarity image, and the high similarity image is recognized as the current situation of a pedestrian. Context-aware indoor walking guidance system using color code.

Where q is the current situation image vector

, t is the basic situation image vector

Means.)
The method of claim 1,
The code processing unit 300,
A code detection module 310 for hierarchically detecting a suitable color code according to the current situation recognized by the situation recognition unit 200; And
A code analysis module 320 for interpreting the corresponding information stored for the code detected by the code detection module 310; Situational awareness based indoor walking guidance system using a color code comprising a.
In the indoor walking guidance method using the indoor walking guidance system based on the situation using a color code including the input / output unit 100, the situation recognition unit 200 and the code processing unit 300,
(a) receiving, by the input / output unit 100, an image of a pedestrian's final destination and a pedestrian's current situation;
(b) The situation recognition unit 200 extracts the SURF feature from the current situation image of the pedestrian received in step (a), calculates the similarity with the basic situation image, and displays the high similarity image of the current situation of the pedestrian. Recognizing;
(c) the code processor 300 hierarchically detecting a color code suitable for the current situation recognized in the step (b) when the current pedestrian situation is recognized from the step (b);
(d) interpreting, by the code processor 300, environment information stored in color codes hierarchically detected in step (c); And
(e) the input / output unit 100 outputting the environmental information analyzed in the step (d) to a pedestrian; Situational awareness based indoor walking guidance method using a color code comprising a.
The method according to claim 6,
before step (b)
(b ') The situation recognition unit 220 extracts the SURF feature from the basic situation data from which the situation images are collected, generates a Vocabulary tree by applying Hierarchical K-means clustering to the extracted SURF feature, and then generates the Vocabulary. storing a basic contextual image by indexing using a tree; Situational awareness based indoor walking guidance method using a color code further comprising.
The method according to claim 6,
(f) After the step (e), the input / output unit 100 communicates between the input module 110 and the output module 120, and the final destination information received in the step (a) and the step (e) Determining whether or not the output environment information is identical; And
(g) if the final destination information input in step (a) and the environmental information transmitted in step (e) do not match,
The input / output unit 100 transmits the current situation image of the pedestrian to the situation recognition unit 200 again, and performs the procedure in step (b) based on situation awareness based indoor walking using color codes. Way.
The method of claim 8,
(g ') when the final destination information input in the step (a) and the environmental information transmitted in the step (e) are identical,
The input / output unit (100) is a situation-based indoor walking guidance method using a color code, characterized in that ending while outputting the destination arrival signal.
The method according to claim 6,
The step (b)
The similarity between the current situation image and the basic situation image is calculated using the following equation, but the image having a small result of s is regarded as a high similarity image, and the high similarity image is recognized as the current situation of a pedestrian. Context-aware indoor walking guidance system using color code.

Where q is the current situation image vector

, t is the basic situation image vector

Means.)

Images