JP6582921B2 - Robot monitor system - Google Patents

Description

  The present invention relates to a system for monitoring a robot body by an image.

  Conventionally, in a facility where robots and people coexist, when the robot controller detects that a person has approached using, for example, a light curtain or a laser sensor, the robot arm can be stopped or operated at a safe low speed. Control to ensure the safety of people.

JP 2010-231713 A

However, when the worker is working with his back to the robot arm, the worker cannot visually observe the robot arm.
The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a robot monitor system capable of monitoring a robot body even when an operator is turning his back to the robot body.

According to the robot monitor system of the first aspect, the position information acquisition unit acquires the position information of the worker located in the work area of the robot body, and the direction information output unit performs the work located in the work area. The direction information for detecting the head direction, which is the direction in which the person's head front is facing, is output.
Display control unit, the located worker in the work area, and the head direction of the operator detected from the direction information, the robot body on the rear side of the operator is in the direction in which the position, The image data of the robot main body is processed so as to be an image of the state of the robot main body that can be seen when the robot main body is put into view from the position of the operator at that time, and is projected onto the display unit of the head-mounted display.

According to this structure, even in a state that can not be visually robot directly by the operator is turned against the robot body, images data of the robot body, the work skill that position the robot body It is projected on the head mounted display in the state of the robot body that can be seen when looking back. Therefore, the operator can confirm the state of the robot body located on the back side of the user by the image displayed on the head-mounted display, and can improve safety during work.

  According to the robot monitor system of the second aspect, the display control unit uses the 3D model image data obtained by modeling the robot body in three dimensions as the processing target. If comprised in this way, an operator can visually recognize the state of the robot main body located in the back side of himself more realistically by displaying a robot main body by a 3D model image.

  According to the robot monitor system of claim 3, when the display control unit acquires the posture information of the robot body, the image data projected on the display unit of the head-mounted display is displayed in a posture corresponding to the posture information at that time. To image data. If comprised in this way, the operator can also recognize the attitude | position of the robot main body at the time from the image displayed on the display part. Therefore, safety can be further improved.

  According to the robot monitor system of claim 4, when the display control unit obtains the distance between the robot body and the worker from the position information, the image projected on the display unit of the head-mounted display according to the distance. Change the data size. With this configuration, the operator can recognize the sense of distance from the robot body at that time from the image displayed on the head-mounted display, and can further improve safety.

  According to the robot monitor system of the fifth aspect, the direction information output unit includes the image pickup device arranged on the head-mounted display, and the image picked up by the image pickup device is processed to determine the head direction of the operator. To detect. That is, if an image with the head mounted on the head-mounted display is used to capture an image with the operator's head facing the front, it is determined whether or not the robot body is in the operator's field of view based on the image. I understand. Therefore, if the image picked up by the image pickup device is processed, the head direction of the operator can be easily detected.

1 is a functional block diagram schematically showing a configuration of a robot monitor system according to an embodiment Side view showing the imaging area of the camera Plan view showing imaging area by camera Flowchart mainly showing processing contents on the eyeglass-type monitor device side The figure which shows an example of the visual field which an operator sees via the display part of a spectacles type monitor

  Hereinafter, an embodiment will be described. FIG. 1 is a functional block diagram schematically showing the configuration of the robot monitor system of the present embodiment. The system 1 includes, for example, a robot arm for assembly 2, a robot controller 3 for controlling the robot arm 2, a camera 4, a spectacle-type monitor 5, an attached camera 6, and a spectacle-type monitor device 7.

  The robot arm 2 which is a robot body is configured as, for example, a 6-axis vertical articulated robot. Although a detailed description of a general configuration is omitted, the robot arm 2 has 6-axis arms each driven by a servo motor, and is housed in, for example, a pallet at the tip of the sixth-axis arm. A hand for gripping a workpiece is provided. The robot arm 2 is connected to the robot controller 3 via a connection cable (not shown), and the servo motors for the respective axes are controlled by the robot controller 3.

  The robot controller 3 corresponding to the display control unit and the robot-side control device is configured by incorporating a control circuit, a servo control unit, a power supply device, and the like (not shown) in a rectangular box-shaped frame. The control circuit is composed mainly of a microcomputer, and controls each axis servo motor of the robot arm 2 via a servo control unit according to a pre-stored operation program, teaching data set by a teaching pendant (not shown), various parameters, etc. To control the robot arm 2 to automatically execute the work assembly work.

  Further, in order to perform the above-described control, the robot controller 3 receives encoder values from encoders (not shown) arranged on the respective axes. Furthermore, the robot controller 3 stores and holds 3D model image data, which is data obtained by three-dimensionally modeling the robot arm 2, in an internal memory.

  For example, as shown in FIG. 2, the camera 4 is installed at a position overlooking the work area from the information of the robot arm 2 so that the imaging area includes at least the work area of the robot arm 2. Image data captured by the camera 4 is input to the robot controller 3. The camera 4 corresponds to a position information acquisition unit. As shown in a plan view in FIG. 3, when the operator 8 is located in the imaging region of the camera 4, image data input by the robot controller 3 is input. By performing the processing, the two-dimensional coordinate value (x, y) that is information indicating the position of the worker 8 is acquired with the position of the robot arm 2 as the origin.

  As shown in FIG. 1, an eyeglass-type monitor 5 which is a head-mounted display is worn by an operator 8 as eyeglasses on a head, and is not shown on a transparent display portion 5D corresponding to a lens portion of the eyeglasses. This is a so-called transmissive display capable of projecting an image via a projection unit. A direction information output unit and an attached camera 6 as an image pickup device are arranged on one side of the frame of the glasses-type monitor 5, and the operator 7 wears the glasses-type monitor 5 on the head. The image of the direction which the head of 8 heads is facing is imaged. The cameras 4 and 6 are constituted by, for example, a CCD (Charge Coupled Device), a CMOS image sensor, or the like.

  The eyeglass-type monitor 5 is connected to the eyeglass-type monitor device 7 that is a display control unit and a display-side control device wirelessly or by wire. The eyeglass-type monitor device 7 includes a microcomputer and the like, and transmits image data to be projected on the display unit 5D of the eyeglass-type monitor 5 and receives image data captured by the attached camera 6. Further, the eyeglass-type monitor device 7 can be wired or wirelessly communicated with the robot controller 3, and the robot controller 2 receives work area monitoring information including position information of the worker 8 from the robot controller 3 and posture information. Each axis encoder value and the 3D model image data described above are acquired.

  Next, the effect | action of this embodiment is demonstrated with reference to FIG.4 and FIG.5. FIG. 4 is a flowchart mainly showing processing contents on the eyeglass-type monitor device 7 side. First, when the eyeglass-type monitor device 7 starts communication with the robot controller 3 and connects (S1), it acquires 3D model image data of the robot arm 2 (S2). The subsequent steps S3 to S14 are an infinite loop.

  When the area monitoring information is acquired in step S4, it is determined whether or not the worker 8 exists in the work area of the robot arm 2 (S5). If it is determined that the worker 8 does not exist and is “none”, the process returns to step S3. If it is determined that the worker 8 is present and “present”, image recognition is performed to process the image data input from the attached camera 6 (S6). Here, when the image data includes an image capturing the robot arm 2 and it is determined “Yes” in the subsequent step S7, the front of the head of the worker 8 is generally directed toward the robot arm 2. Since it is estimated that the vehicle is present, “front determination” is set (S8). In this case, the process proceeds to step S14 without performing special processing.

  On the other hand, if it is determined in step S7 that the image data does not include an image of the robot arm 2 and “no” is present, the front of the head of the operator 8 is generally directed in the opposite direction to the robot arm 2. Since it is estimated that there is a back surface, it is determined as “back surface determination” (S9). Then, the 3D model image data of the robot arm 2 is processed as described below, and image data to be projected on the display unit 5D of the glasses-type monitor 5 is generated.

  First, when the worker 8 tries to see the robot arm 2 from the current position, the direction of the robot arm 2 that is visible to the worker 8 is calculated (S10). Next, when the encoder value of each axis is acquired from the robot controller 3 (S11), the encoder value is reflected in the 3D model image data to reproduce the posture that the robot arm 2 currently takes. At this time, since the distance from the position of the operator 8 to the origin position of the robot arm 2 is known, the size of the image data to be displayed is changed according to the distance (S12). Needless to say, adjustment is made so that the image data is displayed larger as the distance becomes shorter. The step of changing the size of the image data is not necessarily proportional to the resolution of the distance. For example, the length of the distance may be divided into a plurality of steps, and may be changed according to the step.

  When the image data is generated as described above, the data is transmitted to the glasses-type monitor 5 and displayed on the display unit 5D (S13). In this case, the field of view that the operator 8 sees through the display unit 5D is displayed in one area portion of the display unit 5D with a background that can be seen through the display unit 5D as shown in FIG. 5, for example. The 3D image data of the robot arm 2 is superposed. At this time, in order to make the 3D image data easy to see, the image data is generated so that a certain area around the robot arm 2 is filled with transparency.

  As described above, according to the present embodiment, the eyeglass-type monitor device 7 transmits the video signal and the image data to be projected on the display unit 5D of the eyeglass-type monitor 5 worn by the operator 8 on the head. The camera 4 acquires position information of the worker 8 located in the work area of the robot arm 2, and the attached camera 6 is a direction in which the front of the head of the worker 8 located in the work area is facing. Image data is output as direction information for detecting the head direction. The robot controller 3 controls the operation of the robot arm 2 and holds 3D model image data obtained by modeling the robot arm 2 in three dimensions.

  The eyeglass-type monitor device 7 acquires 3D model image data from the robot controller 3, the worker 8 is located in the work area, and the head direction of the worker 8 is in the field of view of the worker 8. If the robot arm 2 is not in the direction, the 3D model image data is processed so that the robot arm 2 can be seen when the robot arm 2 is viewed from the position of the operator 8 at that time. It is displayed on the display unit 5D of the glasses-type monitor 5.

  With this configuration, even if the operator 8 is turned to the robot arm 2 and cannot see the robot arm 2 directly, the 3D model image data of the robot arm 2 is obtained from the position of the operator 8. The image is projected onto the glasses-type monitor 5 in the state of the robot arm 2 that can be seen when looking back to the robot arm 2 side. Therefore, the worker 8 can check the state of the robot arm 2 positioned on the back side of the worker 8 in a realistic state by the 3D image displayed on the glasses-type monitor 5, thereby improving safety during work. be able to.

  The eyeglass-type monitor device 7 acquires the encoder value of the robot arm 2 from the robot controller 3, and processes the 3D model image data projected on the eyeglass-type monitor 5 into image data with a posture corresponding to the encoder value at that time. To do. If comprised in this way, the operator 8 can also recognize the attitude | position of the robot arm 2 at the time from the image displayed on the spectacles type monitor 5. FIG. Therefore, safety can be further improved.

  Further, when the distance between the robot arm 2 and the operator 8 is obtained from the position information, the glasses-type monitor device 7 changes the size of the 3D model image data projected on the glasses-type monitor 5 according to the distance. If comprised in this way, the operator 8 can also recognize the distance feeling with the robot arm 2 at the time by the image displayed on the spectacles type monitor 5, and can improve safety | security further.

  In addition, the eyeglass-type monitor device 7 detects the head direction of the operator 8 by processing an image captured by the attached camera 6 arranged on the eyeglass-type monitor 5. That is, if an image with the head of the worker 8 facing the front is taken by the attached camera 6, it can be seen from the image whether the robot arm 2 is in the field of view of the worker 8. And if the spectacles type monitor device 7 processes the imaged image, the head direction of the worker 8 can be easily detected.

The present invention is not limited to the embodiments described above or shown in the drawings, and the following modifications or expansions are possible.
The image data of the robot body is not limited to the one that generates a 3D model image, but may be one that generates a two-dimensional image.
The robot body is not limited to the robot arm 2, but may be a robot arm having a horizontal 4-axis configuration, a self-propelled robot, or a humanoid robot.
A laser sensor or an infrared sensor may be used for the position information acquisition unit.

If the worker's image captured by the camera 4 is processed, the direction in which the worker's head is facing can be detected. Therefore, the camera 4 may have a function as a direction information output unit, or a camera separate from the camera 4 may be used as the direction information output unit. Further, these image data may be input directly to the eyeglass-type monitor device 7.
It is not always necessary to reflect the posture of the robot arm 2 in the 3D model image data displayed on the display unit 5D. Further, it is not necessary to change the size of the image according to the distance, and the image may always be displayed at a constant size.

The display-side control device and the robot-side control device may be configured as an integrated control device. The “communication” in this case may be performed inside the control device via a bus, for example.
It is not always necessary to distribute the functions as the display control unit between the display-side control device and the robot-side control device, and all of the functions may be provided in one of the control devices. Alternatively, all functions may be provided in an independent third control device.
The head-mounted display is not necessarily the eyeglass-type monitor 5, and may be any display that can project an image on a display unit worn by the operator on the head.

  In the drawings, 1 is a robot monitor system, 2 is a robot arm, 3 is a robot controller, 4 is a camera, 5 is an eyeglass monitor, 6 is an attached camera, 7 is an eyeglass monitor device, and 8 is an operator.

Claims (5)

A head-mounted display configured to be able to project an image on a display unit worn by the operator on the head; and
A position information acquisition unit for acquiring position information of a worker located in the work area of the robot body;
A direction information output unit that outputs direction information for detecting a head direction that is a direction in which the front of the head of the worker located in the work area is facing;
Located worker in the work area, and the head direction of the operator detected from the direction information, when the robot body on the rear side of the operator is in a direction to position the robot A display control unit that processes the image data of the main body so as to be an image that can be seen when the robot main body is put into view from the position of the operator and projects the image data on the display unit of the head-mounted display; Robot monitor system equipped.   The robot monitor system according to claim 1, wherein the display control unit uses 3D model image data obtained by modeling the robot body in three dimensions as the processing target. The display control unit acquires posture information of the robot body,
The robot monitor system according to claim 1 or 2, wherein the image data projected on the display unit of the head-mounted display is processed into image data having a posture corresponding to posture information at that time.   4. The display control unit according to claim 1, wherein the display control unit obtains a distance between the robot body and the worker from the position information, and changes a size of image data projected on the display unit according to the distance. The robot monitor system according to one item. The direction information output unit has an imager disposed on the head-mounted display,
The robot monitor system according to claim 1, wherein the head direction of the worker is detected by processing an image picked up by the image pickup device.

Images