JP5453352B2 - Video display device, video display method and program - Google Patents

Description

  Embodiments described herein relate generally to a video display device, a video display method, and a program.

  2. Description of the Related Art In recent years, a type of digital television apparatus is known in which a camera is incorporated, a viewer is photographed by the built-in camera, and the photographed video is displayed on the display of the apparatus. In such a television apparatus, since the camera is fixed, the position and optical axis of the built-in camera cannot be finely adjusted unlike an external camera.

  Conventionally, when displaying a camera image captured by a camera on a display, a technique for correcting the display position and rotation angle of the camera image, other image distortion, etc. by image processing and displaying the corrected image on the display It has been known.

JP 2009-42162 A

  However, in the prior art, a calibration method for correcting a camera image captured by a camera fixed to a television device is under development, and a calibration method that can be easily performed is desired. It was.

  The problem to be solved by the present invention is to provide a video display device, a video display method, and a program that facilitate calibration of a camera fixed to a television device.

Video display apparatus embodiment comprises a camera, and de Isupurei, a first display means, a detection means, a calculation unit, a position detection means, and correcting means. The first display means displays a predetermined graphic on the display with a predetermined display coordinate as a reference. The detection means obtains detection coordinates at which the mirror image is detected on a camera image obtained by photographing a mirror image of the figure projected on a mirror with the camera. Said calculation means, based on the relative positional relationship between the detected coordinates of the mirror image and the display coordinates of the predetermined graphic, the order of the coordinates in the captured camera image by the camera in correspondence with the display coordinates of the display A correction coefficient is calculated. The position detecting means detects a position of a human face included in the camera image. Wherein the correction means corrects the position of the face with respect to the display coordinates of the display by the correction factor.

FIG. 1 is an external perspective view showing an example of the television apparatus according to the embodiment. FIG. 2 is a block diagram showing a signal processing system of the television apparatus. FIG. 3 is a diagram for explaining an example of a positional shift between a camera image captured by the camera and a display image displayed on the display. FIG. 4 is a block diagram illustrating a software configuration of the television device. FIG. 5 is a flowchart illustrating a procedure of correction coefficient calculation processing performed by the television apparatus. FIG. 6 is a diagram illustrating a method for calculating the relative positional relationship between the camera and the mirror. FIG. 7 is a diagram for explaining a transformation matrix by affine transformation. FIG. 8 is a diagram illustrating a method for calculating the relative positional relationship between a mirror and a mirror image. FIG. 9 is a diagram for explaining the relative positional relationship between the coordinates in the camera video and the display coordinates on the display. FIG. 10 is a block diagram illustrating an operation when the camera image is corrected and displayed on the display. FIG. 11 is a flowchart illustrating a procedure of video display processing performed by the television device.

FIG. 1 is an external perspective view showing an example of a television device 1 which is a video display device according to an embodiment. As shown in FIG. 1, the television apparatus 1 has a rectangular appearance when viewed from the front (when viewed from the front). The television apparatus 1 includes a housing 2 and a display 3. The display 3 is composed of, for example, an LCD (Liquid Crystal Display) or the like, receives a video signal from a video processing unit 17 (see FIG. 2) described later, and displays a video such as a still image or a moving image. The housing 2 is provided with a camera 4. The camera can be used for example for the purpose of capturing the television device 1 of the viewer or the like. The housing 2 is supported by the support portion 5.

  FIG. 2 is a block diagram showing a signal processing system of the television apparatus 1. As shown in FIG. 2, the television apparatus 1 selects a broadcast signal of a desired channel by supplying a digital television broadcast signal received by an antenna 12 to a tuner unit 14 via an input terminal 13. It is possible. Tuner A 141 and tuner B 142 are tuners for receiving terrestrial digital broadcasts, and tuner C 143 is a tuner for receiving BS / CS digital broadcasts. Although FIG. 2 shows a case where three tuners are provided, the number of tuners is not limited to this.

  The tuner unit 14 outputs the received broadcast signal to the demodulation / decoding unit 15. The demodulation / decoding unit 15 restores the input broadcast signal to a digital video signal, an audio signal, and the like, and outputs the restored signal to the input signal processing unit 16.

  The television apparatus 1 is provided with an input terminal 21 for directly inputting a digital video signal or audio signal from the outside of the television apparatus 1. The digital video signal and audio signal input via the input terminal 21 are input to the recording / playback processing unit 29.

  The operation unit 24 is provided in the housing 2 and includes a switch for turning on / off the power, a switch for switching a channel, and the like. The receiving unit 26 receives a signal from a remote controller (hereinafter abbreviated as “remote controller”) 25 and outputs the received signal to the control unit 40. The control unit 40 receives instructions for reproduction, channel switching, recording reservation, and the like based on user operations on the operation unit 24 or the remote controller 25.

  The disk drive unit 27 records and reproduces digital data on an optical disk 28 such as a DVD (Digital Versatile Disk). The disc drive unit 27 reads out digital video signals and audio signals from the optical disc 28 based on instructions from the control unit 40, and outputs them to the recording / playback processing unit 29. The disk drive unit 27 records the digital video signal and audio signal input from the recording / playback processing unit 29 on the optical disk 28.

  The HDD 20 performs recording and reproduction with respect to an internal hard disk. The HDD 20 reads digital video signals and audio signals from the hard disk based on instructions from the control unit 40, and outputs them to the recording / playback processing unit 29. The HDD 20 records the digital video signal and audio signal input from the recording / playback processing unit 29 on the hard disk.

  The recording / reproducing processor 29 encrypts digital video signals and audio signals input from the demodulator / decoder 15, the input terminal 21, the disk drive unit 27, the HDD 20, and a network interface (not shown), and converts them into a predetermined recording format. The converted signal is output to the disk drive unit 27 or the HDD 20. The recording / playback processing unit 29 combines the digital video signal and audio signal read from the disk drive unit 27 or the HDD 20 and outputs them to the input signal processing unit 16.

  The input signal processing unit 16 performs predetermined digital signal processing on the digital video signal and audio signal input from the demodulation / decoding unit 15 or the recording / reproduction processing unit 29. The input signal processing unit 16 outputs a digital video signal to the video processing unit 17 and outputs a digital audio signal to the audio processing unit 18.

  The video processing unit 17 converts the digital video signal input from the input signal processing unit 16 or the control unit 40 into an analog video signal in a format that can be displayed on the display 3 and inputs the analog video signal to the display 3. The display 3 displays a program viewing screen, other video, and the like by the analog video signal output from the video processing unit 17.

  The audio processing unit 18 converts the digital audio signal input from the input signal processing unit 16 into an analog audio signal in a format that can be reproduced by the speaker 19 at the subsequent stage, and inputs the analog audio signal to the speaker 19. The speaker 19 reproduces sound based on the analog sound signal input from the sound processing unit 18.

  The control unit 40 includes a tuner unit 14, a recording / playback processing unit 29, an input signal processing unit 16, an operation unit 24, a receiving unit 26, a disk drive unit 27, an HDD (Hard Disk Drive) 20, a network interface (not shown), a bus, They are connected by an interface and control the operation of these units. Further, the control unit 40 controls the above-described units connected to the control unit 40 based on the operation of the operation unit 24 and the remote controller 25 by the viewer, and program viewing, channel switching, program recording reservation, video content, etc. Perform the process.

  Further, as shown in FIG. 2, the control unit 40 provides a central processing unit (CPU) 40a, a ROM (Read Only Memory) 40b storing a control program executed by the CPU 40a, and a work area for the CPU 40a. RAM (Random Access Memory) 40c.

Here, the camera video camera 4 shoots, the positional deviation of a display image that the display 3 displays will be described. FIG. 3 is a diagram for explaining an example of a positional deviation between a camera image captured by the camera 4 and a display image displayed on the display 3. First, since the center position of the camera image taken by the camera 4 is the center position of the lens of the camera 4 and the center position of the display image displayed on the display 3 is the center position of the display 3, these center positions are It will be separated according to the width of the display 3. Further, as shown in FIG. 3, the optical axis of the camera 4 is installed to be inclined toward the center of the display 3 so that a viewer in front of the display 3 can be photographed. Therefore, the normal vector 51 on the lens surface of the camera 4 and the normal vector 52 on the display surface face different directions. Therefore, when displaying the camera video on the display 3, it is necessary to correct the shift in position and direction according to the relative positional relationship between the camera video and the display video on the display 3. In addition, when the camera 4 is installed in the television apparatus 1, there may be individual differences in the position thereof, and therefore it is necessary to correct the installation position and installation direction of the camera 4 in each television apparatus 1.

  FIG. 4 is a block diagram showing a software configuration of the television device 1. The control unit 40 (see FIG. 2) is configured such that the CPU 40a included in the control unit 40 expands and executes the program stored in the ROM 40b on the RAM 40c, thereby executing a pattern generation unit 41 and a pattern detection unit 42 as shown in FIG. The functions as the coordinate analysis unit 43 and the calibration unit 44 are realized. A correction coefficient storage unit 45 is provided in the nonvolatile storage area of the control unit 40. Next, the function of each unit will be schematically described.

  The pattern generation unit 41 generates a pattern image for calibration. The pattern detection unit 42 detects a mirror pattern image obtained by projecting a pattern image for calibration drawn on a mirror or a pattern image displayed on the display 3 onto the mirror.

  The coordinate analysis unit 43 obtains the relative relationship between the coordinates (x, y, z) in the camera image and the coordinates (Xp, Yp, Zp) in the display image on the display 3 by affine transformation, and corrects the transformation matrix of the affine transformation. The coefficient is stored in the correction coefficient storage unit 45 as a coefficient. An affine transformation is a transformation that translates or rotates without deforming the original figure. The relative positional relationship between the coordinates on the original figure and the coordinates on the transformed figure is linear with the conversion coefficient. It is expressed in In addition, the affine transformation includes a transformation in which the enlargement / reduction magnification is different between the vertical direction and the horizontal direction, and a transformation in which the figure is reversed horizontally and vertically.

That is, the relative positional relationship between the coordinates (x, y, z) of an arbitrary point in the camera image and the coordinates (Xp, Yp, Zp) corresponding to the arbitrary point in the display image of the display 3 is expressed by affine transformation. When expressed using the transformation matrix R, it is expressed as the following Equation 1.
(X, y, z) = R (Xp, Yp, Zp) (Formula 1)
The conversion matrix R is a correction coefficient used when correcting the camera image and the display image on the display 3 in the present embodiment. In the correction coefficient calculation process of the present embodiment, the coordinate analysis unit 43 calculates the conversion matrix R. This is the process to do.

  The calibration unit 44 corrects the camera image with the correction coefficient stored in the correction coefficient storage unit 45 and causes the display 3 to display the corrected image. Note that more detailed functions of each unit will be described later.

  Next, correction coefficient calculation processing performed by the television device 1 will be described with reference to FIGS. 4 and 5. The block diagram of FIG. 4 shows an operation when the television apparatus 1 calculates a correction coefficient used for camera image calibration. FIG. 5 is a flowchart showing a procedure of correction coefficient calculation processing performed by the television apparatus 1.

  First, the television apparatus 1 is installed in front of the mirror 6 so that the display 3 and the mirror 6 face each other (step S1). Note that either side of the television device 1 and the mirror 6 may be moved, and the mirror 6 may be installed in front of the television device 1. Further, since the relative positional relationship between the television device 1 and the mirror 6 is corrected in step S2, the television device 1 and the mirror 6 may be installed so as to be approximately parallel in step S1. If the television apparatus 1 and the mirror 6 are strictly parallel, the procedure of step S2 can be omitted.

  Next, the television apparatus 1 calculates the relative positional relationship between the camera 4 and the mirror 6 provided on the display 3 (step S2).

  FIG. 6 is a diagram for explaining a method for calculating the relative positional relationship between the camera 4 and the mirror 6. As shown in FIG. 6, a print pattern 61 is drawn as a pattern image for calibration on the mirror 6 used in the correction coefficient calculation process. The pattern detection unit 42 detects the coordinates of the lattice points P11, P12, P13, P14,... Of the pattern in the camera image 32 of the print pattern 61 photographed by the camera 4.

  In FIG. 6, a reference pattern 31 indicated by a dotted line is a pattern image to be photographed when the camera 4 and the mirror 6 are facing each other. The coordinates of each grid point in the reference pattern 31 correspond to the coordinates of each grid point in the print pattern 61 and are stored in advance in the storage unit (correction coefficient storage unit 45). The coordinates of the reference pattern 31 are used as a reference value of the coordinates for comparison with the pattern image detected by the camera 4. In the following, an example in which the coordinates in the print pattern 61 and the coordinates in the reference pattern 31 correspond one to one will be described.

  The coordinate analysis unit 43 detects the detection coordinates (second detection coordinates) of the grid points P11, P12, P13, P14,... In the camera image 32 of the print pattern 61 and the grid points P1 corresponding to the grid points in the reference pattern 31. , P2, P3, P4,... And the relative positional relationship between them is calculated using affine transformation. The coordinate analysis unit 43 compares the closest lattice points P11 to P14 with the lattice points P1 to P4 in association with each other.

FIG. 7 is a diagram for explaining coordinate transformation by affine transformation. As shown in FIG. 7, the coordinates (x, y, z) of each grid point in the camera image 32 and the coordinates (Xm, Ym, Zm) of each grid point in the print pattern 61 of the mirror 6 (that is, the reference pattern 31). Is expressed using the transformation matrix R1 (second correction coefficient) as shown in the following equation 2.
(X, y, z) = R1 (Xm, Ym, Zm) (Formula 2)

  More specifically, between the lattice point coordinates (Xm, Ym, Zm) and (x, y, z), the rotation angle in the x-axis direction is α, the rotation angle in the y-axis direction is β, and the rotation angle in the z-axis direction is When the rotation angle is γ, the translation amount in the x-axis direction is Tx, the translation amount in the y-axis direction is Ty, and the translation amount in the z-axis direction is Tz, the transformation matrix R1 has 4 rows and 4 columns as follows. It is expressed by a determinant in which four sets of matrices are arranged.

（式３）
式３において、１番目の行列はＺ軸回転を示し、２番目の行列はＹ軸回転を示し、３番目の行列はＸ軸回転を示し、４番目の行列はＸＹＺ各軸方向の平行移動を示す。

(Formula 3)
In Equation 3, the first matrix represents the Z-axis rotation, the second matrix represents the Y-axis rotation, the third matrix represents the X-axis rotation, and the fourth matrix represents the parallel movement in each XYZ axis direction. Show.

  The coordinate analysis unit 43 (calculation means) establishes simultaneous equations of Equation 3 using a plurality of combinations of lattice point coordinates (Xm, Ym, Zm) and (x, y, z), and obtains this solution to obtain an equation. A plurality of unknowns included in the three transformation matrices R1 are calculated. Thereby, the coordinate analysis unit 43 can calculate the transformation matrix R1 in Equation 2 above. That is, if the affine transformation is performed using the rotation angles α, β, γ and the parallel movement amounts Tx, Ty, Tz calculated by the expression 3, the television apparatus 1 can detect the relative positions of the camera 4 and the mirror 6. By correcting the relationship, a state where the camera 4 and the mirror 6 are parallel can be virtually realized.

  Next, the television apparatus 1 calculates the relative positional relationship between the mirror 6 and the mirror image 7 (see FIG. 7) (step S3).

  FIG. 8 is a diagram for explaining a method for calculating the relative positional relationship between the mirror 6 and the mirror image 7. The pattern generation unit 41 generates the reference pattern 31 as a pattern image used for camera video calibration, and outputs the coordinate information of the reference pattern 31 to the video processing unit 17. As an example, the pattern generation unit 41 outputs the coordinates of the grid points P1, P2, P3, P4,... Of the reference pattern 31 to the video processing unit 17 as coordinates used as a reference when displaying the reference pattern 31. The video processing unit 17 displays the reference pattern 31 on the display 3 based on the coordinate information. Thus, the pattern generation unit 41 and the video processing unit 17 function as first display means. Further, the pattern generation unit 41 outputs the display coordinates of the grid points P 1, P 2, P 3, P 4,... Of the reference pattern 31 to the coordinate analysis unit 43. Hereinafter, the reference pattern 31 displayed on the display 3 may be referred to as a display pattern 31.

  Note that the display pattern 31 generated by the pattern generation unit 41 is indicated by a one-dot chain line in FIG. 8, but is not limited thereto. The display pattern 31 may be displayed with a line having a different color from the print pattern 61.

  The pattern detection unit 42 (detection means, second detection means) displays a mirror image pattern 33 that is a mirror image of the display pattern 31 in a camera image obtained by photographing a mirror image of the display pattern 31 projected on the mirror 6 with the camera 4. To detect. Further, the pattern detection unit 42 detects and acquires the coordinates (second detection coordinates) of the lattice points P21, P22, P23, P24,... In the mirror image pattern 33.

  The coordinate analysis unit 43 detects the coordinates of the grid points P21, P22, P23, P24,... Of the mirror image pattern 33 and the grid points P1, P2, P3, P4,. The display coordinates are compared, and the relative positional relationship between them is calculated using affine transformation.

As shown in FIG. 7, the grid point coordinates (Xm, Ym, Zm) in the print pattern 61 of the mirror 6 and the coordinates (Xp ′, Yp ′, Zp ′) in the mirror image 7 (mirror image pattern 33) of the display pattern 31 Is expressed using the transformation matrix R2 of affine transformation as shown in the following equation 4.
(Xm, Ym, Zm) = R2 (Xp ′, Yp ′, Zp ′) (Formula 4)

  More specifically, between the lattice point coordinates (Xm, Ym, Zm) and (Xp ′, Yp ′, Zp ′), the rotation angle in the x-axis direction is δ, and the rotation angle in the y-axis direction is ε, z If the rotation angle in the axial direction is ζ, the translation amount in the x-axis direction is Tx ′, the translation amount in the y-axis direction is Ty ′, and the translation amount in the z-axis direction is Tz ′, the transformation matrix R2 is As shown by a determinant in which four sets of 4 × 4 matrices are arranged.

（式５）
式５において、１番目の行列はＺ軸回転を示し、２番目の行列はＹ軸回転を示し、３番目の行列はＸ軸回転を示し、４番目の行列はＸＹＺ各軸方向の平行移動を示す。

(Formula 5)
In Equation 5, the first matrix represents the Z-axis rotation, the second matrix represents the Y-axis rotation, the third matrix represents the X-axis rotation, and the fourth matrix represents the translation in the XYZ axis directions. Show.

  The coordinate analysis unit 43 establishes simultaneous equations of Equation 5 using a plurality of combinations of lattice point coordinates (Xm, Ym, Zm) and (Xp ′, Yp ′, Zp ′), and obtains this solution to obtain Equation 5 A plurality of unknowns included in the transformation matrix R2 is calculated. Thereby, the coordinate analysis unit 43 can calculate the transformation matrix R2 in the above equation 5.

  Next, the coordinate analysis unit 43 converts the coordinate system of the mirror image 7 (see FIG. 7) to the display coordinate system of the display 3 (step S4). Here, as the coordinates represented in the display coordinate system of the display 3, for example, the coordinates (Xp, Yp, Zp) of the lattice points P1, P2, P3, P4,... Of the display pattern 31 (that is, the reference pattern 31). is there.

As shown in FIG. 7, the relative relationship between the grid point coordinates (Xm, Ym, Zm) in the print pattern 61 of the mirror 6 and the grid point coordinates (Xp, Yp, Zp) in the display pattern 31 is represented by a transformation matrix R3. Can be expressed as in the following formula 6.
(Xm, Ym, Zm) = R3 (Xp, Yp, Zp) (Formula 6)

  Here, the relationship between the lattice point coordinates (Xp ′, Yp ′, Zp ′) of the mirror image pattern 33 and the display coordinates (Xp, Yp, Zp) of the display 3 is a mirror image inversion relationship. That is, in FIG. 7, the display 3 and the mirror image 7 are mirror image symmetric with respect to the mirror 6. As described above with Equation 5, the coordinate system (Xm, Ym, Zm) in the mirror 6 is different from the coordinate system (Xp ′, Yp ′, Zp ′) in the mirror image 7 for each of the x, y, and z axes. Are rotated by δ, ε, and ζ. Therefore, the coordinate system (Xp ′, Yp ′, Zp ′) in the mirror image 7 is −δ, −ε with respect to the x, y, and z axes, respectively, with respect to the coordinate system (Xm, Ym, Zm) in the mirror 6. , Ζ is rotated.

  Accordingly, the coordinate analysis unit 43 obtains the conversion matrix R3 by the following expression 7 by inverting the signs of the rotation angles of the X-axis rotation and the Y-axis rotation in the conversion matrix R2 of Expression 5 to minus (−). be able to.

（式７）

(Formula 7)

  FIG. 9 is a diagram for explaining the relative positional relationship between coordinates (x, y, z) in a camera image and display coordinates (Xp, Yp, Zp) on the display 3. The coordinate analysis unit 43 uses the transformation matrix R1 calculated by Equation 3 and the transformation matrix R3 calculated by Equation 7, and the coordinates (x, y, z) in the camera image and the display coordinates of the display 3 ( The relative positional relationship with Xp, Yp, Zp) is calculated (step S5).

（式９） That is, when Expression 4 is substituted into Expression 2, the relative relationship between the coordinates (x, y, z) in the camera image and the coordinates (Xp, Yp, Zp) in the display pattern is expressed as the following Expression 8. .
(X, y, z) = R1 · R3 (Xp, Yp, Zp) (Formula 8)
Therefore, when Equation 1 and Equation 8 are compared, the transformation matrix R to be obtained is R = R1 · R3. More specifically, Expression 8 is expressed as Expression 9 by Expression 3 and Expression 7.

(Formula 9)

  The coordinate analysis unit 43 then includes α, β, γ, Tx, Ty, Tz included in the transformation matrix R1 calculated in step S1, and δ, ε, ζ, included in the transformation matrix R3 calculated in step S4. A transformation matrix R is calculated by substituting Tx ′, Ty ′, and Tz ′ into Equation 9 (step S5). Then, the coordinate analysis unit 43 stores the calculated conversion matrix R in the correction coefficient storage unit 45 as a correction coefficient (step S6), and ends the correction coefficient calculation process.

  Next, video display processing performed by the television device 1 will be described with reference to FIGS. 10 and 11. FIG. 10 is a block diagram illustrating an operation when the television apparatus 1 corrects the camera video using the correction coefficient stored in the correction coefficient storage unit 45 and causes the display 3 to display the corrected camera video. FIG. 11 is a flowchart showing a procedure of video display processing performed by the television apparatus 1.

  First, in the television apparatus 1, when the control unit 40 outputs a shooting-on signal for starting shooting to the camera 4, the camera 4 starts shooting (step S11). The calibration unit 44 corrects the video (camera video) captured by the camera 4 using the correction coefficient (conversion matrix R) stored in the correction coefficient storage unit 45 (step S12). The calibration unit 44 outputs the corrected camera video to the video processing unit 17. The video processing unit 17 displays the corrected camera video on the display 3 (step S13). Thus, the calibration unit 44 and the video processing unit 17 function as second display means.

  As described above, according to the present embodiment, the figure displayed on the display is displayed on the mirror, the detected coordinates of the mirror image obtained by photographing the figure displayed on the mirror with the camera, and the display on which the figure is displayed. Based on the coordinates, the display coordinates of the camera image on the display are corrected. As a result, calibration can be performed if there is a self-device and a mirror. Therefore, it is possible to provide a video display device, a video display method, and a program that facilitate calibration of a camera fixed to the television device 1. .

  In the affine transformation expressed by the above-described equations 1 to 3, the transformation matrices R1 to R3 and R may be calculated including various parameters related to the camera optical system.

  In the above description, the correction coefficient is obtained by using the mirror 6 having the correction pattern printed on the surface. However, when the mirror 6 and the display 3 are installed completely in parallel, the pattern is printed. The correction coefficient calculation process may be performed using a mirror that is not present. In this case, the correction coefficient R1 described above is unnecessary (that is, R1 = 1), and therefore the correction coefficient R may be calculated as R = R3.

  Further, the graphic printed on the mirror 6 and the graphic displayed on the display 3 may not be the same. For example, the pattern interval in the print pattern may be different from the pattern interval in the display pattern. Further, the print pattern may be a pattern in which only lattice points are printed. Furthermore, other figures may be used for the print pattern and the display pattern. As other figures, for example, a checkerboard pattern may be used.

  Note that the program executed by the television apparatus 1 of the present embodiment is provided by being incorporated in advance in a ROM or the like. A program executed in the television apparatus 1 of the present embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, and a DVD (DigitalTalsaTile Disk). The information may be provided by being recorded on a recording medium that can be read by the user.

  Furthermore, the program executed by the television device 1 of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed by the television apparatus 1 of the present embodiment may be provided or distributed via a network such as the Internet.

  The mirror 6 of the present embodiment and the program may be provided as a camera video calibration kit. Further, the present embodiment may be applied to the case where a camera is added later to the television apparatus 1 that does not include a camera. That is, when the number of cameras is increased, the above-described correction coefficient calculation process may be performed, and camera video calibration may be performed using the calculated correction coefficient.

  Moreover, although the case where the camera 4 was fixed to the display 3 was demonstrated above, the attachment location of the camera 4 is not limited to this. The camera 4 may be attached to other locations such as the support portion 5 of the television apparatus 1.

  In the above description, the correction coefficient calculation process is performed by arranging the display 3 and the mirror 6 so as to face each other. However, the arrangement relationship between the display 3 and the mirror 6 is not limited to this. As another example, an optical system such as a mirror in which a pattern is not drawn is newly disposed between the display 3 and the mirror 6, and the pattern image on the mirror 6 is diffracted by the optical system and then is captured by the camera 4. You may shoot.

  In addition, although the example which applied the video display apparatus of this embodiment to the television apparatus 1 was demonstrated above, this embodiment is not limited to this, Information processing apparatus, such as PC, and various display apparatuses You may apply. In the above description, the case where the television apparatus 1 is a planar display type has been described. However, the present embodiment may be applied to a stereoscopic display type television apparatus.

  As an application example of the present embodiment, the position of a person or face detected in a camera image may be corrected using the correction coefficient of the present embodiment and displayed on the display 3.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Television apparatus, 2 ... Housing | casing, 3 ... Display, 4 ... Camera, 5 ... Support part, 6 ... Mirror, 7 ... Mirror image, 24 ... Operation part, 31 ... Reference pattern (display pattern), 32 ... Camera image , 33 ... mirror image pattern, 40 ... control unit, 41 ... pattern generation unit, 42 ... pattern detection unit, 43 ... coordinate analysis unit, 44 ... calibration unit, 45 ... correction coefficient storage unit, 51 ... normal vector, 52 ... Normal vector, 61 ... Print pattern

Claims (9)

And the camera,
And de-Isupurei,
A first display means for displaying the predetermined graphic reference to a predetermined display coordinates on the display,
On the camera image obtained by photographing the mirror image of the figure projected on the mirror with the camera, detection means for acquiring detection coordinates for detecting the mirror image;
Based on the relative positional relationship between the detected coordinates of the mirror image and the display coordinates of the predetermined graphic, the coordinates in the captured camera images in the camera to calculate a correction coefficient of order to correspond to the display coordinates of the display A calculation means;
Position detecting means for detecting the position of a human face included in the camera image;
Correction means for correcting the position of the face with respect to the display coordinates of the display by the correction coefficient ;
A video display device comprising:   The calculation means reverses the detection coordinates of the mirror image of the figure on the camera image to calculate the display coordinates serving as a reference for displaying the predetermined figure, thereby calculating the coordinates in the camera image, The video display device according to claim 1, wherein the correction coefficient is calculated by obtaining a relative positional relationship with display coordinates of the display. The detection means further detects, as second detection coordinates, detection coordinates at which the figure is detected on a camera image obtained by photographing the predetermined figure drawn on the mirror,
The calculating means calculates a second correction coefficient for correcting the relative positional relationship between the camera and the mirror based on the relative positional relationship between the display coordinates of the predetermined graphic and the second detection coordinates; calculating the correction coefficient using the second correction coefficient, the image display device according to claim 1 or 2. The first display means displays a lattice-like pattern having lattice points at a plurality of point coordinates on the display as the predetermined figure ,
The detecting device, the camera image obtained a mirror image of the lattice pattern projected on the said mirror and captured by the camera, detects the detected coordinates of each grid point,
The calculating means, the plurality of point coordinates, and calculates the correction factor based on the relative positional relationship between the detected coordinates of the grid points corresponding to the point coordinates, the image according to claim 1 or 2 Display device.   The video display apparatus according to claim 1, wherein the calculation unit calculates the correction coefficient using affine transformation.   The video display device according to claim 1, wherein the calculation unit calculates the correction coefficient including various parameters related to an optical system of the camera. And the camera,
And de-Isupurei,
Wherein to display predetermined graphic reference to a predetermined display coordinates on the display, on a camera image obtained a mirror image of the figure projected on the mirror taken with the camera, the detected coordinates of the mirror image of the figure detect, because in correspondence said calculated based predetermined display coordinates of the graphics and the relative positional relationship between the detected coordinates of the mirror image, the coordinates in the captured camera images in the camera to the display coordinates of the display Storage means for storing the correction coefficient of
Position detecting means for detecting the position of a human face included in the camera image;
Correction means for correcting the position of the face with respect to the display coordinates of the display by the correction coefficient ;
A video display device comprising: And the camera,
And de-Isupurei,
Wherein to display predetermined graphic reference to a predetermined display coordinates on the display, on a camera image obtained a mirror image of the figure projected on the mirror taken with the camera, the detection coordinates detected the mirror image acquired, because in correspondence said calculated based predetermined display coordinates of the graphics and the relative positional relationship between the detected coordinates of the mirror image, the coordinates in the captured camera images in the camera to the display coordinates of the display A video display device comprising: a storage unit that stores the correction coefficient of
A position detection step for detecting the position of a human face included in the camera image;
A correction step of correcting the position of the face with respect to the display coordinates of the display by the correction coefficient. And the camera,
And de-Isupurei,
Wherein to display predetermined graphic reference to a predetermined display coordinates on the display, on a camera image obtained a mirror image of the figure projected on the mirror taken with the camera, the detection coordinates detected the mirror image acquired, because in correspondence said calculated based predetermined display coordinates of the graphics and the relative positional relationship between the detected coordinates of the mirror image, the coordinates in the captured camera images in the camera to the display coordinates of the display A computer that controls the video display device, the storage unit storing the correction coefficient of
Position detecting means for detecting the position of a human face included in the camera image;
A program for functioning as correction means for correcting the position of the face with respect to the display coordinates of the display by the correction coefficient.

Images