JP2006318385A - Image generation system, program, and information storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image generation system capable of simply and appropriately displaying a photographed image and an instruction image. <P>SOLUTION: The image generation system comprises an input part 160 to input viewpoint setting information which indicates a viewpoint, a storage part 120 to store 3D motion data 121 indicating operations of an instructor and photographing data 123 indicating operations of a photograph target person, and an image generation part 111 to generate an instruction image showing operations of the instructor captured from a specified viewpoint from the viewpoint setting information and the 3D motion data 121, and to generate a photographed image showing operations of the photograph target person from the photographing data 123. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image generation system, a program, and an information storage medium that generate a teaching image.

  For example, a user who performs a golf swing, dance, or the like reproduces a video image as a teaching image when imitating a teacher's motion in the teaching image.

  However, with this method, it is difficult for the user to determine whether the teacher's motion in the teaching image matches his / her motion.

  When judging such suitability, for example, the user compares his / her teacher's motion with his / her motion by capturing his / her motion with a video and playing the captured image. It is possible.

  However, in general, the viewpoint of the teaching image, the size of the teacher in the teaching image, etc. do not match the viewpoint of the captured image, the size of the user in the captured image, etc. It was difficult to do.

In order to solve such a problem, it is conceivable that the user converts his / her motion into motion data. For example, Patent Document 1 describes a method of attaching a marker to his / her limbs, and Patent Document 2 describes a method of using a plurality of cameras.
JP 7-185131 A JP 2001-293120 A

  However, in the methods of Patent Documents 1 and 2, the user needs to attach a marker to his / her limbs or the like, and to arrange a plurality of cameras at appropriate positions in order to obtain motion data appropriately. In addition, it required large equipment and was not easy to do at home.

  An object of the present invention is to provide an image generation system, a program, and an information storage medium capable of easily and appropriately presenting a captured image and a teaching image.

In order to solve the above-described problems, an image generation system according to the present invention includes:
An input unit for inputting viewpoint setting information indicating a viewpoint;
A storage unit for storing three-dimensional motion data indicating the operation of the teacher and imaging data indicating the operation of the imaging subject;
Based on the viewpoint setting information and the three-dimensional motion data, a teaching image indicating the operation of the instructor when viewed from the viewpoint is generated, and the operation of the imaging subject is illustrated based on the imaging data. An image generation unit for generating a captured image;
It is characterized by including.

The program according to the present invention is
Computer
An input unit for inputting viewpoint setting information indicating a viewpoint;
A storage unit for storing three-dimensional motion data indicating the operation of the teacher and imaging data indicating the operation of the imaging subject;
Based on the viewpoint setting information and the three-dimensional motion data, a teaching image indicating the operation of the instructor when viewed from the viewpoint is generated, and the operation of the imaging subject is illustrated based on the imaging data. It is made to function as an image generation part which produces | generates a captured image.

  An information storage medium according to the present invention is a computer-readable information storage medium and stores the above program.

  According to the present invention, the image generation system can generate a teaching image indicating a teacher's operation when viewed from a user's desired viewpoint by using the three-dimensional motion data indicating the teacher's operation. . Thereby, the user can confirm the operation | movement of a teacher from the viewpoint match | combined with the viewpoint of his captured image.

  Therefore, the image generation system can present the captured image and the teaching image simply and appropriately.

In the image generation system, the program, and the information storage medium,
The part other than the teacher in the teaching image is transparent or translucent,
The image generation unit may generate a comparative image in which the teaching image and the captured image are overlapped.

  According to this, the image generation system can present the difference between the captured image and the teaching image in an easy-to-understand manner by generating a comparative image in which the teaching image and the captured image are superimposed.

Further, the image generation system determines whether at least one color constituting the teacher in the teaching image matches at least one color constituting the operation subject in the captured image. Including
When the determination unit determines that the colors match, the image generation unit may generate the comparative image by changing a portion of the matching color in the teaching image or the captured image to another color.

Further, the program and the information storage medium determine whether at least one color constituting the teacher in the teaching image matches at least one color constituting the operation target in the captured image. Make the computer function as a determination unit
When the determination unit determines that the colors match, the image generation unit may generate the comparative image by changing a portion of the matching color in the teaching image or the captured image to another color.

  According to this, when the image generation system generates a comparative image in which the teaching image and the captured image are overlapped by changing the color, the difference between the captured image and the teaching image can be presented in an easily understandable manner. Can do.

In the image generation system, the program, and the information storage medium,
The image generation unit may generate a comparative image in which the teaching image and the captured image are arranged in different regions.

  According to this, the image generation system can present the difference between the captured image and the teaching image in an easily understandable manner by arranging the teaching image and the captured image in different regions.

In the image generation system, the program, and the information storage medium,
The storage unit stores address data indicating an address of a portion serving as at least one key frame of the teaching image in the three-dimensional motion data,
The input unit inputs designation information indicating the key frame,
The image generation unit may generate the teaching image of a portion based on the key frame based on the designation information, the address data, and the three-dimensional motion data.

  According to this, the image generation system can smoothly generate a teaching image of a part based on the key frame in accordance with a user's designation.

  Hereinafter, a case where the present invention is applied to a mobile phone functioning as an image generation system will be described as an example with reference to the drawings. In addition, the Example shown below does not limit the content of the invention described in the claim at all. In addition, all the configurations shown in the following embodiments are not necessarily essential as means for solving the invention described in the claims.

  FIG. 1 is an overall view of a system in which the image generation system in the present embodiment is used.

  The system shown in FIG. 1 includes a mobile phone 10 that functions as an image generation system, a server 20, and a network 30.

  The mobile phone 10 transmits and receives various information to and from the server 20 via the network 30. The user of the mobile phone 10 displays various images on the display of the mobile phone 10. In this embodiment, the user images his / her dance operation using the camera of the mobile phone 10, and then displays a comparison image for comparing the dance operation on the display of the mobile phone 10.

  FIG. 2 is a diagram illustrating an example of the comparative image 200 in the present embodiment.

  In the comparative image 200, a teaching image 210 indicating a teacher's operation is displayed in the upper half area, and a captured image 220 indicating a user's (imaging target) operation is displayed in the lower half area.

  As described above, the mobile phone 10 displays the teaching image 210 and the captured image 220 side by side, thereby allowing the user to easily check whether the user's operation is compatible with the operation of the teacher.

  In addition, the teaching image 210 in the present embodiment is generated from three-dimensional motion data (hereinafter referred to as “3D motion data”) so that the operation of the teacher can be confirmed from an arbitrary viewpoint. As a result, the user can adjust the viewpoint of the teaching image 210 in accordance with the viewpoint of the captured image 220. As shown in FIG. 2, the user compares the teaching image 210 and the captured image 220 set at substantially the same viewpoint. The suitability of the operation can be determined.

  Next, functional blocks when such a function is implemented in the mobile phone 10 will be described.

  FIG. 3 is a functional block diagram of the mobile phone 10 in the present embodiment.

  The mobile phone 10 includes a processing unit 110 that performs image generation, a storage unit 120 that stores various data, an input unit 160 that inputs viewpoint setting information indicating a viewpoint, an imaging unit 170, a display 191, A sound output unit 192, a communication unit 196, and an information storage medium 180 are included.

  Further, the processing unit 110 includes an image generation unit 111 that generates various images, a determination unit 112 that performs various determinations, a control unit 113 that controls the imaging unit 170 and the like, an update unit 114 that updates data, And a sound generation unit 115 that generates sound information.

  In addition, the storage unit 120 includes 3D motion data 121 indicating a teacher's operation, address data 122 indicating the address of a part of the 3D motion data 121 serving as a key frame of the teaching image, and imaging data generated by the imaging unit 170. 123 and the like are stored.

  Note that the 3D motion data 121 is generated as three-dimensional data indicating the operation of the teacher by an optical, magnetic, mechanical, or other motion capture system.

  The image generation unit 111 generates an image, and the display 191 displays the image generated by the image generation unit 111. The sound generation unit 115 generates sound information, and the sound output unit 192 outputs sound effects and the like based on the sound information.

  3 include, for example, a CPU and an image processing circuit as the processing unit 110, a RAM and an HDD as the storage unit 120, a button as the input unit 160, and an imaging unit 170 as the imaging unit 170. A CMOS camera, a CCD camera, or the like can be mounted on a computer using a generally used LCD or the like as a display 191, a speaker or the like as a sound output unit 192, and a wireless communication module or the like as a communication unit 196.

  Furthermore, it is also possible for the computer to implement functions such as the image generation unit 111 by reading a program from the information storage medium 180 or the server 20 via the communication unit 196. As the information storage medium 180, for example, a CD-ROM, DVD-ROM, IC card, ROM, RAM, memory card, HDD, or other storage medium using laser or magnetism can be applied. Further, the method of reading the program from the information storage medium 180 may be a contact type or a non-contact type.

  Next, the flow of processing from confirmation processing to comparison processing using these units will be described. Note that the user first confirms the operation of the teacher, and then determines whether or not the operation of the teacher is compatible with the user's operation.

  FIG. 4 is a flowchart showing the flow of processing from the confirmation processing to the comparison processing in the present embodiment.

  First, the mobile phone 10 performs a confirmation process (step S1).

  FIG. 5 is a flowchart of the confirmation process in the present embodiment.

  The determination unit 112 determines whether there is an instruction to display the teaching image 210 based on the operation information from the input unit 160 (step S11).

  When there is a display instruction, the determination unit 112 determines whether or not viewpoint setting information is included in the operation information from the input unit 160, that is, whether or not there is a viewpoint change instruction (step S12).

  When there is a viewpoint change instruction, the image generation unit 111 changes the viewpoint (viewpoint position and line-of-sight direction) based on the viewpoint setting information (step S13), and when viewed from the viewpoint based on the 3D motion data 121. A teaching image 210 is generated (step S14).

  On the other hand, when there is no viewpoint change instruction, the image generation unit 111 generates a teaching image 210 when viewed from a predetermined viewpoint based on the 3D motion data 121 (step S14).

  Then, the display 191 displays the teaching image 210 (Step S15).

  Through the above procedure, the teaching image 210 is displayed, and the user can confirm the operation of the teacher.

  After confirming the operation of the teacher, the user images his / her operation using the imaging unit 170. More specifically, when the operation information of the imaging instruction is input by the input unit 160, the control unit 113 controls the imaging unit 170 to perform imaging.

  Then, the determination unit 112 determines whether there is input of the imaging data 123 from the imaging unit 170 (step S2).

  When the imaging data 123 is input, the update unit 114 stores the imaging data 123 in the storage unit 120.

  Then, the cellular phone 10 executes a comparison process for comparing the operation of the teacher and the operation of the user (step S4).

  FIG. 6 is a flowchart of the comparison process in the present embodiment.

  The determination unit 112 determines whether there is an instruction to display the comparative image 200 based on the operation information from the input unit 160 (step S21).

  When there is a display instruction, the determination unit 112 determines whether or not viewpoint setting information is included in the operation information from the input unit 160, that is, whether or not there is a viewpoint change instruction (step S22).

  When there is a viewpoint change instruction, the image generation unit 111 changes the viewpoint based on the viewpoint setting information (step S23), and based on the 3D motion data 121, the captured image 220 and the teaching image 210 when viewed from the viewpoint are displayed. Is generated (step S14).

  On the other hand, when there is no instruction to change the viewpoint, the image generation unit 111 generates a comparison image 200 including the captured image 220 and the teaching image 210 when viewed from a predetermined viewpoint, based on the 3D motion data 121 (step S24). .

  Then, the display 191 displays the comparison image 200 (Step S25).

  Through the above procedure, the comparison image 200 is displayed, and the user can confirm the operation while comparing the operation of the instructor with his own operation.

  As described above, according to the present embodiment, the image generation system can generate the teaching image 210 indicating the operation of the teacher when viewed from the viewpoint desired by the user. Thereby, the user can confirm the operation of the teacher from the viewpoint that matches the viewpoint of the captured image 220 of the user or the desired viewpoint of the user.

  As a result, the user can also check a portion that has been hidden in a conventional teaching image such as a video image.

  Further, according to the present embodiment, the image generation system generates the comparison image 200 in which the teaching image 210 and the captured image 220 are arranged in different regions, so that the user can easily grasp the difference between the two. You can easily determine whether your behavior is appropriate.

  Furthermore, according to the present embodiment, the user can easily generate the captured image 220 by capturing an image with the camera built in the mobile phone 10.

  Therefore, the image generation system can present the captured image 220 and the teaching image 210 simply and appropriately.

(Modification)
In addition, application of this invention is not limited to the Example mentioned above, A various deformation | transformation is possible.

  For example, in the above-described embodiment, the mobile phone 10 displays the teaching image 210 and the captured image 220 side by side, but displays the teaching image 210 and the captured image 220 alternately. Or may be displayed.

  FIG. 7 is a diagram illustrating another example of the comparative image 202 in the present embodiment.

  In the comparative image 202, the background portion of the teacher in the teaching image 210 is transparent (may be translucent). The image generation unit 111 generates the comparison image 202 by superimposing the teaching image 210 on the captured image 220.

  As a result, as shown in FIG. 7, the comparison image 202 in a state where the teacher and the user overlap each other is displayed, so that the user can easily discriminate the difference in operation.

  In this case, the determination unit 112 may determine whether at least one color constituting the teacher in the teaching image 210 matches at least one color constituting the user in the captured image 220.

  Then, if the determination unit 112 determines that the colors match, the image generation unit 111 may generate the comparison image 202 by changing the matching color portion in the teaching image 210 or the captured image 220 to another color. Good.

  According to this, when the image generation system generates the comparison image 202 in which the teaching image 210 and the captured image 220 are overlapped by changing the color, the difference between the captured image 220 and the teaching image 210 is further increased. It can be presented in an easy-to-understand manner.

  Further, the image generation unit 111 may generate the comparison image 202 based on the user operation information from the input unit 160 by shifting the position of the teacher in the comparison image 202 from the user position, as shown in FIG. Good.

  Further, the input unit 160 may input designation information indicating a key frame. The image generation unit 111 then uses the key information as a reference (for example, several frames before and after the key frame, several frames after the key frame, key frame, based on the designation information, the address data 122, and the 3D motion data 121. A teaching image 210 of several previous frames or the like may be generated.

  According to this, the image generation system can smoothly generate the teaching image 210 of the part based on the key frame in accordance with the user's designation.

  More specifically, for example, in the case of golf swing motion, the mobile phone 10 sets each of address, top, impact, and finish as key frames, and when the user designates the top, several frames before and after the top state The teaching image 210 may be displayed.

  In addition, the mobile phone 10 uses the 3D motion data 121 to generate the teaching image 210 for each part of the teacher (right arm, left foot, head, etc.), or generate the teaching image 210 by enlarging or reducing it. The playback speed of the teaching image 210 may be adjusted, or the playback start position of the teaching image 210 may be adjusted.

  The teacher image in the teaching image 210 may be a captured image of the teacher or a character image.

  Moreover, the user's imaging data 123 may be not only one type but also a plurality of types. For example, the image generation unit 111 may generate a comparative image in which a captured image based on user-specified imaging data and a captured image based on the latest imaging data are arranged. According to this, the user can compare his past motion with the current motion.

  The mobile phone 10 may use the captured image 220 based on the captured data 123 as a so-called standby image.

  Further, the communication unit 196 may transmit the imaging data 123 to another mobile phone or the like. A mobile phone or the like to which the imaging data 123 is transmitted may display the captured image.

  Further, the server 20 may distribute the 3D motion data 121 to the mobile phone 10. According to this, since the 3D motion data 121 is updated, the teaching image 210 is also updated, so that the user can use the system for a long time.

  Further, the 3D motion data 121 may be configured by a wire frame. In this case, the image generation unit 111 may generate the teaching image 210 by pasting a texture based on the texture data on the wire frame.

  The image generation system can be mounted not only on the mobile phone 10 but also on various devices such as a home game device, an arcade game device, a portable game device, a PC, and a PDA.

  For example, when the image generation system is mounted on a portable game device, the portable game device inputs the user's image data 123 imaged with a digital video camera or the like, thereby providing the same effects as the above-described cellular phone 10. Can be played.

  The application target of the present invention is not limited to the dance described above, and the present invention can be applied to sports such as golf and baseball, ballet, theater, fitness, musical instrument performance, and the like.

1 is an overall view of a system in which an image generation system in the present embodiment is used. It is a figure which shows an example of the comparative image in a present Example. It is a functional block diagram of the mobile phone in a present Example. It is a flowchart which shows the flow of a process from the confirmation process in a present Example to a comparison process. It is a flowchart of the confirmation process in a present Example. It is a flowchart of the comparison process in a present Example. It is a figure which shows another example of the comparative image in a present Example.

Explanation of symbols

10 Mobile phone (image generation system)
20 server 111 image generation unit 112 determination unit 120 storage unit 121 3D motion data 122 address data 123 imaging data 160 input unit 170 imaging unit 180 information storage medium 196 communication unit 200, 202 confirmation image 210 teaching image 220 captured image

Claims (7)

An input unit for inputting viewpoint setting information indicating a viewpoint;
A storage unit for storing three-dimensional motion data indicating the operation of the teacher and imaging data indicating the operation of the imaging subject;
Based on the viewpoint setting information and the three-dimensional motion data, a teaching image indicating the operation of the instructor when viewed from the viewpoint is generated, and the operation of the imaging subject is illustrated based on the imaging data. An image generation unit for generating a captured image;
An image generation system comprising: In claim 1,
The part other than the teacher in the teaching image is transparent or translucent,
The image generation system generates a comparison image in which the teaching image and the captured image are overlapped with each other. In claim 2,
A determination unit that determines whether at least one color constituting the teacher in the teaching image matches at least one color constituting the operation target person in the captured image;
When the determination unit determines that the colors match, the image generation unit generates the comparison image by changing a part of the matching color in the teaching image or the captured image to another color. Image generation system. In claim 1,
The image generation system generates a comparison image in which the teaching image and the captured image are arranged in different regions. In any one of Claims 1-4,
The storage unit stores address data indicating an address of a portion serving as at least one key frame of the teaching image in the three-dimensional motion data,
The input unit inputs designation information indicating the key frame,
The image generation system, wherein the image generation unit generates the teaching image of a portion based on the key frame based on the designation information, the address data, and the three-dimensional motion data. Computer
An input unit for inputting viewpoint setting information indicating a viewpoint;
A storage unit for storing three-dimensional motion data indicating the operation of the teacher and imaging data indicating the operation of the imaging subject;
Based on the viewpoint setting information and the three-dimensional motion data, a teaching image indicating the operation of the instructor when viewed from the viewpoint is generated, and the operation of the imaging subject is illustrated based on the imaging data. A program that functions as an image generation unit that generates a captured image.   A computer-readable information storage medium storing the program according to claim 6.

Images