KR100934391B1 - Hand-based Grabbing Interaction System Using 6-DOF Haptic Devices - Google Patents

Abstract

The hand-based catching interaction system using the 6-degree of freedom haptic device according to the present invention comprises a hardware unit for generating hand tracking information according to a change in the position of a finger and a hand, and controlling the position and rotational reaction of the hand based on the force information. , Using a hand tracking information to control the virtual hand model in the virtual environment, a collision processing unit for generating collision object information when a collision occurs between the virtual hand model and the object, receiving collision object information from the collision processing unit, and unique characteristics of the object A dynamics processor which provides the motion information of the virtual hand model and the object according to the collision, and updates the virtual environment data based on the motion information, and generates the hand joint information based on the change of the virtual hand model according to the update. The physical-based motion visualization unit, and the sense of force information is provided to the device unit based on the hand joint information. It includes a force rendering unit for controlling the position of the hand and the sense of rotation reaction force.

As such, the present invention can provide a natural and direct interaction by simulating a grabbing operation using haptic equipment in applications such as provisional review and virtual training.

Description

Grasp Interaction using 6DOF Haptic Device and Method Thereof}

The present invention relates to a system capable of hand-based grabbing interaction with a virtual environment using a haptic device, and more particularly, to track the movement of a finger and the entire hand of a hand in the real space, and using this to virtualize the virtual space. The present invention relates to a hand-based grabbing interaction system using a six degree of freedom haptic device that enables a user to interact naturally with a virtual environment by precisely and precisely controlling a hand model.

Hand-based grabbing interaction technology refers to a technology relating to the interaction between the human hand and each object of the virtual environment. The hand-based catching interaction technology is a human-oriented interaction technology that provides a higher level of realism by supplementing existing visual and auditory effects, and is widely used in the field of games and the like. In particular, application applications that require realistic interactions such as virtual review during the virtual product production process require various catching interactions according to the characteristics of virtual objects in the virtual environment.

As the first conventional technology related to the hand interface technology, Korean Patent Laid-Open Publication No. 2002-0096807, Armored data input device and a recognition method thereof are for inputting data of a mobile phone or a keyboard using a pressure sensor of a finger, and the data of a small terminal device. There is an advantage to facilitate input.

As a second conventional technique, Korean Patent Laid-Open Publication No. 2001-0023426, a fingerless globe for dialogue with a data processing system includes a curved sensor for detecting a user's finger curve and a sensor for detecting a curve at a specific range of angle and acceleration. Sensors can be used to give the effect of simple virtual typing to select specific items on a computer in a portable environment.

U.S. Patent Publication No. 2004/0164880 A1, Wearable Data Input Device Employing Wrist and Finger Movements, uses a switchable sensor for wrist and finger movement to replace the input of information using a common keyboard to locate the key keyboard. Globe-type interface device that can be effective. Wrist movements, finger movements, and keyboard layouts can be mapped and used as key input means.

However, the first conventional technology has a disadvantage in that it does not support a high-reality catching interaction method capable of simulating a virtual design, evaluation, assembly, etc. of a product.

The second conventional technology is insufficient to be used as an interactive interface that can be used in an environment requiring high realism, such as virtual design, evaluation, and assembly, and the third conventional technology is suitable for manipulating virtual reality objects through precise grabbing support. not.

For this reason, virtual reality applications in a work environment where high realism and smooth interaction such as virtual review and virtual training are supported require a system that supports a natural and direct interaction method that matches the real space.

The present invention provides a system to which interactions such as an intuitive and empirical constraint-based grabbing method reflecting the unique characteristics of an object and a physics-based grabbing method reflecting physical elements such as actual friction, gravity, and elasticity are selectively applied.

The hand-based catching interaction system using the 6-degree of freedom haptic device according to the present invention comprises a hardware unit for generating hand tracking information according to a change in the position of a finger and a hand, and controlling the position and rotational reaction of the hand based on the force information. A collision processor configured to control a virtual hand model in a virtual environment using the hand tracking information, generate collision object information when a collision occurs between the virtual hand model and an object, and receive collision object information from the collision processor; A dynamics processor which provides the virtual hand model according to the collision and the motion information of the object based on a unique characteristic of the object, and updates the virtual environment data based on the motion information, and the virtual hand according to the update A physical-based motion visualization unit for generating hand joint information based on a model change; And a force generator for providing the force information to the device based on the hand joint information to control the position and the rotational reaction force of the hand.

The present invention provides a natural and direct interaction by using haptic equipment to simulate a grab motion in applications such as hypothetical review and virtual training, and thus provides a hand-based approach to virtual environment applications that rely only on conventional three-dimensional visual information. Interactions have the effect of providing a simulation environment for intuitive product design, evaluation, and assembly.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

In the preferred embodiment of the present invention, a system that can simulate a catching motion using haptic equipment in an application such as virtual training will be described.

1 is a block diagram illustrating a hand-based grabbing interaction system using a six degree of freedom haptic device according to a preferred embodiment of the present invention. The hardware unit 100 including the device 120 and the mounting device 130 on which the 6 degrees of freedom haptic device 120 is mounted, and a software unit for performing catching and reverse processing through hand simulation when a collision with a virtual object is detected. And 150. Here, the software unit 150 and the hardware unit 100 are connected through the haptic device driver 170.

Such a system according to a preferred embodiment of the present invention is intuitive, reflecting the unique characteristics of the object, and interactions such as a constraint-based catching method using experience and a physical-based catching method reflecting physical elements such as actual friction, gravity, and elasticity. This is optional.

The hardware unit 100 senses the movement of the finger joint using the hand interface 110 and transmits a reaction force to the entire hand through the six degree of freedom haptic device 120.

The hand interface 110 is a glove-shaped globe, and is equipped with a micro absolute sensor that measures a finger joint angle with respect to a user's hand movement, and an analog signal sensed by the micro absolute sensor, that is, the joint of each finger. Calculate the joint angle of each finger based on the signal for the bend value. The calculated joint angles are used to control the movement of the virtual hand model and to interact with the virtual hand model and the objects in the virtual environment.

The six-degree of freedom haptic device 120 may detect the hand position information and the rotation information, and control the hand reaction force by controlling the hand position and the rotation based on the inverse information provided from the software unit 150.

The six degrees of freedom haptic device 120, as shown in Figure 2, is composed of a position control mechanism of three degrees of freedom and three degrees of freedom attitude control mechanism to control the position and rotational reaction of the entire hand. It is composed of Base Rotate (position control first axis) 200, Main Link (position control second axis) 201, Sub Link (position control third axis) 202 for position control of three degrees of freedom, It consists of a hand roll (first wrist rotation) (210), a hand pitch (second wrist rotation) 211, and a hand yaw (third rotation wrist) 212 for the posture control of three degrees of freedom.

The motion consisting of three axes 210, 211, and 212 of the attitude control mechanism is represented by the motion of three rotational axes perpendicular to each other with respect to a point.

The movement consisting of three axes 200, 201, 202 of the position control mechanism is in the form of a hemisphere with two systems. The posture control mechanism represents the posture of the hand, that is, the direction, and the position control mechanism represents the position of the hand in space.

This combination makes it easier to control the instrument, and detects the position and rotation information of the hand, and controls the position and rotation of the hand through the control of the instrument to control the sense of reaction force of the hand.

The six degrees of freedom haptic device 120 is mounted on the mounting device 130, which can move back and forth, left and right as shown in Figure 3a, the six degrees of freedom haptic device 120 is also mounted by the mounting device in accordance with the user's movement in space By moving together, it compensates for a wider range of activity than when fixed at a point. This mounting apparatus 120 is composed of a power supply 300, a controller 302, a motor 304, an encoder 306, and a bridge module 308. The leg module 308 includes a straight input 310, a steering input 312, a straight output wheel 314, a steering output body 316, to enable straight input and steering input, as shown in FIG. 4B. A steering stationary body 318 and a straight secondary wheel 320.

The joint angle of the finger calculated by the hand interface 110 and the position and rotation information of the hand detected by the six degree of freedom haptic device 120 are provided to the software unit 150 through the haptic device driver 170. Here, the position and rotation information of the hand detected by the joint angle of the finger and the six degrees of freedom haptic device 120 are defined as hand tracking information.

The software unit 150 selectively simulates two catching motions upon collision detection based on a unique feature of the virtual object, and transmits a sixth degree of freedom haptic device 120 to transmit a rotational reaction to the entire hand through reverse rendering. It transmits the sense of force information to, and includes a collision processing unit 152, dynamics processing unit 154, a physical-based motion visualization unit 156, a force rendering unit 158 and the device control unit 160.

Here, the dynamics processing unit 154 is composed of a configuration-based interaction module 154a and a physical-based interaction module 154b that operate according to the unique characteristics of the virtual object.

The collision processing unit 152 of the software unit 150 receives the hand tracking information and the virtual environment data provided from the virtual environment production unit (not shown), updates the virtual hand model and the transformation matrix for controlling the virtual environment, and performs real-time collision processing. And interaction processing between hand gestures. In this case, the collision processor 152 controls the virtual hand model expressing the smooth joint by applying a changeable skin deformation technique for realistic and accurate hand correction work. This is done through calibration.

In particular, the collision processor 152 processes collision processing calculations between the millions of polygon data models and the virtual hand models in real time in order to increase the sensitivity, and for this purpose, the virtual environment data is loaded into a scene graph structure for real-time impulse. Hierarchical collision detection is supported. Here, the hierarchical collision searching technique supported by the collision processing unit 152 is a method of pre-classifying and processing partial objects of interest for a large data model and real-time collision processing, thereby reducing computation time required for real-time work. Can be.

The collision processor 152 controls the virtual hand model in the virtual environment by using the hand tracking information, generates collision object information when a collision occurs between the virtual hand model and the object, and provides the generated dynamic object to the dynamics processor 154.

The dynamics processing unit 154 generates the virtual hand model and the motion information of the object according to the collision occurrence based on the collision object information and provides it to the physical-based motion visualization unit 156.

Here, the dynamics processing unit 154 selectively operates any one of the constraint based module 154a and the physics based module 154b operated according to the unique characteristics of the object. Module that applies constraint-based grabbing interaction when the object is constrained in a specific axial direction such as, gearstick, door and glove box. Control position and angle.

The physics-based module 154b is a module that applies a physics-based grabbing method when an object has a characteristic capable of freely moving in all directions, such as a mobile phone. A physics-based module 154b includes a virtual hand model and physical properties in which hand tracking information is reflected. Connect between hand models with springs and joints. At this time, the virtual hand model connected to the spring does not perform the collision check with the virtual object according to the motion of the hand when interacting with the virtual object, and the dependent hand connected with the spring is affected by the collision check and the dynamic simulation. Therefore, the virtual hand model operating based on the hand tracking information according to the motion of the tracking hand (actual) when interacting with the virtual object describes the similar motion as much as possible, but is properly projected on the surface of the object when a collision with the virtual object occurs.

In this way, you can manipulate objects in the palm of your hand as if it were real by reflecting various physical elements such as friction, elasticity and gravity.

Through this process, the constraint-based interaction module 154a and the physical-based interaction module 154b generate the virtual hand model and the motion information of the object according to the collision, that is, the constraint-based interaction module 154a is based on the center point of the object. By generating the motion information of the virtual hand model and the object obtained by controlling the position and angle of the virtual hand model to the physical-based motion visualization unit 156, the physical-based interaction module 154b is based on the hand tracking information By generating the motion information of the object and the virtual hand model according to the motion of the virtual hand model to operate to provide to the physical-based motion visualization unit 156.

The physical-based motion visualization unit 156 updates the virtual environment data based on the motion information, generates the hand joint information based on the change of the virtual hand model according to the update, and provides the generated hand joint information to the desensitization rendering unit 158.

 The sensitization rendering unit 158 calculates the sensitization information, for example, the overall position of the hand and the sense of rotation reaction force, based on the hand joint information, and provides the same to the six degree of freedom haptic device 120 connected through the device controller 160.

Accordingly, the 6-degree of freedom haptic control device 120 controls the position of the entire hand based on the adverse reaction information and controls the rotational reaction force provided to the actual hand, thereby preventing the virtual hand model from actually entering the virtual object. can do.

According to a preferred embodiment of the present invention, in providing an IPTV service, a subscriber may provide a function for controlling broadcast content.

The present invention has been limited to the embodiments of the present invention, but it is obvious that the technology of the present invention can be easily modified by those skilled in the art. Such modified embodiments should be included in the technical spirit described in the claims of the present invention.

1 is a diagram illustrating a hand-based grabbing interaction system using a six degree of freedom haptic device according to an embodiment of the present invention.

Figure 2 is a detailed configuration of the position and rotation reaction mechanism of the six degree of freedom haptic device according to the present invention,

3A to 3B are detailed configuration diagrams of a mobile mobile device of a six degree of freedom haptic device according to the present invention.

<Description of the code | symbol about the principal part of drawing>

100: hardware unit 110: hand interface

120: 6 degree of freedom haptic device 130: mounting device

150: software unit 152: device control unit

154: collision processing unit 156: dynamics processing unit

158: physics-based motion visualization unit 160: force rendering unit

170: Haptic Device Driver

Claims (7)

A hardware unit for generating hand tracking information according to a change in the position of a finger and a hand, and for controlling the position and rotational reaction of the hand based on the force information; A collision processor configured to control a virtual hand model in a virtual environment using the hand tracking information, and generate collision object information when a collision occurs between the virtual hand model and an object; A dynamics processor configured to receive collision object information from the collision processor, and to provide motion information of the virtual hand model and the object according to the collision based on the unique characteristics of the object; A physical-based motion visualization unit for updating the virtual environment data based on the motion information and generating hand joint information based on a change of the virtual hand model according to the update; Calculation unit for calculating the force information based on the hand joint information and providing it to the hardware unit to control the position and rotation reaction force of the hand Hand-based grabbing interaction system using a six degree of freedom haptic device comprising a. The method of claim 1, The hardware unit, A hand interface for providing the joint angle information of the finger to the collision processing unit; A six degree of freedom haptic device that provides position information of the hand to the collision processor and controls the position and reaction force of the hand based on the force information. Hand-based grabbing interaction system using a six degree of freedom haptic device comprising a. The method of claim 2, The six degrees of freedom haptic device, Position control 1, 2 and 3 axes for controlling the position of the hand; Wrist rotation first, second and third axis for controlling the position of the hand Hand-based grabbing interaction system using a six degree of freedom haptic device comprising a. The method of claim 3, wherein The hand-based grabbing interaction system using the six degrees of freedom haptic device, The 6 degree of freedom haptic device can be mounted, the mounting device for controlling the movement of the 6 degree of freedom haptic device Hand-based grabbing interaction system using a six degree of freedom haptic device further comprising. The method of claim 1, The dynamics processing unit, A constraint interaction module for controlling the position and angle of the virtual hand model based on the center point of the object preset when the collision occurs when the inherent property of the object is fixed and fixed in a specific axis direction. Hand-based grabbing interaction system using a six degree of freedom haptic device comprising a. The method of claim 1, The dynamics processing unit, When the unique characteristic of the object can be freely moved, a physical base that provides the motion information by controlling the virtual hand model based on the hand tracking information when the collision occurs and reflecting physical properties of the virtual hand model and the object. Grab Interaction Module Hand-based grabbing interaction system using a six degree of freedom haptic device comprising a. The method of claim 1, The collision processing unit may load the virtual graphic model data in the virtual environment into a scene graph structure and generate the collision information using a hierarchical collision search technique for real-time collision. Catching Interaction System.

Images