Facial Rigging For 3D Character

International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 F ACIAL RIGGING F OR 3D CHARACTE R Matahari Bhakti Nendya1, Eko Mulyanto Yuniarno2 and Samuel Gandang Gunanto3 1,2 Department of Electrical Engineering, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia 3 Department of Animation, Institut Seni Indonesia, Yogyakarta, Indonesia ABSTRACT Facial animation is an important aspect to make character more life in virtual world. Early stage to make facial animation is facial rigging. Facial rigging is the process of creating the animation controls for a facial model and the animator’s interface to those controls. Facial feature points which associated with facial skeleton makes process of rigging easy to do. Weighting feature points on the face makes facial skeleton used as the basis of the manufacture of facial control to manual or automatic animation with motion data. KEYWORDS Facial Animation, Rigging, Feature Points, Facial Motion Data 1. INTRODUCTION The main aspect in 3D virtual environment to presents the existence of the characters is facial animation. Facial animation can be used in industrial 3D games, interactive software and 3D animated movies. Because of the complexity of human facial expressions and natural movement, facial animation take a long time to produce even by an expert animator [1]. Today, the development of facial animation is still a lot of research produced. There are main aspect in facial animation research: the process of rigging facial expression and transfers animation between two pieces of 3D virtual character or between people into a 3D character. Facial rigging needs requires a long time manual processing time for an artist. Simple methods like shape-blending needs a model as a reference. Skinning techniques require manual definition of the joints can be affected the movement of face, such as eyebrows forehead, cheek, jaw and mouth. Facial animation using a simulation approach to facial skin and muscles and animator as artist need intervention to setting the facial skin with bone or skull and configuration of connections facial muscles in motion. In this case the production of facial animation for the face can’t be reused directly to other face because of complexity and specialization. 2. F UNDAMENTALS 2.1. 3D Facial Modelling The main problem in facial animation is to modelling 3D geometry of an individual face. Scanner digitizer or stereo disparity can measure three-dimensional coordinate but those processes are DOI : 10.5121/ijcga.2014.4302 21 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 often poorly suited for facial animation. Information about the facial structures is missing; measurement noise produces distracting artifacts, and model vertices are poorly distributed. Also, many measurement methods produce incomplete models, lacking hair, ears, eyes, etc. Therefore, post-processing on the measured data is often necessary 2.2. Facial Rigging Process of creating the animation controls for a facial model and the animator’s interface to those controls. Hanrahan and Sturman develop system which allowed the animator to establish functional relationships between interactive input devices and control parameters [2]. For example, animator might define a functional link between an input dial value and a control parameter. The dial value, RSMILE, might affect the right corner of the mouth by pulling it up and back as the dial value increases. This system provided great flexibility in defining the functional links between the input values and the generated control values. 2.2.1 Articulated Joints One widely used approach is to embed within the face model an articulated joint hierarchy. Each joint corresponds to the articulated connection between two skeletal segments or bones. These joints are manipulated by varying their orientation angles. Influences are established between the surface vertices or control points and one or more of these joints. As the joints are manipulated and their orientation angles adjusted, the associated surface points are rotated along with the joints. The influence between a specific surface point and a given joint is usually a weighting value. Some surface points close to the joint might be completely influenced by it, moving exactly as it moves. Other points might be only partially influenced by a joint, only moving a percentage of its motion. Each surface point is likely to have a different weight or influence from a given joint. Some points on the surface may be influenced by several joints. Establishing the joint structure and determining the surface point weighting values for the joints is a part of the rigging process [3]. 2.2.2 Blend Shapes Blend shapes are a form of surface shape interpolation. Here the surface is sculpted into two or more shapes. One of these is the base shape, while the others are called target shapes. The differences between the base and target shapes are represented as vector sets. Each vector set corresponds to the difference between the base shape and one of the target shapes. Each vector in a blend shape is the difference in location between a point in the base shape and its corresponding point in the target shape. When a blend shape is applied to the base shape, it takes on the target shape for that blend. If the blend shape is only partially applied, the base shape moves partially toward the target shape. This allows interpolation between the base and target shapes. This interpolation is controlled by a blending coefficient. If the coefficient is 0.0, the surface has the base shape. If the coefficient is 1.0, the surface takes on the target shape. For values between 0.0 and 1.0, the surface takes on a shape between the base and target shapes. For a given blend shape, perhaps only a few of the surface points are affected. That is, for a given blend shape, some of the surface points may have zero length vectors. This means that the positions for these points are the same in both the base and target shapes. Also, a given surface 22 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 point may be affected by more than one blend shape. If more than one blend shape is applied to a surface, each surface point will be moved, based on the sum of the point vectors for each blend shape, weighted by their respective coefficients. Figure 1. Blend shapes used to mimic muscle actions: (a) right brow raiser, (b) right lip corner puller, (c) right lip corner depressor, (d) lip pucker, (e) chin raiser. 2.2.3 Clusters A cluster is a group of points that are associated with a coordinate transformation. This transformation might be a scale, a translation, a rotation, or a combination of these. Clusters allow groups of surface points to be scaled, translated, and rotated relative to a defined cluster origin location. The effect of a cluster transformation on a given point in the cluster is determined by a weighting value. Different points in the cluster typically have different weighting values, or weights. The point weights allow the effects of a cluster transformation to vary across the set of cluster points. Typically, the effect of the cluster transformation is greatest near the origin of the cluster and tapers off further away from this origin. The right-most image in Figure 2 shows this tapered weighting for one of the clusters near the end of the nose. In this figure, white corresponds to maximum weight, while black corresponds to zero weight. Grey values correspond to weights between maximum and zero. Figure 2. Use of clusters: (left) rendered face model, (middle) cluster locations, (right) weighting of one cluster near the tip of the nose. 2.2.4 The User Interface Usually, the control values or parameters for the joint angles, blend shape weights, cluster transformations, and functional expressions are set at specific times or key frames of the desired animation. The parameter values at other frame times are derived from these key frame values using some form of linear or spline interpolation. An interactive curve editor is often provided as part of the animation system. The curve editor allows the animator to manipulate the interpolation curves used in transitioning from one key frame to the next. Usually, the curve editing can be applied on each parameter individually. In fact, different parameters may have different sets of key frame times. 23 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 Figure 3. Example of GUI Interface 3. METHODS AND DESIGN Flow of our process can be shown below: Figure 4. Facial rigging for 3D character process 3.1. Facial Feature Points Facial feature points is a main factor to generation of facial animation. Facial feature points usually associated with facial movement system like joints, muscles and jaw. In this research, we used 10 facial feature points with the laying of position shown image below. Figure 5: Facial feature points positition 24 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 A variety of approaches can be taken to determine the location of feature points on the 3D model. 3.2 Mesh Deformation Deformation defined on the facial mesh surface to produces quality animation. It ignores underlying facial anatomy or true muscle structures. Instead, the focus is on creating various facial expressions by the manipulation of the thin shell mesh. This category includes morphing between different models and simulated pseudo muscles in the form of splines [4], wires [5], or free-form deformations [6]. 3.2.1 2D and 3D Morphing Pighin et al. [7] combine 2D morphing with 3D transformations of a geometric model. They animate key facial expressions with 3D geometric interpolation while image morphing is performed between corresponding texture maps. This approach achieves viewpoint independent realism; however, animations are still limited to interpolations between predefined key facial expressions. The 2D and 3D morphing methods can produce quality facial expressions, but they share similar limitations with the interpolation approaches. Selecting corresponding points in target images is manually intensive, dependent on viewpoint, and not generalizable to different faces. Also, the animation viewpoint is constrained to approximately that of the target images. 3.2.2 Free-Form Deform Free-form deformation (FFD) deforms volumetric objects by manipulating control points arranged in a three-dimensional cubic lattice [8]. Conceptually, a flexible object is embedded in an imaginary, clear, and flexible control box containing a 3D grid of control points. As the control box is squashed, bent, or twisted into arbitrary shapes, the embedded object deforms accordingly. The basis for the control points is a trivariate tensor product Bernstein polynomial. FFDs can deform many types of surface primitives, including polygons; quadric, parametric, and implicit surfaces; and solid models. Figure 6. Free-form deformation. The controlling box and embedded object are shown. When the controlling box is deformed by manipulating control points, so is the embedded object. 4. EXPERIMENT RESULT AND ANALYSIS Our experiment using fan art of Gaido character system from Disney’s cars movie. Model by Masoud zamani Copyright Cars 2 (Pixar and disney), using cycles for render and released under a Attribution-NonCommercial-Share Alike 3.0 Unported License http:// creativecommons.org/ licenses/ by-nc-sa/3.0/. Character model can downloaded from http://www.blendswap.com/ blends/view/ 52200. 25 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 Figure 7. Guido character 4.1 Preparation of Facial Skeleton Preparation of facial skeleton refers to facial feature point model on figure 5. Facial feature points can be associated with the model of facial bone. (a) Facial bone (b) Attachment to mesh Figure 8. Formation of the facial skeleton 3D character models. 4.2 Facial Feature Points Weighting Weighting of facial feature points on this characters done automatically by using automatic feature weight. (a) Left mouth (b) Right mouth Figure 9. Weighting facial feature points on a 3-dimensional model of character Weighting adjustments in facial feature points is done manually to get the suitable model of facial animation. 26 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 4.3 Facial Animation Control Full set of facial animation facial deformation based on the position of the face frame weighs facial feature points associated with the shape of the face mesh. Generally facial animation control divided into: 4.3.1 Manual Facial Animation Controlling Manual control of facial animation in 3D character is set manually by animators. Generating and animating can be done by changing shape of the face mesh with the associated with the feature points. Result of face mesh deformation done manually by changing shape of facial mesh are shown as: (a) Left eyebrow (b) Right eyebrow (c) Left mouth (d) Right Mouth Figure 10. The results of facial animation manually change 4.3.2 Facial Animation Controlling Using Motion Data Motion data is a specific data format which result in the movement of facial animation in 3D character. Motion data usually in cartesian coordinate x; y; and z. Unity 3D game engine use to animated character from motion data. The animation process refers to displacement coordinates of facial feature points. This process occurs continuously until the data was renewed motion. To make facial animation control automatically, we use facial motion data shown below. 27 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 Figure 11. Motion data And the results of the control of facial animation using motion data is: (a) Left eyebrow (c) Left mouth (b) Right eyebrow (d) Right Mouth Figure 12. Results of changes in facial animation based on motion of data 5. C ONCLUSIONS AND FEATURE WORK Facial rigging for 3D character are easy to use. The process is linier and need no more complex computation. Facial skeleton which associate with facial feature points and assigning weights to the point of facial features by using the automatic weight control allows the formation of a good facial animation control by manual or based on motion data. Facial feature point weighting adjustment absolutely necessary to give effect to the movement of more expressive facial animation based on a mesh shape face model used. Further research on facial feature point weighting to adjust the shape of the face mesh. REFERENCES [1] Parke, F, (1972) “Computer Generated Animation of Face”. Proceedings of the ACM annual conference, Vol. 1, pp451-457. 28 International Journal of Computer Graphics & Animation (IJCGA) Vol.4, No.3, July 2014 [2] [3] [4] [5] [6] [7] [8] [9] P. Hanrahan. & D. Sturman, (1985) “Interactive Animation of Parametric Models”. The Visual Computer 1:4. J. Osipa, (2003) Stop Staring: Facial Modeling and Animation Done Right, Sybex. M.L. Viad & H. Yahia, (1992) “Facial Animation With Wrinkles”. In Proceedings of the Third Eurographics Workshop on Animation and Simulation. K. Singh K, & E. Fiume, (1998) “Wires: A geometric deformation technique”. In SIGGRAPH Proceedings, pp405-414. P. Kalra, A. Mangili, N.M. Thalmann, & D. Thalmann, (1992) “Simulation of facial muscle actions based on rational free from deformations”. In Eurographics, Vol. 11, pp59-69. F. Pighin, J. Auslander, D. Lischinski, D.H. Salesin, & R. Szeliski, (1997) “Realistic facial animation using image-based 3D morphing”, Technical report UW-CSE-97-01-03. T.W. Sederberg, & S.R. Parry, (1996) Free-form deformation of solid geometry models. In Computer Graphics, SIGGRAPH, Vol. 20, pp151-160. Parke, F., & Waters, K, (2008) Computer Facial Animation 2nd Edition. AK Peters, Massachusetts Authors Matahari Bhakti Nendya graduated from Department of Informatics Engineering Duta Wacana Christian University Yogyakarta for his bachelor in 2012. Since 2013, he has been studying at Graduate School of Electrical Engineering Institut Teknologi Sepuluh Nopember Surabaya Indonesia as postgraduate student in game technology. He works as an assistant lecturer of Department of Animation, Institut Seni Indonesia Yogyakarta. His research interest includes multimedia processing and game technology. Eko Mulyanto Yuniarno graduated from Electrical Engineering Institut Teknologi Sepuluh Nopember (ITS) Surabaya, Indonesia, for his bachelor in 1995 and received MT and Dr degree in 2005 and 2013 respectively from Graduate School of Electrical Engineering ITS Surabaya. He is currently lecturer of Electrical Engineering Dept ITS. His research interest include computer vision, image processing and multimedia processing. Samuel Gandang Gunanto graduated from Department of Informatics Engineering, Duta Wacana Christian University Yogyakarta and Departmen of Physic Gadjah Mada University Yogyakarta Indonesia for his bachelor in 2004 and received MT degree in 2010 from Graduate School of Electrical Engineering Institut Teknologi Sepuluh Nopember (ITS) Surabaya. Since 2011, he has been studying at Graduate School of Electrical Engineering ITS Surabaya Indonesia as a doctoral student. He works as a lecturer of Department Animation, Institut Seni Indonesia Yogyakarta. His research interest include game technology, multimedia processing and computer vision. 29