research-article

Optimizing UI layouts for deformable face-rig manipulation

Authors:

Karan SinghAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 40, Issue 4

Article No.: 172, Pages 1 - 12

https://doi.org/10.1145/3450626.3459842

Published: 19 July 2021 Publication History

Abstract

Complex deformable face-rigs have many independent parameters that control the shape of the object. A human face has upwards of 50 parameters (FACS Action Units), making conventional UI controls hard to find and operate. Animators address this problem by tediously hand-crafting in-situ layouts of UI controls that serve as visual deformation proxies, and facilitate rapid shape exploration. We propose the automatic creation of such in-situ UI control layouts. We distill the design choices made by animators into mathematical objectives that we optimize as the solution to an integer quadratic programming problem. Our evaluation is three-fold: we show the impact of our design principles on the resulting layouts; we show automated UI layouts for complex and diverse face rigs, comparable to animator handcrafted layouts; and we conduct a user study showing our UI layout to be an effective approach to face-rig manipulation, preferable to a baseline slider interface.

Supplementary Material

VTT File (3450626.3459842.vtt)

Download
25.01 KB

MP4 File (a172-kim.mp4)

Download
102.78 MB

MP4 File (3450626.3459842.mp4)

Presentation.

Download
534.02 MB

References

[1]

Rinat Abdrashitov, Fanny Chevalier, and Karan Singh. 2020. Interactive Exploration and Refinement of Facial Expression Using Manifold Learning. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST '20). Association for Computing Machinery, New York, NY, USA, 778--790.

Digital Library

[2]

Maneesh Agrawala, Wilmot Li, and Floraine Berthouzoz. 2011. Design Principles for Visual Communication. Commun. ACM 54, 4 (April 2011), 60--69.

Digital Library

[3]

Eric Allen and Kelly L. Murdock. 2008. Body Language: Advanced 3D Character Rigging (pap/cdr ed.). SYBEX Inc., USA.

[4]

Norman I Badler, Brian A Barsky, and David Zeltzer. 1990. Making Them Move: Mechanics, Control & Animation of Articulated Figures. Routledge.

[5]

Stephen W. Bailey, Dalton Omens, Paul Dilorenzo, and James F. O'Brien. 2020. Fast and Deep Facial Deformations. ACM Trans. Graph. 39, 4, Article 94 (July 2020), 15 pages.

Digital Library

[6]

Eric Allan Bier. 1987. Skitters and jacks: interactive 3D positioning tools. In Proceedings of the 1986 workshop on Interactive 3D graphics. ACM, 183--196.

Digital Library

[7]

Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. 2010. Polygon mesh processing. CRC press.

[8]

Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 103--112.

Digital Library

[9]

Ozan Cetinaslan and Verónica Orvalho. 2018. Direct Manipulation of Blendshapes Using a Sketch-Based Interface. In Proceedings of the 23rd International ACM Conference on 3D Web Technology (Poznań, Poland) (Web3D '18). Association for Computing Machinery, New York, NY, USA, Article 14, 10 pages.

Digital Library

[10]

Pif Edwards, Chris Landreth, Mateusz Popławski, Robert Malinowski, Sarah Watling, Eugene Fiume, and Karan Singh. 2020. JALI-Driven Expressive Facial Animation and Multilingual Speech in Cyberpunk 2077. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Talks (Virtual Event, USA) (SIGGRAPH '20). Association for Computing Machinery, New York, NY, USA, Article 60, 2 pages.

Digital Library

[11]

Rosenberg Ekman. 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.

[12]

Krzysztof Z Gajos, Mary Czerwinski, Desney S Tan, and Daniel S Weld. 2006. Exploring the design space for adaptive graphical user interfaces. In Proceedings of the working conference on Advanced visual interfaces. 201--208.

Digital Library

[13]

Sarah Gibson, Paul Beardsley, Wheeler Ruml, Thomas Kang, Brian Mirtich, Joshua Seims, William Freeman, Jessica Hodgins, Hanspeter Pfister, Joe Marks, et al. 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. (1997).

[14]

Michael Gleicher. 1992. Integrating constraints and direct manipulation. In Symposium on Interactive 3 D Graphics: Proceedings of the 1992 symposium on Interactive 3 D graphics, Vol. 1992. 171--174.

Digital Library

[15]

Martin Guay, Marie-Paule Cani, and Rémi Ronfard. 2013. The line of action: an intuitive interface for expressive character posing. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1--8.

Digital Library

[16]

Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 65--74.

Digital Library

[17]

Paul G Kry, Doug L James, and Dinesh K Pai. 2002. Eigenskin: real time large deformation character skinning in hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. 153--159.

Digital Library

[18]

Manfred Lau, Jinxiang Chai, Ying-Qing Xu, and Heung-Yeung Shum. 2009. Face poser: Interactive modeling of 3D facial expressions using facial priors. ACM Transactions on Graphics (TOG) 29, 1 (2009), 3.

Digital Library

[19]

Binh H. Le, Mingyang Zhu, and Zhigang Deng. 2013. Marker Optimization for Facial Motion Acquisition and Deformation. IEEE Transactions on Visualization and Computer Graphics 19, 11 (Nov. 2013), 1859--1871.

Digital Library

[20]

John P Lewis and Ken-ichi Anjyo. 2010. Direct manipulation blendshapes. IEEE Computer Graphics and Applications 30, 4 (2010), 42--50.

Digital Library

[21]

Hao Li, Jihun Yu, Yuting Ye, and Chris Bregler. 2013. Realtime Facial Animation with On-the-Fly Correctives. ACM Trans. Graph. 32, 4, Article 42 (July 2013), 10 pages.

Digital Library

[22]

José Carlos Miranda, Xenxo Alvarez, João Orvalho, Diego Gutierrez, A Augusto Sousa, and Verónica Orvalho. 2011. Sketch express: facial expressions made easy. In Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling. ACM, 87--94.

Digital Library

[23]

Niloy J. Mitra, Mark Pauly, Michael Wand, and Duygu Ceylan. 2012. Symmetry in 3D Geometry: Extraction and Applications. In EUROGRAPHICS State-of-the-art Report.

Digital Library

[24]

Fionn Murtagh and Pedro Contreras. 2012. Algorithms for hierarchical clustering: an overview. WIREs Data Mining and Knowledge Discovery 2, 1 (2012), 86--97. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/widm.53

[25]

Victor Navone. 2020. Facial Animation for Feature Animated Films: Animating stylized facial expressions. https://www.thegnomonworkshop.com/tutorials/facial-animation-for-feature-animated-films (2020).

[26]

Jason Osipa. 2010. Stop Staring: Facial Modeling and Animation Done Right (3rd ed.). SYBEX Inc., USA.

Digital Library

[27]

Rick Parent. 2012. Computer animation: algorithms and techniques. Newnes.

[28]

Ryan Schmidt, Karan Singh, and Ravin Balakrishnan. 2008. Sketching and composing widgets for 3d manipulation. In Computer graphics forum, Vol. 27. Wiley Online Library, 301--310.

[29]

Thomas W Sederberg and Scott R Parry. 1986. Free-form deformation of solid geometric models. In Proceedings of the 13th annual conference on Computer graphics and interactive techniques. 151--160.

Digital Library

[30]

Mike Seymour. 2016. Put your (digital) game face on. (2016). https://www.fxguide.com/fxfeatured/put-your-digital-game-face-on/

[31]

Mike Seymour. 2018. Making Thanos Face the Avengers. (2018). https://www.fxguide.com/fxfeatured/making-thanos-face-the-avengers/

[32]

Mike Seymour. 2019. Bebyface in Bebylon. (2019). https://www.fxguide.com/fxfeatured/bebyface-in-bebylon/

[33]

Karan Singh and Eugene Fiume. 1998. Wires: a geometric deformation technique. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 405--414.

Digital Library

[34]

Marc P Stevens, Robert C Zeleznik, and John F Hughes. 1994. An architecture for an extensible 3D interface toolkit. In Proceedings of the 7th annual ACM symposium on User interface software and technology. 59--67.

Digital Library

[35]

Tanasai Sucontphunt, Zhenyao Mo, Ulrich Neumann, and Zhigang Deng. 2008. Interactive 3D facial expression posing through 2D portrait manipulation. In Proceedings of graphics interface 2008. Canadian Information Processing Society, 177--184.

[36]

Jerry O Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory modeling with collaborative design spaces. ACM Transactions on Graphics-TOG 28, 5 (2009), 167.

Digital Library

[37]

J. Rafael Tena, Fernando De la Torre, and Iain Matthews. 2011. Interactive Region-Based Linear 3D Face Models. ACM Trans. Graph. 30, 4, Article 76 (July 2011), 10 pages.

Digital Library

[38]

Ayush Tewari, Michael Zollöfer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Theobalt Christian. 2017. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In The IEEE International Conference on Computer Vision (ICCV).

[39]

Thibaut Weise, Sofien Bouaziz, Hao Li, and Mark Pauly. 2011. Realtime performance-based facial animation. In ACM transactions on graphics (TOG), Vol. 30. ACM.

[40]

Lance Williams. 1990. Performance-driven facial animation. In ACM SIGGRAPH Computer Graphics, Vol. 24. ACM, 235--242.

Digital Library

[41]

Eduard Zell, JP Lewis, Junyong Noh, Mario Botsch, et al. 2017. Facial retargeting with automatic range of motion alignment. ACM Transactions on Graphics (TOG) 36, 4 (2017), 154.

Cited By

Leake MDaly R(2024)ScrapMap: Interactive Color Layout for Scrap QuiltingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676404(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676404
Kim HShakeri MShin JWoo W(2024)Space-adaptive Artwork Placement Based on Content Similarities for Curating Thematic Spaces in a Virtual MuseumJournal on Computing and Cultural Heritage 10.1145/363113417:1(1-21)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3631134
Qiu QWang XOtani MEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Multimodal Color Recommendation in Vector Graphic DocumentsProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612139(4003-4011)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612139
Show More Cited By

Index Terms

Optimizing UI layouts for deformable face-rig manipulation
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Accurate face rig approximation with deep differential subspace reconstruction

To be suitable for film-quality animation, rigs for character deformation must fulfill a broad set of requirements. They must be able to create highly stylized deformation, allow a wide variety of controls to permit artistic freedom, and accurately ...
A demo of a facial UI design approach for digital artists
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User Interfaces

In the character animation industry, animators use facial UI's to animate a character's face. A facial UI provides widgets and handles that the animator interacts with to control the character's facial regions. This paper presents a facial UI design ...
Modeling Expressive Wrinkles of Face For Animation
ICIG '07: Proceedings of the Fourth International Conference on Image and Graphics

Vivid facial expressions contribute greatly to the visual realism of 3D face models. However, it is often tedious and expensive to model facial details which are authentic and real-time. In this paper, a simple and functional approach for modeling ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 40, Issue 4

August 2021

2170 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3450626

Editor:
Sylvain Paris
Adobe Inc.

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2021

Published in TOG Volume 40, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)7

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Leake MDaly R(2024)ScrapMap: Interactive Color Layout for Scrap QuiltingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676404(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676404
Kim HShakeri MShin JWoo W(2024)Space-adaptive Artwork Placement Based on Content Similarities for Curating Thematic Spaces in a Virtual MuseumJournal on Computing and Cultural Heritage 10.1145/363113417:1(1-21)Online publication date: 13-Jan-2024
https://dl.acm.org/doi/10.1145/3631134
Qiu QWang XOtani MEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Multimodal Color Recommendation in Vector Graphic DocumentsProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612139(4003-4011)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612139
Hou JZhao ZWeng D(2023)UI Binding Transfer for Bone-driven Facial Rigs2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW58643.2023.00172(659-660)Online publication date: Mar-2023
https://doi.org/10.1109/VRW58643.2023.00172
Yan SXu SYang WZhang S(2023)Image recoloring based on fast and flexible palette extractionMultimedia Tools and Applications10.1007/s11042-023-15114-582:30(47793-47810)Online publication date: 12-May-2023
https://dl.acm.org/doi/10.1007/s11042-023-15114-5
Ma GYang J(2022)Analysis of the Application Prospect of Computer Image Processing Technology in UI DesignAdvances in Multimedia10.1155/2022/59183992022Online publication date: 24-Jun-2022
https://dl.acm.org/doi/10.1155/2022/5918399
Zell EMcDonnell R(2022)Compact Facial Landmark Layouts for Performance CaptureComputer Graphics Forum10.1111/cgf.1446341:2(121-133)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14463

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents