skip to main content
research-article

Guided Learning of Control Graphs for Physics-Based Characters

Published: 18 May 2016 Publication History

Abstract

The difficulty of developing control strategies has been a primary bottleneck in the adoption of physics-based simulations of human motion. We present a method for learning robust feedback strategies around given motion capture clips as well as the transition paths between clips. The output is a control graph that supports real-time physics-based simulation of multiple characters, each capable of a diverse range of robust movement skills, such as walking, running, sharp turns, cartwheels, spin-kicks, and flips. The control fragments that compose the control graph are developed using guided learning. This leverages the results of open-loop sampling-based reconstruction in order to produce state-action pairs that are then transformed into a linear feedback policy for each control fragment using linear regression. Our synthesis framework allows for the development of robust controllers with a minimal amount of prior knowledge.

Supplementary Material

liu (liu.zip)
Supplemental movie, appendix, image and software files for, Guided Learning of Control Graphs for Physics-Based Characters

References

[1]
Mazen Al Borno, Martin de Lasa, and Aaron Hertzmann. 2013. Trajectory optimization for full-body movements with complex contacts. TVCG 19, 8 (2013), 1405--1414.
[2]
Mazen Al Borno, Eugene Fiume, Aaron Hertzmann, and Martin de Lasa. 2014. Feedback control for rotational movements in feature space. Comput. Graph. Forum 33, 2 (2014), 225--233.
[3]
Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust task-based control policies for physics-based characters. ACM Trans. Graph. 28, 5, Article 170 (Dec. 2009), 9 pages. 10.1145/1618452.1618516
[4]
Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized biped walking control. ACM Trans. Graph. 29, 4, Article 130 (July 2010), 9 pages.
[5]
Marco Da Silva, Yeuhi Abe, and J. Popović. 2008. Simulation of human motion data using short-horizon model-predictive control. In Comput. Graph. Forum, Vol. 27. Wiley Online Library, 371--380.
[6]
Marco da Silva, Frédo Durand, and Jovan Popović. 2009. Linear Bellman combination for control of character animation. ACM Trans. Graph. 28, 3, Article 82 (July 2009), 10 pages. 1531326.1531388
[7]
Martin de Lasa, Igor Mordatch, and Aaron Hertzmann. 2010. Feature-based locomotion controllers. ACM Trans. Graph. 29, 4, Article 131 (July 2010), 10 pages. 10.1145/1778765.1781157
[8]
Kai Ding, Libin Liu, Michiel van de Panne, and KangKang Yin. 2015. Learning reduced-order feedback policies for motion skills. In Proceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’15). ACM, New York, NY, 83--92. 10.1145/2786784.2786802
[9]
Arnaud Doucet and Adam M. Johansen. 2011. A tutorial on particle filtering and smoothing: Fifteen years later. In Handbook of Nonlinear Filtering. Oxford, UK: Oxford University Press.
[10]
Petros Faloutsos, Michiel van de Panne, and Demetri Terzopoulos. 2001. Composable controllers for physics-based character animation. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). ACM, New York, NY, USA, 251--260.
[11]
Thomas Geijtenbeek and Nicolas Pronost. 2012. Interactive character animation using simulated physics: A state-of-the-art review. In Comput. Graph. Forum, Vol. 31. Wiley Online Library, 2492--2515.
[12]
Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Trans. Graph. 32, 6 (2013), 206.
[13]
Sehoon Ha, Yuting Ye, and C. Karen Liu. 2012. Falling and landing motion control for character animation. ACM Trans. Graph. 31, 6, Article 155 (Nov. 2012), 9 pages.
[14]
Hirotaka Hachiya, Jan Peters, and Masashi Sugiyama. 2009. Efficient sample reuse in EM-based policy search. In Machine Learning and Knowledge Discovery in Databases (Lecture Notes in Computer Science), Vol. 5781. Springer, Berlin, 469--484.
[15]
Jessica K. Hodgins, Wayne L. Wooten, David C. Brogan, and James F. O'Brien. 1995. Animating human athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95), Susan G. Mair and Robert Cook (Eds.). ACM, New York, NY, USA, 71--78.
[16]
Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion graphs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’02). ACM, New York, NY, 473--482.
[17]
Taesoo Kwon and Jessica Hodgins. 2010. Control systems for human running using an inverted pendulum model and a reference motion capture sequence. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'10). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 129--138.
[18]
Jehee Lee and Kang Hoon Lee. 2006. Precomputing avatar behavior from human motion data. Graph. Models 68, 2 (2006), 158--174.
[19]
Yoonsang Lee, Sungeun Kim, and Jehee Lee. 2010a. Data-driven biped control. ACM Trans. Graph. 29, 4, Article 129 (July 2010), 8 pages.
[20]
Yongjoon Lee, Kevin Wampler, Gilbert Bernstein, Jovan Popović, and Zoran Popović. 2010b. Motion fields for interactive character locomotion. ACM Trans. Graph. 29, 6, Article 138 (Dec. 2010), 8 pages. org/10.1145/1882261.1866160
[21]
Sergey Levine and Vladlen Koltun. 2013. Guided policy search. In Proceedings of the 30th International Conference on Machine Learning (ICML’13).
[22]
Sergey Levine and Vladlen Koltun. 2014. Learning complex neural network policies with trajectory optimization. In Proceedings of the 31st International Conference on Machine Learning (ICML’14).
[23]
Libin Liu, KangKang Yin, and Baining Guo. 2015. Improving sampling-based motion control. Comput. Graph. Forum 34, 2 (May 2015), 415--423.
[24]
Libin Liu, KangKang Yin, Michiel van de Panne, and Baining Guo. 2012. Terrain runner: Control, parameterization, composition, and planning for highly dynamic motions. ACM Trans. Graph. 31, 6 (2012), Article 154.
[25]
Libin Liu, KangKang Yin, Michiel van de Panne, Tianjia Shao, and Weiwei Xu. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph. 29, 4 (2010), Article 128.
[26]
Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and control of skeleton-driven soft body characters. ACM Trans. Graph. 32, 6 (2013), Article 215.
[27]
Adriano Macchietto, Victor Zordan, and Christian R. Shelton. 2009. Momentum control for balance. ACM Trans. Graph. 28, 3, Article 80 (July 2009), 8 pages.
[28]
Igor Mordatch, Martin de Lasa, and Aaron Hertzmann. 2010. Robust physics-based locomotion using low-dimensional planning. ACM Trans. Graph. 29, 4, Article 71 (July 2010), 8 pages.
[29]
Igor Mordatch and Emo Todorov. 2014. Combining the benefits of function approximation and trajectory optimization. In Proceedings of Robotics: Science and Systems. Berkeley, CA.
[30]
Igor Mordatch, Emanuel Todorov, and Zoran Popović. 2012. Discovery of complex behaviors through contact-invariant optimization. ACM Trans. Graph. 31, 4, Article 43 (July 2012), 8 pages.
[31]
Uldarico Muico, Yongjoon Lee, Jovan Popović, and Zoran Popović. 2009. Contact-aware nonlinear control of dynamic characters. ACM Trans. Graph. 28, 3, Article 81 (July 2009), 9 pages.
[32]
Uldarico Muico, Jovan Popović, and Zoran Popović. 2011. Composite control of physically simulated characters. ACM Trans. Graph. 30, 3, Article 16 (May 2011), 11 pages. 1966394.1966395
[33]
Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2015. Dynamic terrain traversal skills using reinforcement learning. ACM Trans. Graph. 34, 4, Article 80 (July 2015), 11 pages. 2766910
[34]
Jan Peters and Stefan Schaal. 2007. Reinforcement learning by reward-weighted regression for operational space control. In Proceedings of the 24th International Conference on Machine Learning (ICML’07). ACM, New York, NY, 745--750. 10.1145/1273496.1273590
[35]
Jan Peters and Stefan Schaal. 2008. Reinforcement learning of motor skills with policy gradients. Neural Netw. 21, 4 (May 2008), 682--697. 10.1016/j.neunet.2008.02.003
[36]
Zoran Popović and Andrew Witkin. 1999. Physically based motion transformation. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., 11--20.
[37]
Marc H. Raibert and Jessica K. Hodgins. 1991. Animation of dynamic legged locomotion. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 349--358.
[38]
Stephane Ross, Geoffrey Gordon, and J. Andrew (Drew) Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS’11).
[39]
Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26, 3 (2007), Article 107.
[40]
Adnan Sulejmanpašić and Jovan Popović. 2005. Adaptation of performed ballistic motion. ACM Trans. Graph.24, 1 (2005), 165--179.
[41]
Jie Tan, Yuting Gu, C. Karen Liu, and Greg Turk. 2014. Learning bicycle stunts. ACM Trans. Graph. 33, 4, Article 50 (July 2014), 12 pages.
[42]
Jie Tan, C. Karen Liu, and Greg Turk. 2011. Stable proportional-derivative controllers. IEEE Comput. Graph. Appl. 31, 4 (2011), 34--44.
[43]
Yuval Tassa, Tom Erez, and Emanuel Todorov. 2012. Synthesis and stabilization of complex behaviors through online trajectory optimization. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’12). IEEE, 4906--4913.
[44]
Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal character animation with continuous control. ACM Trans. Graph. 26, 3 (July 2007), Article 7.
[45]
Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Trans. Graph. 28, 3 (2009), Article 60.
[46]
Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2009. Optimizing walking controllers. ACM Trans. Graph. 28, 5 (2009), Article 168.
[47]
Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2010. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29, 4, Article 73 (July 2010), 8 pages. 10.1145/1778765.1778810
[48]
Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31, 4 (2012), 25.
[49]
Yuting Ye and C. Karen Liu. 2010. Optimal feedback control for character animation using an abstract model. ACM Trans. Graph. 29, 4, Article 74 (July 2010), 9 pages.
[50]
KangKang Yin, Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2008. Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 3 (2008), Article 81.
[51]
KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: Simple biped locomotion control. ACM Trans. Graph. 26, 3 (2007), Article 105.
[52]
Victor Zordan, David Brown, Adriano Macchietto, and KangKang Yin. 2014. Control of rotational dynamics for ground and aerial behavior. IEEE Trans. Visual. Comput. Graphics 20, 10 (Oct 2014), 1356--1366. 10.1109/TVCG.2014.2330610
[53]
Victor Brian Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic response for motion capture animation. ACM Trans. Graph. 24, 3 (July 2005), 697--701.

Cited By

View all

Index Terms

  1. Guided Learning of Control Graphs for Physics-Based Characters

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 35, Issue 3
    June 2016
    128 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/2903775
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 May 2016
    Accepted: 01 February 2016
    Revised: 01 December 2015
    Received: 01 September 2015
    Published in TOG Volume 35, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Motion control
    2. control graphs
    3. guided policy search
    4. human simulation

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)63
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 01 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media