research-article

Synthesis of detailed hand manipulations using contact sampling

Authors:

C. Karen LiuAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 4

Article No.: 41, Pages 1 - 10

https://doi.org/10.1145/2185520.2185537

Published: 01 July 2012 Publication History

Abstract

Capturing human activities that involve both gross full-body motion and detailed hand manipulation of objects is challenging for standard motion capture systems. We introduce a new method for creating natural scenes with such human activities. The input to our method includes motions of the full-body and the objects acquired simultaneously by a standard motion capture system. Our method then automatically synthesizes detailed and physically plausible hand manipulation that can seamlessly integrate with the input motions. Instead of producing one "optimal" solution, our method presents a set of motions that exploit a wide variety of manipulation strategies. We propose a randomized sampling algorithm to search for as many as possible visually diverse solutions within the computational time budget. Our results highlight complex strategies human hands employ effortlessly and unconsciously, such as static, sliding, rolling, as well as finger gaits with discrete relocation of contact points.

Supplementary Material

JPG File (a41-ye.jpg)

Download
8.51 KB

AVI File (a41-ye.avi)

Download
100.39 MB

References

[1]

Albrecht, I., Haber, J., and Seidel, H.-P. 2003. Construction and animation of anatomically based human hand models. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 98--109.

Digital Library

[2]

Andersen, E. D., Jensen, B., Jensen, J., and Sandvik, R. 2009. Mosek version 6. Tech. Rep. TR-2009-3, Ulf Worsøe, October.

[3]

Aydin, Y., and Nakajima, M. 1999. Database guided computer animation of human grasping using forward and inverse kinematics. Computers and Graphics 23, 1, 145--154.

[4]

Cai, C., and Roth, B. 1987. On the spatial motion of a rigid body with point contact. In Proc. IEEE Int. Con. Robotics and Automation, 686--695.

[5]

Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. In SISGGRAPH, 219--228.

Digital Library

[6]

Cherif, M., and Gupta, K. K. 1999. Planning quasi-static fingertip manipulations for reconfiguring objects. IEEE Trans. on Robotics and Automation 15, 5, 837--848.

[7]

Choi, M. G., Lee, J., and Shin, S. Y. 2003. Planning biped locomotion using motion capture data and probabilistic roadmaps. ACM Transactions on Graphics 22, 182--203.

Digital Library

[8]

Ciocarlie, M. T., and Allen, P. K. 2008. On-line interactive dexterous grasping. Eurohaptics (June).

Digital Library

[9]

Ciocarlie, M. T., Lackner, C., and Allen, P. K. 2007. Soft finger model with adaptive contact geometry for grasping and manipulation tasks. IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (March).

Digital Library

[10]

Cole, A., Hsu, P., and Sastry, S. 1992. Dynamic control of sliding by robot hands for regrasping. IEEE Trans. on Robotics and Automation 8, 1, 42--52.

[11]

Coumans, E., 2005. Bullet physics engine. http://bulletphysics.org.

[12]

Cyberglove Systems, http://www.cyberglovesystems.com/.

[13]

Deshpande, A., Ko, J., Fox, D., and Matsuoka, Y. 2009. Anatomically correct testbed hand control: Muscle and joint control strategies. In Robotics and Automation, 2009. ICRA '09. IEEE International Conference on, 4416--4422.

Digital Library

[14]

ElKoura, G., and Singh, K. 2003. Handrix: Animating the human hand. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 110--119.

Digital Library

[15]

Gill, P., Saunders, M., and Murray, W. 1996. Snopt: An sqp algorithm for large-scale constrained optimization. Tech. Rep. NA 96-2, University of California, San Diego.

[16]

Gleicher, M. 1998. Retargeting motion to new characters. In SIGGRAPH, 33--42.

Digital Library

[17]

Hamer, H., Gall, J., Urtasun, R., and Gool, L. V. 2011. Data-driven animation of hand-object interactions. In IEEE Conference on Automatic Face and Gesture Recognition, 360--367.

[18]

Han, L., and Trinkle, J. 1998. Dextrous manipulation by rolling and finger gaiting. In ICRA, IEEE, 730--735.

[19]

Ho, E. S., Komura, T., and Tai, C.-L. 2010. Spatial relationship preserving character motion adaptation. ACM Trans. Graph 29, 3.

Digital Library

[20]

Hong, J., Lafferriere, G., Mishra, B., and Tang, X. 1990. Fine manipulation with multifinger hand. In ICRA, IEEE, 1568--1573.

[21]

Huang, Z., Boulic, R., and Thalmann, D. 1995. A multi-sensor approach for grasping and 3-D interaction. In Computer Graphics International '95.

Digital Library

[22]

Jain, S., and Liu, C. K. 2009. Interactive synthesis of human-object interaction. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 173--182.

Digital Library

[23]

Jain, S., and Liu, C. K. 2011. Controlling physics-based characters using soft contacts. ACM Trans. Graph. (SIGGRAPH Asia) 30 (Dec.), 163:1--163:10.

Digital Library

[24]

Joerg, S., Hodgins, J., and Sullivan, C. 2010. The perception of finger motions. Applied Perception in Graphics and Visualization (APGV).

Digital Library

[25]

Kavraki, L., Svestka, P., Latombe, J.-C., and Overmars, M. H. 1996. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. on Robotics and Automation 12, 4, 566--580.

[26]

Kim, J., Cordier, F., and Magnenat-Thalmann, N. 2000. Neural network-based violinistâs hand animation. In Conference on Computer Graphics International, 37--44.

Digital Library

[27]

Koga, Y., Kondo, K., Kuffner, J., and Latombe, J.-C. 1994. Planning motions with intentions. In SIGGRAPH, 395--408.

Digital Library

[28]

Kry, P. G., and Pai, D. K. 2006. Interaction capture and synthesis. ACM Trans. on Graphics 25, 3 (Aug.), 872--880.

Digital Library

[29]

Lavalle, S. M., and Kuffner, J. J. 2000. Rapidly-exploring random trees: Progress and prospects.

[30]

Liu, L., Yin, K., van de Panne, M., Shao, T., and Xu, W. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph.(SIGGRAPH) 29, 4, 1--10.

Digital Library

[31]

Liu, C. K. 2009. Dextrous manipulation from a single grasping pose. ACM Transactions on Graphics (SIGGRAPH) 28, 3 (Aug.).

Digital Library

[32]

Majkowska, A., Zordan, V. B., and Faloutsos, P. 2006. Automatic splicing for hand and body animations. In Eurographics/SIGGRAPH Symposium on Computer Animation.

Digital Library

[33]

Miller, A., and Allen, P. 1999. Examples of 3d grasp quality computations. In Robotics and Automation, 1999. Proceedings. 1999 IEEE International Conference on, vol. 2, 1240--1246 vol.2.

[34]

Ngo, J. T., and Marks, J. 1993. Spacetime constraints revisited. In SIGGRAPH, vol. 27, 343--350.

Digital Library

[35]

Pollard, N. S., and Hodgins, J. K. 2002. Generalizing demonstrated manipualtion tasks. In Workshop on the Algorithmic Foundations of Robotics (WAFR '02).

[36]

Pollard, N. S., and Wolf, A. 2004. Grasp synthesis from example: tuning the example to a task or object. In Workshop on Multi-point Interaction in Robotics and Virtual Reality, IEEE International Conference on Robotics and Automation, 77--90.

[37]

Pollard, N. S., and Zordan, V. B. 2005. Physically based grasping control from example. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 311--318.

Digital Library

[38]

Pollard, N. S. 2004. Closure and quality equivalence for efficient synthesis of grasps from examples. International Journal of Robotics Research 23, 6 (June), 595--614.

[39]

Sims, K. 1994. Evolving virtual creatures. In SIGGRAPH.

Digital Library

[40]

Sok, K. W., Kim, M., and Lee, J. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. (SIGGRAPH) 26, 3, 107.

Digital Library

[41]

Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. on Graphics 27, 3 (Aug.).

Digital Library

[42]

Tournassoud, P., Lozano-Perez, T., and Mazer, E. 1987. Regrasping. In Proc. IEEE Int. Con. Robotics and Automation, 1924--1928.

[43]

Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: An anatomically accurate inverse dynamics solution for unconstrained hand motion. In Eurographics/SIGGRAPH Symposium on Computer Animation, 1--10.

Digital Library

[44]

Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM Trans. Graph 26, 3.

Digital Library

[45]

van de Panne, M., and Fiume, E. 1993. Sensor-actuator networks. In SIGGRAPH, vol. 27, 335--342.

Digital Library

[46]

Xu, J., Koo, T.-K. J., and Li, Z. 2007. Finger gaits planning for multifingered manipulation. In IROS, IEEE, 2932--2937.

[47]

Yamane, K., Kuffner, J. J., and Hodgins, J. K. 2004. Synthesizing animations of human manipulation tasks. ACM Trans. Graph 23, 3, 532--539.

Digital Library

Cited By

Starke SStarke PHe NKomura TYe Y(2024)Categorical Codebook Matching for Embodied Character ControllersACM Transactions on Graphics10.1145/365820943:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658209
Hu HYi XCao ZYong JXu F(2024)Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with PhysicsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657505(1-10)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657505
Han DLee RKim KKang H(2024)VR-HandNet: A Visually and Physically Plausible Hand Manipulation System in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.325599130:7(4170-4182)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3255991
Show More Cited By

Index Terms

Synthesis of detailed hand manipulations using contact sampling
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Video-based hand manipulation capture through composite motion control

This paper describes a new method for acquiring physically realistic hand manipulation data from multiple video streams. The key idea of our approach is to introduce a composite motion control to simultaneously model hand articulation, object movement, ...
Building Hand Motion-Based Character Animation: The Case of Puppetry
CW '10: Proceedings of the 2010 International Conference on Cyberworlds

Automatic motion generation for digital character under the real-time user control is a challenging problem for computer graphic research and virtual environment applications such as on-line games. The present study introduces a methodology to generate ...
Keyframe-based multi-contact motion synthesis
Abstract
Most of the human daily activities include acyclic multi-contact motions. Yet, generating such motions is challenging because of its high-dimensional and nonlinear solution space made by combinations of individual movements of body parts. In this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 4

July 2012

935 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2185520

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Published in TOG Volume 31, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Division of Computing and Communication Foundations

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

102
Total Citations
View Citations
1,337
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Starke SStarke PHe NKomura TYe Y(2024)Categorical Codebook Matching for Embodied Character ControllersACM Transactions on Graphics10.1145/365820943:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658209
Hu HYi XCao ZYong JXu F(2024)Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with PhysicsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657505(1-10)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657505
Han DLee RKim KKang H(2024)VR-HandNet: A Visually and Physically Plausible Hand Manipulation System in Virtual RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.325599130:7(4170-4182)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3255991
Yang LZhan XLi KXu WZhang JLi JLu C(2024)Learning a Contact Potential Field for Modeling the Hand-Object InteractionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337210246:8(5645-5662)Online publication date: Aug-2024
https://doi.org/10.1109/TPAMI.2024.3372102
Wang FShi MChen CZhu JWang X(2024)Push-Assisted Functional Grasping Method Based on Prioritized Experience Replay in Clutter2024 IEEE 14th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER)10.1109/CYBER63482.2024.10749308(450-455)Online publication date: 16-Jul-2024
https://doi.org/10.1109/CYBER63482.2024.10749308
Shimada SMueller FBednarik JDoosti BBickel BTang DGolyanik VTaylor JTheobalt CBeeler T(2024)MACS: Mass Conditioned 3D Hand and Object Motion Synthesis2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00082(1082-1091)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00082
Taheri OZhou YTzionas DZhou YCeylan DPirk SBlack M(2024)GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistency2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00064(933-943)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00064
Zhang HChristen SFan ZZheng LHwangbo JSong JHilliges O(2024)ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00016(235-246)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00016
Choi MMousas CAdamo NPatankar SHauser KZhao FMayer R(2024)ASAP: animation system for agent-based presentationsThe Visual Computer10.1007/s00371-024-03622-wOnline publication date: 3-Oct-2024
https://doi.org/10.1007/s00371-024-03622-w
Li JWu JLiu C(2023)Object Motion Guided Human Motion SynthesisACM Transactions on Graphics10.1145/361833342:6(1-11)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618333
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents