skip to main content
research-article

Informing the Design of Novel Input Methods with Muscle Coactivation Clustering

Published: 14 January 2015 Publication History

Abstract

This article presents a novel summarization of biomechanical and performance data for user interface designers. Previously, there was no simple way for designers to predict how the location, direction, velocity, precision, or amplitude of users’ movement affects performance and fatigue. We cluster muscle coactivation data from a 3D pointing task covering the whole reachable space of the arm. We identify 11 clusters of pointing movements with distinct muscular, spatio-temporal, and performance properties. We discuss their use as heuristics when designing for 3D pointing.

References

[1]
Sergei Adamovich, Michail Berkinblit, Olga Fookson, and Howard Poizner. 1999. Pointing in 3D space to remembered targets II: Effects of movement speed toward kinesthetically defined targets. Experimental Brain Research 125, 2 (1999), 200--210.
[2]
Myroslav Bachynskyi, Antti Oulasvirta, Gregorio Palmas, and Tino Weinkauf. 2013. Biomechanical simulation in the analysis of aimed movements. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (CHI EA’13). ACM, New York, NY, 277--282.
[3]
Myroslav Bachynskyi, Antti Oulasvirta, Gregorio Palmas, and Tino Weinkauf. 2014. Is motion capture-based biomechanical simulation valid for HCI studies?: Study and implications. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 3215--3224.
[4]
Amartya Banerjee. 2012. Remote Multitouch: In-air Pointing Techniques for Large Display Interactions. Master’s thesis. Queen’s University, Kingston, Ontario, Canada.
[5]
Gabriel Baud-Bovy and Paolo Viviani. 1998. Pointing to kinesthetic targets in space. The Journal of Neuroscience 18, 4 (1998), 1528--1545.
[6]
Roberto Caminiti, Paul B. Johnson, Cesare Galli, Stefano Ferraina, and Yves Burnod. 1991. Making arm movements within different parts of space: The premotor and motor cortical representation of a coordinate system for reaching to visual targets. The Journal of Neuroscience 11, 5 (1991), 1182--1197.
[7]
Roberto Caminiti, Paul B. Johnson, and Antonio Urbano. 1990. Making arm movements within different parts of space: Dynamic aspects in the primate motor cortex. The Journal of Neuroscience 10, 7 (1990), 2039--2058.
[8]
Yeonjoo Cha and Rohae Myung. 2013. Extended Fitts’ law for 3D pointing tasks using 3D target arrangements. International Journal of Industrial Ergonomics 43, 4 (2013), 350--355.
[9]
Ankit Chaudhary, Jagdish Lal Raheja, Karen Das, and Sonia Raheja. 2013. Intelligent approaches to interact with machines using hand gesture recognition in natural way: A survey. arXiv preprint arXiv:1303.2292 (2013).
[10]
David J. Cooke and Susan H. Brown. 1994. Movement-related phasic muscle activation. Experimental Brain Research 99, 3 (1994), 473--482.
[11]
Paolo de Leva. 1996. Adjustments to Zatsiorsky-Seluyanov’s segment inertia parameters. Journal of Biomechanics 29, 9 (1996), 1223--1230.
[12]
Scott L. Delp, Frank C. Anderson, Allison S. Arnold, Peter Loan, Ayman Habib, and others. 2007. Opensim: Open-source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomedical Engineering 54, 11 (2007), 1940--1950. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.95.5297
[13]
Scott L. Delp, Srikanth Suranarayanan, Wendy M. Murray, Jim Uhlir, and Ronald J. Triolo. 2001. Architecture of three major trunk muscles. Journal of Biomechanics 34 (2001), 371--371.
[14]
Paul M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (1954), 381.
[15]
Martha Flanders. 1991. Temporal patterns of muscle activation for arm movements in three-dimensional space. The Journal of Neuroscience 11, 9 (1991), 2680--2693.
[16]
Hans-Joachim Freund and Hans-Joachim Büdingen. 1978. The relationship between speed and amplitude of the fastest voluntary contractions of human arm muscles. Experimental Brain Research 31, 1 (1978), 1--12.
[17]
Luigi Gallo, Alessio Pierluigi Placitelli, and Mario Ciampi. 2011. Controller-free exploration of medical image data: Experiencing the kinect. In Proceedings of the 2011 24th International Symposium on Computer-Based Medical Systems (CBMS’11). IEEE, 1--6.
[18]
Stan C. A. M. Gielen, K. Van den Oosten, and F. Pull ter Gunne. 1985. Relation between EMG activation patterns and kinematic properties of aimed arm movements. Journal of Motor Behavior 17, 4 (1985), 421--442.
[19]
Paul L. Gribble, Lucy I. Mullin, Nicholas Cothros, and Andrew Mattar. 2003. Role of cocontraction in arm movement accuracy. Journal of Neurophysiology 89, 5 (2003), 2396--2405.
[20]
Tovi Grossman and Ravin Balakrishnan. 2004. Pointing at trivariate targets in 3D environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 447--454.
[21]
Christopher M. Harris and Daniel M. Wolpert. 1998. Signal-dependent noise determines motor planning. Nature 394, 6695 (1998), 780--784.
[22]
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.). Springer.
[23]
Errol R. Hoffmann, Colin G. Drury, and Carol J. Romanowski. 2011. Performance in one-, two- and three-dimensional terminal aiming tasks. Ergonomics 54, 12 (2011), 1175--1185.
[24]
Eve Hoggan, John Williamson, Antti Oulasvirta, Miguel Nacenta, Per Ola Kristensson, and Anu Lehtiö. 2013. Multi-touch rotation gestures: Performance and ergonomics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3047--3050.
[25]
Katherine R. S. Holzbaur, Wendy M. Murray, and Scott L. Delp. 2005. A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control. Annals of Biomedical Engineering 33, 6 (2005), 829--840. http://dx.doi.org/10.1007/s10439-005-3320-7
[26]
Di-An Hong, Daniel M. Corcos, and Gerald L. Gottlieb. 1994. Task dependent patterns of muscle activation at the shoulder and elbow for unconstrained arm movements. Journal of Neurophysiology 71, 3 (1994), 1261--1265.
[27]
Anil K. Jain, M. Narasimha Murty, and Patrick J. Flynn. 1999. Data clustering: A review. ACM Computing Surveys (CSUR) 31, 3 (1999), 264--323.
[28]
Eleanor Jones, Jason Alexander, Andreas Andreou, Pourang Irani, and Sriram Subramanian. 2010. Gestext: Accelerometer-based gestural text-entry systems. In Proceedings of the 28th International Conference on Human Factors in Computing Systems. ACM, 2173--2182.
[29]
Andreas Kontaxis, Andrea Giovanni Cutti, Garth R. Johnson, and Dirk-Jan E. J. Veeger. 2009. A framework for the definition of standardized protocols for measuring upper-extremity kinematics. Clinical Biomechanics 24 (2009), 246--253.
[30]
Gail F. Koshland and Ziaul Hasan. 1994. Selection of muscles for initiation of planar, three-joint arm movements with different final orientations of the hand. Experimental Brain Research 98, 1 (1994), 157--162.
[31]
Francesco Lacquaniti, John F. Soechting, and S. A. Terzuolo. 1986. Path constraints on point-to-point arm movements in three-dimensional space. Neuroscience 17, 2 (1986), 313--324.
[32]
Belinda Lange, Chien-Yen Chang, Evan Suma, Bradley Newman, Albert Skip Rizzo, and Mark Bolas. 2011. Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In Proceedings of the 2011 Annual International Conference of the IEEE on Engineering in Medicine and Biology Society (EMBC’11). IEEE, 1831--1834.
[33]
Jui-Feng Lin and Yi-Cheng Ho. 2011. Verification of ballistic movement models in a true 3D environment. In Proceedings of the 2nd East Asian Ergonomics Federation Symposium. CRC Press.
[34]
Morten Enemark Lund, Mark de Zee, Michael Skipper Andersen, and John Rasmussen. 2012. On validation of multibody musculoskeletal models. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine 226 (2012), 82--94.
[35]
Scott I. MacKenzie. 1992. Fitts’ law as a research and design tool in human-computer interaction. Human-Computer Interaction 7, 1 (1992), 91--139.
[36]
Scott I. MacKenzie and William Buxton. 1992. Extending Fitts’ law to two-dimensional tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 219--226.
[37]
Silvestro Micera, Angelo M. Sabatini, and Paolo Dario. 2000. On automatic identification of upper-limb movements using small-sized training sets of EMG signals. Medical Engineering & Physics 22, 8 (2000), 527--533.
[38]
Silvestro Micera, Angelo M. Sabatini, Paolo Dario, and Bruno Rossi. 1999. A hybrid approach to EMG pattern analysis for classification of arm movements using statistical and fuzzy techniques. Medical Engineering & Physics 21, 5 (1999), 303--311.
[39]
Akihiko Murai, Kosuke Kurosaki, Katsu Yamane, and Yoshihiko Nakamura. 2010. Musculoskeletal-see-through mirror: Computational modeling and algorithm for whole-body muscle activity visualization in real time. Progress in Biophysics and Molecular Biology 103, 2--3 (2010), 310--317. Special Issue on Biomechanical Modelling of Soft Tissue Motion.
[40]
Chandrasekhar Narayanaswami and Mandayam T. Raghunath. 2000. Application design for a smart watch with a high resolution display. In Proceedings of the 4th International Symposium on Wearable Computers. IEEE, 7--14.
[41]
Tao Ni, Doug Bowman, and Chris North. 2011. Airstroke: Bringing unistroke text entry to freehand gesture interfaces. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems. ACM, 2473--2476.
[42]
Gregorio Palmas, Myroslav Bachynskyi, Antti Oulasvirta, Hans-Peter Seidel, and Tino Weinkauf. 2014. Movexp: A versatile visualization tool for human-computer interaction studies with 3D performance and biomechanical data. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2014), 1--1.
[43]
Réjean Plamondon and Adel M. Alimi. 1997. Speed/accuracy trade-offs in target-directed movements. Behavioral and Brain Sciences 20, 2 (1997), 279--303.
[44]
John Rasmussen, Vit Vondrak, Michael Damsgaard, Mark De Zee, Søren T. Christensen, and Zdenek Dostal. 2002. The anybody project--computer analysis of the human body. Biomechanics of Man (2002), 270--274.
[45]
Marta S. Santos, Vera Moniz-Pereira, Andre Lourenco, Ana Fred, and Antonio P. Veloso. 2014. Relevant elderly gait features for functional fitness level grouping. In Proceedings of the International Conference on Physiological Computing Systems. SCITEPRESS, 153--160.
[46]
Richard A. Schmidt, Howard Zelaznik, Brian Hawkins, James S. Frank, and John T. Quinn Jr. 1979. Motor-output variability: A theory for the accuracy of rapid motor acts. Psychological Review 86, 5 (1979), 415.
[47]
Garth Shoemaker, Leah Findlater, Jessica Q. Dawson, and Kellogg S. Booth. 2009. Mid-air text input techniques for very large wall displays. In Proceedings of Graphics Interface 2009. Canadian Information Processing Society, 231--238.
[48]
Jeff Sinclair, Philip Hingston, and Martin Masek. 2007. Considerations for the design of exergames. In Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia (GRAPHITE’07). ACM, New York, NY, USA, 289--295.
[49]
John F. Soechting, Christopher A. Buneo, Uta Herrmann, and Martha Flanders. 1995. Moving effortlessly in three dimensions: Does Donders’ law apply to arm movement? The Journal of Neuroscience 15, 9 (1995), 6271--6280.
[50]
Darryl G. Thelen, Frank C. Anderson, and Scott L. Delp. 2003. Generating dynamic simulations of movement using computed muscle control. Journal of Biomechanics 36, 3 (2003), 321--328.
[51]
Emanuel Todorov and Michael I. Jordan. 1998. Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. Journal of Neurophysiology 80, 2 (1998), 696--714.
[52]
Matthew C. Tresch, Philippe Saltiel, and Emilio Bizzi. 1999. The construction of movement by the spinal cord. Nature Neuroscience 2, 2 (1999), 162--167.
[53]
Matthieu B. Trudeau, Tawan Udtamadilok, Amy K. Karlson, and Jack T. Dennerlein. 2012a. Thumb motor performance varies by movement orientation, direction, and device size during single-handed mobile phone use. Human Factors: The Journal of the Human Factors and Ergonomics Society 54, 1 (2012), 52--59.
[54]
Matthieu B. Trudeau, Justin G. Young, Devin L. Jindrich, and Jack T. Dennerlein. 2012b. Thumb motor performance varies with thumb and wrist posture during single-handed mobile phone use. Journal of Biomechanics 45, 14 (2012), 2349--2354.
[55]
Yoji Uno, Mitsuo Kawato, and Ryoji Suzuki. 1989. Formation and control of optimal trajectory in human multijoint arm movement. Biological Cybernetics 61, 2 (1989), 89--101.
[56]
Anita N. Vasavada, Siping Li, and Scott L. Delp. 1998. Influence of muscle morphometry and moment arms on the moment-generating capacity of human neck muscles. Spine 23 (1998), 412--422.
[57]
Antonio Veloso, G. Esteves, Samuel Silva, Carlos Ferreira, and F. Brandão. 2006. Biomechanics modeling of human musculoskeletal system using Adams multibody dynamics package. In Proceedings of the 24th IASTED International Conference on Biomedical Engineering (BioMed’06). ACTA Press, Anaheim, CA, 401--407. http://dl.acm.org/citation.cfm?id=1166506.1166577
[58]
Nicolas Vignais, David M. Cocchiarella, Aaron M. Kociolek, and Peter J. Keir. 2013. Dynamic assessment of finger joint loads using kinetic and kinematic measurements. In Proceedings of the 2nd International Digital Human Modeling Symposium.
[59]
Daniel Vogel and Ravin Balakrishnan. 2005. Distant freehand pointing and clicking on very large, high resolution displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST’05). ACM, New York, NY, USA, 33--42.
[60]
Wytse J. Wadman, Jan J. Denier van der Gon, Reint H. Geuze, and C. R. Mol. 1979. Control of fast goal-directed arm movements. Journal of Human Movement Studies 5 (1979), 3--17.
[61]
Joe H. Ward Jr. 1963. Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association 58, 301 (1963), 236--244.
[62]
Alan Traviss Welford. 1968. Fundamentals of Skill. Methuen.
[63]
Thomas G. Whisenand and Henry H. Emurian. 1999. Analysis of cursor movements with a mouse. Computers in Human Behavior 15, 1 (1999), 85--103.
[64]
Margaret M. Wierzbicka, Allen W. Wiegner, and Bhagwan T. Shahani. 1986. Role of agonist and antagonist muscles in fast arm movements in man. Experimental Brain Research 63, 2 (1986), 331--340.
[65]
Christian Winkler, Ken Pfeuffer, and Enrico Rukzio. 2012. Investigating mid-air pointing interaction for projector phones. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS’12). ACM, New York, NY, 85--94.
[66]
Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, 1083--1092.

Cited By

View all

Index Terms

  1. Informing the Design of Novel Input Methods with Muscle Coactivation Clustering

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Computer-Human Interaction
    ACM Transactions on Computer-Human Interaction  Volume 21, Issue 6
    Special Issue on Physiological Computing for Human-Computer Interaction
    January 2015
    144 pages
    ISSN:1073-0516
    EISSN:1557-7325
    DOI:10.1145/2722827
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 January 2015
    Accepted: 01 September 2014
    Revised: 01 August 2014
    Received: 01 December 2013
    Published in TOCHI Volume 21, Issue 6

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Muscle coactivation clustering
    2. biomechanical simulation
    3. physical ergonomics
    4. user interface design
    5. user performance

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    • Cluster of Excellence for Multimodal Computing and Interaction at Saarland University
    • Max Planck Center for Visual Computing and Communication and the International Max Planck Research School for Computer Science at the Max Planck Institute for Informatics

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)38
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 09 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media