skip to main content
10.1145/3173574.3174082acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Neuromechanics of a Button Press

Published: 21 April 2018 Publication History

Abstract

To press a button, a finger must push down and pull up with the right force and timing. How the motor system succeeds in button-pressing, in spite of neural noise and lacking direct access to the mechanism of the button, is poorly understood. This paper investigates a unifying account based on neuromechanics. Mechanics is used to model muscles controlling the finger that contacts the button. Neurocognitive principles are used to model how the motor system learns appropriate muscle activations over repeated strokes though relying on degraded sensory feedback. Neuromechanical simulations yield a rich set of predictions for kinematics, dynamics, and user performance and may aid in understanding and improving input devices. We present a computational implementation and evaluate predictions for common button types.

References

[1]
Kenichi Akagi. 1992. A computer keyboard key feel study in performance and preference. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 36. SAGE Publications Sage CA: Los Angeles, CA, 523--527.
[2]
Gisa Aschersleben. 2002. Temporal control of movements in sensorimotor synchronization. Brain and cognition 48, 1 (2002), 66--79.
[3]
Gisa Aschersleben, Jörg Gehrke, and Wolfgang Prinz. 2004. A psychophysical approach to action timing. Psychophysics beyond sensation: Laws and invariants of human cognition (2004), 117--136.
[4]
Gisa Aschersleben and Wolfgang Prinz. 1995. Synchronizing actions with events: The role of sensory information. Attention, Perception,&Psychophysics 57, 3 (1995), 305--317.
[5]
W Ross Ashby. 1961. An introduction to cybernetics. Chapman&Hall Ltd.
[6]
Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, and Tino Weinkauf. 2015. Informing the design of novel input methods with muscle coactivation clustering. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 6 (2015), 30.
[7]
Joanna Bergstrom-Lehtovirta, Antti Oulasvirta, and Stephen Brewster. 2011. The effects of walking speed on target acquisition on a touchscreen interface. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 143--146.
[8]
W Thomas Bourbon. 1995. Perceptual control theory. Comparative approaches to cognitive science (1995), 151--172.
[9]
Hans Brunner and Rose Mae Richardson. 1984. Effects of keyboard design and typing skill on user keyboard preferences and throughput performance. In Proceedings of the Human Factors Society Annual Meeting, Vol. 28. SAGE Publications Sage CA: Los Angeles, CA, 267--271.
[10]
Etienne Burdet, David W Franklin, and Theodore E Milner. 2013. Human robotics: neuromechanics and motor control. MIT press.
[11]
Timothy A Carey, Warren Mansell, and Sara J Tai. 2014. A biopsychosocial model based on negative feedback and control. Frontiers in human neuroscience 8 (2014).
[12]
Géry Casiez, Thomas Pietrzak, Damien Marchal, Sébastien Poulmane, Matthieu Falce, and Nicolas Roussel. 2017. Characterizing Latency in Touch and Button-Equipped Interactive Systems. In UIST 2017.
[13]
Andy Clark. 2013. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences 36, 3 (2013), 181--204.
[14]
Matthew JC Crump and Gordon D Logan. 2010. Warning: This keyboard will deconstruct--The role of the keyboard in skilled typewriting. Psychonomic bulletin&review 17, 3 (2010), 394--399.
[15]
Hannes Eisler, Anna D Eisler, and Åke Hellström. 2008. Psychophysical issues in the study of time perception. Psychology of time (2008), 75--109.
[16]
Roger M Enoka. 2008. Neuromechanics of human movement. Human kinetics.
[17]
Marc O Ernst. 2006. A Bayesian view on multimodal cue integration. Human body perception from the inside out 131 (2006), 105--131.
[18]
Marc O Ernst and Heinrich H Bülthoff. 2004. Merging the senses into a robust percept. Trends in cognitive sciences 8, 4 (2004), 162--169.
[19]
PSE Farrell, JG Hollands, MM Taylor, and HD Gamble. 1999. Perceptual control and layered protocols in interface design: I. Fundamental concepts. International Journal of Human-Computer Studies 50, 6 (1999), 489--520.
[20]
Jose Gaspar, Mihail Fontul, Elsa Henriques, and Arlindo Silva. 2017. Haptics of in-car radio buttons and its relationship with engineering parameters. International Journal of Industrial Ergonomics 59 (2017), 29--45.
[21]
Werner Goebl and Caroline Palmer. 2009. Synchronization of timing and motion among performing musicians. Music Perception: An Interdisciplinary Journal 26, 5 (2009), 427--438.
[22]
Hiroaki Gomi. 2008. Implicit online corrections of reaching movements. Current opinion in neurobiology 18, 6 (2008), 558--564.
[23]
Taku Hachisu and Hiroyuki Kajimoto. 2017. Vibration Feedback Latency Affects Material Perception during Rod Tapping Interactions. IEEE transactions on haptics (2017).
[24]
AV Hill. 1938. The heat of shortening and the dynamic constants of muscle. Proceedings of the Royal Society of London B: Biological Sciences 126, 843 (1938), 136--195.
[25]
Eve Hoggan, Stephen A Brewster, and Jody Johnston. 2008. Investigating the effectiveness of tactile feedback for mobile touchscreens. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 1573--1582.
[26]
Jakob Hohwy. 2013. The predictive mind. Oxford University Press.
[27]
Michael J Hove and Peter E Keller. 2010. Spatiotemporal relations and movement trajectories in visuomotor synchronization. Music Perception: An Interdisciplinary Journal 28, 1 (2010), 15--26.
[28]
Bruce Hutcheon and Yosef Yarom. 2000. Resonance, oscillation and the intrinsic frequency preferences of neurons. Trends in neurosciences 23, 5 (2000), 216--222.
[29]
Curt B Irwin and Mary E Sesto. 2012. Performance and touch characteristics of disabled and non-disabled participants during a reciprocal tapping task using touch screen technology. Applied Ergonomics 43, 6 (Nov. 2012), 1038--1043.
[30]
Richard J Jagacinski and John M Flach. 2003. Control theory for humans: Quantitative approaches to modeling performance. CRC Press.
[31]
Roland S Johansson and J Randall Flanagan. 2009. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature reviews. Neuroscience 10, 5 (2009), 345.
[32]
Lynette A Jones and Susan J Lederman. 2006. Human hand function. Oxford University Press.
[33]
Topi Kaaresoja, Stephen Brewster, and Vuokko Lantz. 2014. Towards the temporally perfect virtual button: touch-feedback simultaneity and perceived quality in mobile touchscreen press interactions. ACM Transactions on Applied Perception (TAP) 11, 2 (2014), 9.
[34]
Antti Kangasrääsiö, Kumaripaba Athukorala, Andrew Howes, Jukka Corander, Samuel Kaski, and Antti Oulasvirta. 2017. Inferring Cognitive Models from Data using Approximate Bayesian Computation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1295--1306.
[35]
Mitsuo Kawato. 1999. Internal models for motor control and trajectory planning. Current opinion in neurobiology 9, 6 (1999), 718--727.
[36]
Jeong Ho Kim, Lovenoor Aulck, Michael C Bartha, Christy A Harper, and Peter W Johnson. 2014. Differences in typing forces, muscle activity, comfort, and typing performance among virtual, notebook, and desktop keyboards. Applied ergonomics 45, 6 (2014), 1406--1413.
[37]
Sunjun Kim, Jeongmin Son, Geehyuk Lee, Hwan Kim, and Woohun Lee. 2013. TapBoard: making a touch screen keyboard more touchable. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 553--562.
[38]
Sung Soo Kim, Arun P Sripati, and Sliman J Bensmaia. 2010. Predicting the timing of spikes evoked by tactile stimulation of the hand. Journal of neurophysiology 104, 3 (2010), 1484--1496.
[39]
David C Knill and Alexandre Pouget. 2004. The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences 27, 12 (2004), 712--719.
[40]
Konrad P Körding and Daniel M Wolpert. 2006. Bayesian decision theory in sensorimotor control. Trends in cognitive sciences 10, 7 (2006), 319--326.
[41]
Byungjoo Lee, Qiao Deng, Eve Hoggan, and Antti Oulasvirta. 2017. Boxer: A Multimodal Collision Technique for Virtual Objects. In Proceedings of 19th ACM International Conference on Multimodal Interaction (ICMI'17). ACM.
[42]
Byungjoo Lee and Antti Oulasvirta. 2016. Modelling error rates in temporal pointing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1857--1868.
[43]
Byungjoo Lee, Kim Sunjun, Antti Oulasvirta, Jong-In Lee, and Eunji Park. 2018. Moving Target Selection: A Cue Integration Model. In Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems (CHI'18). ACM Press.
[44]
Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics (TOG) 28, 4 (2009), 99.
[45]
Sung-Hee Lee and Demetri Terzopoulos. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM Transactions on Graphics (TOG), Vol. 25. ACM, 1188--1198.
[46]
James R. Lewis, Kathleen M. Potosnak, and Regis L. Magyar. 1997. Chapter 54 - Keys and Keyboards. In Handbook of Human-Computer Interaction (Second Edition) (second edition ed.), Marting G. Helander, Thomas K. Landauer, and Prasad V. Prabhu (Eds.). North-Holland, Amsterdam, 1285 --1315.
[47]
Jon Moeller and Andruid Kerne. 2012. ZeroTouch: an optical multi-touch and free-air interaction architecture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2165--2174.
[48]
Jon Moeller, Nic Lupfer, Bill Hamilton, Haiqiao Lin, and Andruid Kerne. 2011. intangibleCanvas: free-air finger painting on a projected canvas. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. ACM, 1615--1620.
[49]
John Morton, Steve Marcus, and Clive Frankish. 1976. Perceptual centers (P-centers). Psychological Review 83, 5 (1976), 405.
[50]
Jörg Müller, Antti Oulasvirta, and Roderick Murray-Smith. 2017. Control Theoretic Models of Pointing. ACM Transactions on Computer-Human Interaction (TOCHI) 24, 4 (2017), 27.
[51]
Mark Nagurka and Richard Marklin. 2005. Measurement of stiffness and damping characteristics of computer keyboard keys. Journal of dynamic systems, measurement, and control 127, 2 (2005), 283--288.
[52]
Kiisa Nishikawa, Andrew A Biewener, Peter Aerts, Anna N Ahn, Hillel J Chiel, Monica A Daley, Thomas L Daniel, Robert J Full, Melina E Hale, Tyson L Hedrick, and others. 2007. Neuromechanics: an integrative approach for understanding motor control. Integrative and Comparative Biology 47, 1 (2007), 16--54.
[53]
Dan Odell and Eric Faggin. 2014. The typing performance and preference costs of reducing tactile feedback and tactile landmarks in tablet keyboards. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 58. SAGE Publications Sage CA: Los Angeles, CA, 1790--1794.
[54]
Antti Oulasvirta and Joanna Bergstrom-Lehtovirta. 2010. A simple index for multimodal flexibility. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1475--1484.
[55]
Denham L Phipps and Don Harris. 2014. Perceptual control and feedback control in the analysis of complex tasks. Theoretical Issues in Ergonomics Science 15, 5 (2014), 505--516.
[56]
William T Powers. 2009. Perceptual Control Theory: Science&Applications: A Book of Readings. Living Control Systems Publ.
[57]
William T Powers and William T Powers. 1973. Behavior: The control of perception. Aldine Chicago.
[58]
Robert G Radwin and One-Jang Jeng. 1997. Activation force and travel effects on overexertion in repetitive key tapping. Human Factors 39, 1 (1997), 130--140.
[59]
Robert G Radwin and Barry A Ruffalo. 1999. Computer key switch force-displacement characteristics and short-term effects on localized fatigue. Ergonomics 42, 1 (1999), 160--170.
[60]
David Rempel, Elaine Serina, Edward Klinenberg, Bernard J Martin, Thomas J Armstrong, James A Foulke, and Sivakumaran Natarajan. 1997. The effect of keyboard keyswitch make force on applied force and finger flexor muscle activity. Ergonomics 40, 8 (1997), 800--808.
[61]
Bruno H Repp. 2005. Sensorimotor synchronization: a review of the tapping literature. Psychonomic bulletin&review 12, 6 (2005), 969--992.
[62]
Stefan Schaal and Nicolas Schweighofer. 2005. Computational motor control in humans and robots. Current opinion in neurobiology 15, 6 (2005), 675--682.
[63]
Steven C Seow. 2005. Information theoretic models of HCI: a comparison of the Hick-Hyman law and Fitts' law. Human-Computer Interaction 20, 3 (2005), 315--352.
[64]
Anil K Seth. 2014. The cybernetic Bayesian brain. In Open Mind. Open MIND. Frankfurt am Main: MIND Group.
[65]
Tobias Siebert, Christian Rode, Walter Herzog, Olaf Till, and Reinhard Blickhan. 2008. Nonlinearities make a difference: comparison of two common Hill-type models with real muscle. Biological cybernetics 98, 2 (2008), 133--143.
[66]
Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta. 2015. Investigating the dexterity of multi-finger input for mid-air text entry. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3643--3652.
[67]
Kim Sunjun, Byungjoo Lee, and Antti Oulasvirta. 2018. Impact Activation Improves Rapid Button Pressing. In Proceedings of the 36th Annual ACM Conference on Human Factors in Computing Systems (CHI'18). ACM Press.
[68]
Lena H Ting and J Lucas McKay. 2007. Neuromechanics of muscle synergies for posture and movement. Current opinion in neurobiology 17, 6 (2007), 622--628.
[69]
Kjerstin Torre and Ramesh Balasubramaniam. 2009. Two different processes for sensorimotor synchronization in continuous and discontinuous rhythmic movements. Experimental Brain Research 199, 2 (2009), 157--166.
[70]
Michel Treisman. 1963. Temporal discrimination and the indifference interval: Implications for a model of the" internal clock". Psychological Monographs: General and Applied 77, 13 (1963), 1.
[71]
Macy Ann Valk. 1985. An experiment to study touchscreen button design. In Proceedings of the Human Factors Society Annual Meeting, Vol. 29. SAGE Publications Sage CA: Los Angeles, CA, 127--131.
[72]
Michael Wiertlewski and Vincent Hayward. 2012. Mechanical behavior of the fingertip in the range of frequencies and displacements relevant to touch. Journal of biomechanics 45, 11 (2012), 1869--1874.
[73]
John Z Wu, Ren G Dong, S Rakheja, and AW Schopper. 2002. Simulation of mechanical responses of fingertip to dynamic loading. Medical Engineering&Physics 24, 4 (2002), 253--264.
[74]
John Z Wu, Ren G Dong, S Rakheja, AW Schopper, and WP Smutz. 2004. A structural fingertip model for simulating of the biomechanics of tactile sensation. Medical engineering&physics 26, 2 (2004), 165--175.

Cited By

View all

Index Terms

  1. Neuromechanics of a Button Press

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
    April 2018
    8489 pages
    ISBN:9781450356206
    DOI:10.1145/3173574
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2018

    Check for updates

    Badges

    • Honorable Mention

    Author Tags

    1. buttons
    2. control theory
    3. input devices
    4. input engineering
    5. neuromechanics
    6. perceptual control
    7. probabilistic motor control

    Qualifiers

    • Research-article

    Funding Sources

    • european research council
    • national research council of korea

    Conference

    CHI '18
    Sponsor:

    Acceptance Rates

    CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)709
    • Downloads (Last 6 weeks)46
    Reflects downloads up to 09 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media