research-article

Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING

Authors:

Soheil Kianzad,

Karon E. MacLeanAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 12

https://doi.org/10.1145/3313831.3376134

Published: 23 April 2020 Publication History

Abstract

When sketching, we must choose between paper (expressive ease, ruler and eraser) and computational assistance (parametric support, a digital record). PHysically Assisted SKetching provides both, with a pen that displays force constraints with which the sketcher interacts as they draw on paper. Phasking provides passive, "bound" constraints (like a ruler); or actively "brings" the sketcher along a commanded path (e.g., a curve), which they can violate for creative variation. The sketcher modulates constraint strength (control sharing) by bearing down on the pen-tip. Phasking requires untethered, graded force-feedback, achieved by modifying a ballpoint drive that generates force through rolling surface contact. To understand phasking's viability, we implemented its interaction concepts, related them to sketching tasks and measured device performance. We assessed the experience of 10 sketchers, who could understand, use and delight in phasking, and who valued its control-sharing and digital twinning for productivity, creative control and learning to draw.

Supplemental Material

MP4 File

Download
46.16 MB

MP4 File

Preview video

Download
7.79 MB

MP4 File

Supplemental video

Download
99.89 MB

ZIP File

The Dataset here presents the results of drawing experiments Neo: No assist (NeoSmart pen) Bring: Physical assists-active constraint Bound: physical assists-passive constraint SC: shared control NSC: No shared control

Download
9.13 MB

References

[1]

David A. Abbink, Mark Mulder, and Erwin R. Boer. 2012. Haptic shared control: smoothly shifting control authority. Cognition, Technology Work 14, 1 (2012), 19--28. http://dx.doi.org/10.1007/s10111-011-0192--5

Digital Library

[2]

Carlo A. Avizzano, Massimo Satler, Giovanni Cappiello, Andrea Scoglio, Emanuele Ruffaldi, and Massimo Bergamasco. 2011. MOTORE: A mobile haptic interface for neuro-rehabilitation. In 2011 RO-MAN. IEEE, 383--388.

[3]

B. Bayart, A. Pocheville, and A. Kheddar. 2005. An adaptive haptic guidance software module for I-TOUCH: example through a handwriting teaching simulation and a 3D maze. In IEEE International Workshop on Haptic Audio Visual Environments and their Applications. 6 pp.--.

[4]

BOSCH. 2015. Absolute orientation sensor. (2015). https://www.bosch-sensortec.com/bst/products/all_ products/bno055

[5]

J.E. Colgate, W. Wannasuphoprasit, and M.A. Peshkin. 1996. Cobots: Robots for Collaboration with Human Operators. In the 5th Ann. Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems, ASME/IMECE, Vol. DSC:58. Atlanta, GA, 443--439.

[6]

Heather Culbertson and Katherine J Kuchenbecker. 2017. Ungrounded Haptic Augmented Reality System for Displaying Roughness and Friction. IEEE/ASME Transactions on Mechatronics 22, 4 (2017), 1839--1849. https://ieeexplore.ieee.org/abstract/document/7917356

[7]

Daniel Dixon, Manoj Prasad, and Tracy Hammond. 2010. iCanDraw: Using Sketch Recognition and Corrective Feedback to Assist a User in Drawing Human Faces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Atlanta, Georgia, USA New York, NY, USA, 897--906.

Digital Library

[8]

Mohamad A. Eid, Mohamed Mansour, Abdulmotaleb H. El Saddik, and Rosa Iglesias. 2007. A Haptic Multimedia Handwriting Learning System. In Proceedings of the International Workshop on Educational Multimedia and Multimedia Education (Emme '07). ACM, NY, NY, USA, 103--108.

Digital Library

[9]

Pololu Rootics & Electronics. 2019. Micro Metal Gearmotors. (2019). https://www.pololu.com/category/60/micro-metal-gearmotors

[10]

Piyum Fernando, Roshan Lalintha Peiris, and Suranga Nanayakkara. 2014. I-Draw: Towards a Freehand Drawing Assistant. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures the Future of Design - OzCHI'14. ACM Press, New York, New York, USA, 208--211.

Digital Library

[11]

Ben Forsyth and Karon E. MacLean. 2006. Predictive Haptic Guidance: Intelligent User Assistance for the Control of Dynamic Tasks. IEEE Trans on Visualization and Computer Graphics 12, 1 (2006), 103--113. http:// www.computer.org/csdl/trans/tg/2006/01/v0103-abs.html

Digital Library

[12]

Paul Griffiths and R. Brent Gillespie. 2004. Shared Control between Human and Machine: Haptic Display of Automation during Manual Control of Vehicle Heading. In the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS'04). 358--366.

[13]

Anoto Inc. 2017. Anoto Digital Pen. (2017). http://www.anoto.com/

[14]

Neo Smartpen Inc. 2018. Neo smartpen M1. (2018). https://www.neosmartpen.com

[15]

Oliver Beren Kaul, Max Pfeiffer, and Michael Rohs. 2016. Follow the Force: Steering the Index Finger Towards Targets Using EMS. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, NY, NY, USA, 2526--2532.

Digital Library

[16]

Soheil Kianzad and Karon E MacLean. 2018. Harold's purple crayon rendered in haptics: Large-stroke, handheld ballpoint force feedback. In 2018 IEEE Haptics Symposium (HAPTICS). 106--111.

[17]

Yong Jae Lee, C. Lawrence Zitnick, and Michael F. Cohen. 2011. ShadowDraw: Real-time User Guidance for Freehand Drawing. ACM Trans. Graph. 30, 4 (2011), 27:1--27:10.

Digital Library

[18]

Golan Levin. 2000. Painterly Interfaces for Audiovisual Performance. Ph.D. Dissertation. Massachusetts Institute of Technology, Cambridge.

[19]

Y. Li, J. C. Huegel, V. Patoglu, and M. K. O'Malley. 2009. Progressive shared control for training in virtual environments. In 3rd Worldhaptics Conference (WHC '09). IEEE Press, Salt Lake City, UT, 332--337.

Digital Library

[20]

Riverbank Computing Limited. 2019. PyQT. (2019). https://riverbankcomputing.com/software/pyqt/intro

[21]

Long-Fei Lin, Shan-Yuan Teng, Rong-Hao Liang, and Bing-Yu Chen. 2016. Stylus Assistant: Designing Dynamic Constraints for Facilitating Stylus Inputs on Portable Displays. In SIGGRAPH ASIA 2016 Emerging Technologies (SA '16). ACM, NY, NY, USA, Article 14, 2 pages.

Digital Library

[22]

Pedro Lopes, Doaa Yuksel, Francois Guimbretiere, and Patrick Baudisch. 2016. Muscle-plotter: an Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output. In 29th Annual Symposium on User Interface Software and Technology. ACM Press, New York, New York, USA, 207--217.

Digital Library

[23]

Pam A. Mueller and Daniel M. Oppenheimer. 2014. The Pen Is Mightier Than the Keyboard: Advantages of Longhand Over Laptop Note Taking. Psychological Science 25, 6 (2014), 1159--1168. https://doi.org/10.1177/0956797614524581

[24]

J. Mullins, C. Mawson, and S. Nahavandi. 2005. Haptic handwriting aid for training and rehabilitation. In 2005 IEEE International Conference on Systems, Man and Cybernetics, Vol. 3. 2690--2694 Vol. 3.

[25]

Ken Nakagaki and Yasuaki Kakehi. 2014. Comp*Pass: A Compass-based Drawing Interface. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). ACM, NY, NY, USA, 447--450.

Digital Library

[26]

Akira Nomoto, Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2016. Supporting Precise Manual-handling Task Using Visuo-haptic Interaction. In Proceedings of the 7th Augmented Human International Conference 2016 (AH '16). ACM, NY, NY, USA, Article 10, 8 pages.

Digital Library

[27]

Ayberk Özgür, Wafa Johal, Francesco Mondada, and Pierre Dillenbourg. 2017. Haptic-Enabled Handheld Mobile Robots: Design and Analysis. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 2449--2461.

Digital Library

[28]

S. Papert. 1980. Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, New York.

Digital Library

[29]

Mark Price and Frank C. Sup. 2016. A robotic touchscreen totem for two-dimensional haptic force display. In 2016 IEEE Haptics Symposium (HAPTICS). IEEE, 72--77.

[30]

Mitchel Resnick, Fred Martin, Robert Berg, Rick Borovoy, Vanessa Colella, Kwin Kramer, and Brian Silverman. 1998. Digital Manipulatives: New Toys to Think with. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., Los Angeles, California, USA New York, NY, USA, 281--287.

Digital Library

[31]

Hasti Seifi, Gunhyuk Park, Farimah Fazlollahi, John Sastrillo, Jessica Ip, Ashutosh Agrawal, Katherine Kuchenbecher, and Karon E. MacLean. 2019. Haptipedia: Accelerating Haptic Device Discovery to Support Interaction Engineering Design. In Conference on Human Factors in Computing Systems (CHI'19). ACM, Glasgow, UK, 12 pgs.

[32]

Yu shan Chang, Mavis Yi-Ching Chen, Meng-Jung Chuang, and Chia hui Chou. 2019. Improving creative self-efficacy and performance through computer-aided design application. Thinking Skills and Creativity 31 (2019), 103 -- 111.

[33]

Scott Snibbe, Sean Anderson, and Bill Verplank. 1998. Springs and Constraints for 3D Drawing. In Third Phantom User's Group, Vol. M.I.T. Artificial Intelligence Laboratory Technical Report AITR-1643. 1--4.

[34]

Hyunyoung Song, Tovi Grossman, George Fitzmaurice, François Guimbretiere, Azam Khan, Ramtin Attar, and Gordon Kurtenbach. 2009. PenLight: Combining a Mobile Projector and a Digital Pen for Dynamic Visual Overlay. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 143--152.

Digital Library

[35]

Jürgen Steimle. 2009. Designing pen-and-paper user interfaces for interaction with documents. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction - TEI '09. ACM Press, New York, New York, USA, 197--204.

Digital Library

[36]

A Teranishi, G Korres, W Park, and M Eid. 2018. Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development. IEEE Transactions on Haptics 11, 4 (2018), 509--517. https://ieeexplore.ieee.org/abstract/document/8400473

Digital Library

[37]

P. Vandoren, T. Van Laerhoven, L. Claesen, J. Taelman, C. Raymaekers, and F. Van Reeth. 2008. IntuPaint: Bridging the gap between physical and digital painting. In 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems. 65--72.

[38]

Junichi Yamaoka and Yasuaki Kakehi. 2013. dePENd: Augmented Handwriting System Using Ferromagnetism of a Ballpoint Pen. In 26th Annual ACM Symposium on User Interface Software and Technology. ACM, St. Andrews, Scotland, United Kingdom New York, NY, USA, 203--210.

Digital Library

[39]

Raphael F Zory, Delphine Boërio, Marc Jubeau, and Nicola A Maffiuletti. 2005. Central and peripheral fatigue of the knee extensor muscles induced by electromyostimulation. International journal of sports medicine 26 10 (2005), 847--53.

Cited By

Kianzad SSeifi H(2024)MagicDraw: Haptic-Assisted One-Line Drawing with Shared ControlAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686753(1-3)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686753
Fang CHuang LKuang QLieberman ZMaes PIshii H(2024)An Accessible, Three-Axis Plotter for Enhancing Calligraphy Learning through Generated MotionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642792(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642792
Fender ARoberts TLuong THolz C(2023)InfinitePaint: Painting in Virtual Reality with Passive Haptics Using Wet Brushes and a Physical Proxy CanvasProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580927(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580927
Show More Cited By

Index Terms

Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Tactile and hand-based interfaces
      1. Haptic devices
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
  2. Interaction design
    1. Systems and tools for interaction design

Recommendations

Sketching user experiences tutorial: stories, strategies, surfaces
ITS '13: Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces

Paper-pencil sketches are a valuable tool during different stages of experience design in human-computer interaction. This hands-on tutorial will demonstrate how to integrate sketching into researchers' and interaction designers' everyday practice -- ...
Sketching beyond paper and pencil: lessons learned from using Sketchify
DESIRE '10: Proceedings of the 1st DESIRE Network Conference on Creativity and Innovation in Design

Some recent discussions in the interaction design community have called attention to sketching as an omnipresent element of any disciplined activity of design, and have pointed out that sketching should be extended beyond the simple creation of a pencil ...
Hands-on sketching course
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

Sketching as a technique to quickly draw something on a piece of paper is used to explore and communicate ideas. This course introduces basic sketching techniques and a visual language which participants can apply in research and practice. It is a hands-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

NSERC

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
1,148
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kianzad SSeifi H(2024)MagicDraw: Haptic-Assisted One-Line Drawing with Shared ControlAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686753(1-3)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686753
Fang CHuang LKuang QLieberman ZMaes PIshii H(2024)An Accessible, Three-Axis Plotter for Enhancing Calligraphy Learning through Generated MotionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642792(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642792
Fender ARoberts TLuong THolz C(2023)InfinitePaint: Painting in Virtual Reality with Passive Haptics Using Wet Brushes and a Physical Proxy CanvasProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580927(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580927
Seifi HChew SNascè ALowther WFrier WHornbæk K(2023)Feellustrator: A Design Tool for Ultrasound Mid-Air HapticsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580728(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580728
Pourjafarian NMjaku FKoelle MSchmitz MBorchers JSteimle J(2023)Handheld Tools Unleashed: Mixed-Initiative Physical Sketching with a Robotic PrinterProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580691(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580691
Cimolino GGraham T(2022)Two Heads Are Better Than One: A Dimension Space for Unifying Human and Artificial Intelligence in Shared ControlProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517610(1-21)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517610
Pourjafarian NKoelle MMjaku FStrohmeier PSteimle J(2022)Print-A-Sketch: A Handheld Printer for Physical Sketching of Circuits and Sensors on Everyday SurfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502074(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502074
Yonezawa TTanaka F(2022)Handwriting Messenger by Which the User Can Feel the Presence of Communication PartnersCompanion Proceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490100.3516454(58-61)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490100.3516454
Morisaki TMori RMori RSerizawa KMakino YItoh YYamakawa YShinoda H(2021)Ultrasound-driven Curveball in Table TennisProceedings of the ACM on Human-Computer Interaction10.1145/34885485:ISS(1-20)Online publication date: 5-Nov-2021
https://dl.acm.org/doi/10.1145/3488548
MacLean K(2021)Incorporating Haptics into the Theatre of Multimodal Experience designProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3480860(1-2)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3480860
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten