abstract

Virtual Reality for User-Centered Design and Evaluation of Touch-free Interaction Techniques for Navigating Medical Images in the Operating Room

Authors:

Anke Verena Reinschluessel,

Jeffrey Bissel,

Melanie van Eikeren,

Johannes Ganser,

Felicia Koeller,

Fenja Kollasch,

Thomas Mildner,

Gabriel Zachmann,

Rainer MalakaAuthors Info & Claims

CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Pages 2001 - 2009

https://doi.org/10.1145/3027063.3053173

Published: 06 May 2017 Publication History

Abstract

Computer-assisted surgery has pervaded the operating room (OR). While display and imaging technologies advance rapidly, keyboard and mouse are still the dominant input devices, even though they cause sterility problems. We present an interactive virtual operating room (IVOR), intended as a tool to develop and study interaction methods for the OR, and two novel touch-free interaction techniques using hand and foot gestures. All was developed and evaluated with 20 surgeons. The results show that our techniques can be used with minimal learning time and no significant differences regarding completion time and usability compared to the control condition relying on verbal instruction of an assistant. Furthermore, IVOR as a tool was well received by the surgeons, although they had no prior experience with virtual reality. This confirms IVOR is an effective tool for user-centered design and evaluation, providing a portable, yet realistic substitution for a real OR for early evaluations.

References

[1]

Alexandra Branzan Albu. 2006. Vision-Based User Interfaces for Health Applications: A Survey. Springer Berlin Heidelberg, Berlin, Heidelberg, 771--782.

Digital Library

[2]

Jason Alexander, Teng Han, William Judd, Pourang Irani, and Sriram Subramanian. 2012. Putting your best foot forward: investigating real-world mappings for foot-based gestures. In Proceedings of the 30th international conference on Human factors in computing systems (CHI 2012). ACM, Austin, Texas, 1229--1238.

Digital Library

[3]

M. E. Allaf, S. V. Jackman, P. G. Schulam, J. A. Cadeddu, B. R. Lee, R. G. Moore, and L. R. Kavoussi. 1998. Laparoscopic visual field. Surgical Endoscopy 12, 12 (1998), 1415--1418.

[4]

Nigel Bevan and Ian Curson. 1999. Planning and Implementing User-centred Design. In CHI '99 Extended Abstracts on Human Factors in Computing Systems (CHI EA '99). ACM, New York, NY, USA, 137--138.

Digital Library

[5]

M Billinghurst, J Savage, P Oppenheimer, and C Edmond. 1996. The Expert Surgical Assitant: An Intelligent Virtual Environment with Multimodal Input. Studies in health technology and informatics (1996), 590--607.

[6]

Anthony G Gallagher, E Matt Ritter, Howard Champion, Gerald Higgins, Marvin P Fried, Gerald Moses, C Daniel Smith, and Richard M Satava. 2005. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Annals of surgery 241, 2 (2005), 364--372.

[7]

Sébastien Grange, Terrence Fong, and Charles Baur. 2004. M/ORIS: a medical/operating room interaction system. In Proceedings of the 6th international conference on Multimodal interfaces. ACM, 159--166.

Digital Library

[8]

Teodor P Grantcharov, VB Kristiansen, Jørgen Bendix, L Bardram, J Rosenberg, and Peter Funch-Jensen. 2004. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. British Journal of Surgery 91, 2 (2004), 146--150.

[9]

C. Grätzel, T. Fong, S. Grange, and C. Baur. 2004. A non-contact mouse for surgeon-computer interaction. Technology and health care : official journal of the European Society for Engineering and Medicine 12, 3 (2004), 245--257.

[10]

Shahram Jalaliniya, Jeremiah Smith, Miguel Sousa, Lars Büthe, and Thomas Pederson. 2013. Touch-less interaction with medical images using hand & foot gestures. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, September 8 - 12, 2013, Zurich, Switzerland ; co-located with the 2013 International Symposium on Wearable Computers (ISWC) ; [including] UbiComp 2013 adjunct publication, Marc Langheinrich (Ed.). ACM, New York, NY, 1265--1274.

Digital Library

[11]

Rose Johnson, Kenton O'Hara, Abigail Sellen, Claire Cousins, and Antonio Criminisi. 2011. Exploring the potential for touchless interaction in image-guided interventional radiology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3323--3332.

Digital Library

[12]

Franklin King, Jagadeesan Jayender, Sharath K Bhagavatula, Paul B Shyn, Steve Pieper, Tina Kapur, Andras Lasso, and Gabor Fichtinger. 2016. An immersive virtual reality environment for diagnostic imaging. Journal of Medical Robotics Research 1, 01 (2016), 1640003.

[13]

T Kipshagen, M Graw, V Tronnier, M Bonsanto, and UG Hofmann. 2009. Touch-and marker-free interaction with medical software. In World Congress on Medical Physics and Biomedical Engineering, September 7--12, 2009, Munich, Germany. Springer, 75--78.

[14]

John Kirby and Heather A. Heathfield. 1993. Analysis and Design Techniques for User Centred Design. In INTERACT '93 and CHI '93 Conference Companion on Human Factors in Computing Systems (CHI '93). ACM, New York, NY, USA, 205--206.

Digital Library

[15]

M Ma, P Fallavollita, S Habert, S Weidert, and N Navab. 2016. Device-and system-independent personal touchless user interface for operating rooms: One personal UI to control all displays in an operating room. International journal of computer assisted radiology and surgery 11, 6 (2016), 853--861.

[16]

Agata Manolova. 2014. System for touchless interaction with medical images in surgery using Leap Motion. In Proceedings og the 9th INTERNATIONAL CONFERENCE on Communications, Electromagnetics and Medical Applications. 2--6.

[17]

Helena M Mentis, Kenton O'Hara, Abigail Sellen, and Rikin Trivedi. 2012. Interaction proxemics and image use in neurosurgery. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 927--936.

Digital Library

[18]

A Mewes, P Saalfeld, O Riabikin, M Skalej, and C Hansen. 2016. A gesture-controlled projection display for CT-guided interventions. International journal of computer assisted radiology and surgery 11, 1 (2016), 157--164.

[19]

Kenton O'Hara, Gerardo Gonzalez, Graeme Penney, Abigail Sellen, Robert Corish, Helena Mentis, Andreas Varnavas, Antonio Criminisi, Mark Rouncefield, Neville Dastur, and others. 2014. Interactional order and constructed ways of seeing with touchless imaging systems in surgery. Computer Supported Cooperative Work (CSCW) 23, 3 (2014), 299--337.

Digital Library

[20]

A. L. Rector, B. Horan, M. Fitter, S. Kay, P. D. Newton, W. A. Nowlan, D. Robinson, and A. Wilson. 1992. User Centered Development of a General Practice Medical Workstation: The PEN&PAD Experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). ACM, New York, NY, USA, 447--453.

Digital Library

[21]

Felix Ritter, Christian Hansen, Kjen Wilkens, Alexander Köhn, and Heinz-Otto Peitgen. 2009. User Interfaces for Direct Interaction with 3D Planning Data in the Operating Room. i-com - Zeitschrift für interaktive und kooperative Medien 8, 1 (2009). https://www.researchgate.net/profile/Felix_ Ritter/publication/231188071_User_Interfaces_ for_Direct_Interaction_with_3D_Planning_Data_ in_the_Operating_Room/links/ 53f4a16b0cf2fceacc6e93f0.pdf

[22]

Patrick Saalfeld, Andre Mewes, Maria Luz, Bernhard Preim, and Christian Hansen. 2015. Comparative Evaluation of Gesture and Touch Input for Medical Software. In Proc. of Mensch und Computer. https://www.degruyter.com/downloadpdf/books/ 9783110443929/9783110443929-016/ 9783110443929-016.pdf

[23]

Neal E Seymour. 2008. VR to OR: a review of the evidence that virtual reality simulation improves operating room performance. World journal of surgery 32, 2 (2008), 182--188.

[24]

Neal E Seymour, Anthony G Gallagher, Sanziana A Roman, Michael K O'Brien, Vipin K Bansal, Dana K Andersen, and Richard M Satava. 2002. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Annals of surgery 236, 4 (2002), 458--464.

[25]

William R. Sherman and Alan B. Craig. 2003. Understanding virtual reality: Interface, application, and design. Kaufmann, Amsterdam u.a.

[26]

Édimo Sousa Silva and Maria Andréia Formico Rodrigues. 2014. Design and Evaluation of a Gesture-Controlled System for Interactive Manipulation of Medical Images and 3D Models. SBC Journal on Interactive Systems 5, 3 (2014), 53--65.

[27]

Helman I Stern, Juan P Wachs, and Yael Edan. 2008. Optimal consensus intuitive hand gesture vocabulary design. In Semantic Computing, 2008 IEEE International Conference on. IEEE, 96--103.

Digital Library

[28]

Ulrich von Zadow, Sandra Buron, Kai Sostmann, and Raimund Dachselt. 2013. The SimMed Experience: Medical Education on Interactive Tabletops. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, New York, NY, USA, 297--300.

Digital Library

[29]

Juan P Wachs, Helman I Stern, Yael Edan, Michael Gillam, Jon Handler, Craig Feied, and Mark Smith. 2008. A gesture-based tool for sterile browsing of radiology images. Journal of the American Medical Informatics Association 15, 3 (2008), 321--323.

Cited By

Riviere JMalo RVinet LPrié Y(2024)Using Micro-phenomenological Interviews To Collect And Compare Lived Experiences In Virtual Reality: A Review Of Two StudiesProceedings of the 35th Conference on l'Interaction Humain-Machine10.1145/3649792.3649795(1-14)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1145/3649792.3649795
Polenz LJoeres FHansen CHeinrich F(2024)Simulating projective Augmented Reality Visualizations in Virtual Reality: Is VR a feasible Environment for medical AR Evaluations?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650843(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650843
Rivière JVinet LPrié Y(2024)Towards the use of VR prototypes in architecture to collect user experiences: An assessment of the comparability of patient experiences in a virtual and a real ambulatory pathwayInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103342(103342)Online publication date: Aug-2024
https://doi.org/10.1016/j.ijhcs.2024.103342
Show More Cited By

Index Terms

Virtual Reality for User-Centered Design and Evaluation of Touch-free Interaction Techniques for Navigating Medical Images in the Operating Room
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Computer-based virtual reality simulator for phacoemulsification cataract surgery training

Recent research in virtual reality indicates that computer-based simulators are an effective technology to use for surgeons learning to improve their surgical skills in a controlled environment. This article presents the development of a virtual reality ...
Evaluation of properties and usability of virtual reality interaction techniques in craniomaxillofacial computer-assisted surgical simulation
OzCHI '23: Proceedings of the 35th Australian Computer-Human Interaction Conference

Although there are promising results from the use of virtual reality (VR) in the craniomaxillofacial field, there is still a need to validate the usability and properties of the VR environment and interaction techniques. The aim of this study was to ...
Evaluation of haptic virtual reality user interfaces for medical marking on 3D models
Highlights
- Two haptic VR interfaces are proposed to interact with volumetric medical images for improving current computer-aided medical diagnosis and planning.
- Compared with the traditional 2D interface using a screen-based 2D display and a ...
Abstract
Three-dimensional (3D) visualization has been widely used in computer-aided medical diagnosis and planning. To interact with 3D models, current user interfaces in medical systems mainly rely on the traditional 2D interaction techniques by ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

May 2017

3954 pages

ISBN:9781450346566

DOI:10.1145/3027063

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 May 2017

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI EA '17 Paper Acceptance Rate 1,000 of 5,000 submissions, 20%;

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
683
Total Downloads

Downloads (Last 12 months)53
Downloads (Last 6 weeks)12

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Riviere JMalo RVinet LPrié Y(2024)Using Micro-phenomenological Interviews To Collect And Compare Lived Experiences In Virtual Reality: A Review Of Two StudiesProceedings of the 35th Conference on l'Interaction Humain-Machine10.1145/3649792.3649795(1-14)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1145/3649792.3649795
Polenz LJoeres FHansen CHeinrich F(2024)Simulating projective Augmented Reality Visualizations in Virtual Reality: Is VR a feasible Environment for medical AR Evaluations?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650843(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650843
Rivière JVinet LPrié Y(2024)Towards the use of VR prototypes in architecture to collect user experiences: An assessment of the comparability of patient experiences in a virtual and a real ambulatory pathwayInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103342(103342)Online publication date: Aug-2024
https://doi.org/10.1016/j.ijhcs.2024.103342
Nilsson TRometsch FBecker LDufresne FDemedeiros PGuerra ECasini AVock AGaeremynck FCowley A(2023)Using Virtual Reality to Shape Humanity’s Return to the Moon: Key Takeaways from a Design StudyProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580718(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580718
Schott DHeinrich FStallmeister LMoritz JHensen BHansen C(2023)Is this the vReal Life? Manipulating Visual Fidelity of Immersive Environments for Medical Task Simulation2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00134(1171-1180)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00134
Graf LAltmeyer MEmmerich KHerrlich MKrekhov ASpiel K(2022)Development and Validation of a German Version of the Player Experience Inventory (PXI)Proceedings of Mensch und Computer 202210.1145/3543758.3543763(265-275)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3543763
Nilsson TRometsch FCasini AGuerra EBecker LTreuer Ade Medeiros PSchnellbaecher HVock ACowley A(2022)Using Virtual Reality to Design and Evaluate a Lunar Lander: The EL3 Case StudyExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519775(1-7)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519775
Muender TReinschluessel ASalzmann DLück TSchenk AWeyhe DDöring TMalaka R(2022)Evaluating Soft Organ-Shaped Tangibles for Medical Virtual RealityExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519715(1-8)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519715
Sadeghi Milani ACecil-Xavier AGupta ACecil JKennison S(2022)A Systematic Review of Human–Computer Interaction (HCI) Research in Medical and Other Engineering FieldsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.211653040:3(515-536)Online publication date: 14-Sep-2022
https://doi.org/10.1080/10447318.2022.2116530
Zhou XTeng FDu XLi JJin MXue C(2022)H-GOMS: a model for evaluating a virtual-hand interaction system in virtual environmentsVirtual Reality10.1007/s10055-022-00674-y27:2(497-522)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1007/s10055-022-00674-y
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents