research-article

Authorable augmented reality instructions for assistance and training in work environments

Authors:

Jonas Blattgerste,

Patrick Renner,

Thies PfeifferAuthors Info & Claims

MUM '19: Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia

Article No.: 34, Pages 1 - 11

https://doi.org/10.1145/3365610.3365646

Published: 26 November 2019 Publication History

Abstract

Augmented Reality (AR) is a promising technology for assistance and training in work environments, as it can provide instructions and feedback contextualised. Not only, but especially impaired workers can benefit from this technology. While previous work mostly focused on using AR to assist or train specific predefined tasks, "general purpose" AR applications, that can be used to intuitively author new tasks at run-time, are widely missing.

The contribution of this work is twofold: First we develop an AR authoring tool on the Microsoft HoloLens in combination with a Smartphone as an additional controller following considerations based on related work, guidelines and focus group interviews. Then, we evaluate the usability of the authoring tool itself and the produced AR instructions on a qualitative level in realistic scenarios and gather feedback. As the results reveal a positive reception, we discuss authorable AR as a viable form of AR assistance or training in work environments.

References

[1]

Maneesh Agrawala, Doantam Phan, Julie Heiser, John Haymaker, Jeff Klingner, Pat Hanrahan, and Barbara Tversky. 2003. Designing effective step-by-step assembly instructions. In ACM Transactions on Graphics (TOG), Vol. 22. ACM, 828--837.

Digital Library

[2]

Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies 4, 3 (2009), 114--123.

Digital Library

[3]

Bruce Lawrence Berg, Howard Lune, and Howard Lune. 2004. Qualitative research methods for the social sciences. Vol. 5. Pearson Boston, MA.

[4]

Jonas Blattgerste, Patrick Renner, and Thies Pfeiffer. 2018. Advantages of Eye-gaze over Head-gaze-based Selection in Virtual and Augmented Reality Under Varying Field of Views. In Proceedings of the Workshop on Communication by Gaze Interaction (COGAIN '18). ACM, New York, NY, USA, Article 1, 9 pages.

Digital Library

[5]

Jonas Blattgerste, Patrick Renner, and Thies Pfeiffer. 2019. Augmented Reality Action Assistance and Learning for Cognitively Impaired People. A Systematic Literature Review. In The 12th PErvasive Technologies Related to Assistive Environments Conference (PETRAâă&Zacute;19).

[6]

Jonas Blattgerste, Patrick Renner, Benjamin Strenge, and Thies Pfeiffer. 2018. In-Situ Instructions Exceed Side-by-Side Instructions in Augmented Reality Assisted Assembly. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference (PETRA '18). ACM, New York, NY, USA, 133--140.

Digital Library

[7]

Jonas Blattgerste, Benjamin Strenge, Patrick Renner, Thies Pfeiffer, and Kai Essig. 2017. Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17). ACM, New York, NY, USA, 75--82.

Digital Library

[8]

John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[9]

Philip Burnard. 1991. A method of analysing interview transcripts in qualitative research. Nurse education today 11, 6 (1991), 461--466.

[10]

Sebastian Büttner, Markus Funk, Oliver Sand, and Carsten Röcker. 2016. Using head-mounted displays and in-situ projection for assistive systems: A comparison. In Proceedings of the 9th ACM international conference on pervasive technologies related to assistive environments. ACM, 44.

Digital Library

[11]

Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Swan. 2018. A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014. Frontiers in Robotics and AI 5 (2018), 37.

[12]

Kai Essig, Benjamin Strenge, and Thomas Schack. 2016. ADAMAAS: towards smart glasses for mobile and personalized action assistance. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. ACM, 46.

Digital Library

[13]

Markus Funk, Thomas Kosch, Romina Kettner, Oliver Korn, and Albrecht Schmidt. 2016. motioneap: An overview of 4 years of combining industrial assembly with augmented reality for industry 4.0. In Proceedings of the 16th international conference on knowledge technologies and datadriven business.

[14]

Markus Funk, Lars Lischke, Sven Mayer, Alireza Sahami Shirazi, and Albrecht Schmidt. 2018. Teach Me How! Interactive Assembly Instructions Using Demonstration and In-Situ Projection. In Assistive Augmentation. Springer, 49--73.

[15]

Rebecca A Grier. 2015. How high is high? A meta-analysis of NASA-TLX global workload scores. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 59. SAGE Publications Sage CA: Los Angeles, CA, 1727--1731.

[16]

Sinem Guven, Steven Feiner, and Ohan Oda. 2006. Mobile augmented reality interaction techniques for authoring situated media on-site. In 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 235--236.

Digital Library

[17]

Tracey E Hall, Charles A Hughes, and Melinda Filbert. 2000. Computer assisted instruction in reading for students with learning disabilities: A research synthesis. Education and Treatment of Children (2000), 173--193.

[18]

Matthias Haringer and Holger T Regenbrecht. 2002. A pragmatic approach to augmented reality authoring. In Proceedings. International Symposium on Mixed and Augmented Reality. IEEE, 237--245.

[19]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.

[20]

Wolfgang Hürst and Casper Van Wezel. 2013. Gesture-based interaction via finger tracking for mobile augmented reality. Multimedia Tools and Applications 62, 1 (2013), 233--258.

[21]

Oliver Korn. 2012. Industrial playgrounds: how gamification helps to enrich work for elderly or impaired persons in production. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems. ACM, 313--316.

Digital Library

[22]

Tobias Langlotz, Stefan Mooslechner, Stefanie Zollmann, Claus Degendorfer, Gerhard Reitmayr, and Dieter Schmalstieg. 2012. Sketching up the world: in situ authoring for mobile Augmented Reality. Personal and ubiquitous computing 16, 6 (2012), 623--630.

[23]

Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and evaluation of a user experience questionnaire. In Symposium of the Austrian HCI and Usability Engineering Group. Springer, 63--76.

Digital Library

[24]

Gun A Lee, Claudia Nelles, Mark Billinghurst, and Gerard Jounghyun Kim. 2004. Immersive authoring of tangible augmented reality applications. In Proceedings of the 3rd IEEE/ACM international Symposium on Mixed and Augmented Reality. IEEE Computer Society, 172--181.

Digital Library

[25]

Kangdon Lee. 2012. Augmented reality in education and training. TechTrends 56, 2 (2012), 13--21.

[26]

Bibeg Hang Limbu, Halszka Jarodzka, Roland Klemke, and Marcus Specht. 2018. Using sensors and augmented reality to train apprentices using recorded expert performance: A systematic literature review. Educational Research Review (2018).

[27]

Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proceedings of the 17th annual ACM symposium on User interface software and technology. ACM, 197--206.

Digital Library

[28]

David L Morgan. 1998. Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative health research 8, 3 (1998), 362--376.

[29]

Siti Soleha Muhammad Nizam, Rimaniza Zainal Abidin, Nurhazarifah Che Hashim, Meng Chun Lam, Haslina Arshad, and Nazatul Aini Abd Majid. 2018. A Review of Multimodal Interaction Technique in Augmented Reality Environment. International Journal on Advanced Science, Engineering and Information Technology 8, 4--2 (2018), 1460--1469.

[30]

Riccardo Palmarini, John Ahmet Erkoyuncu, Rajkumar Roy, and Hosein Torabmostaedi. 2018. A systematic review of augmented reality applications in maintenance. Robotics and Computer-Integrated Manufacturing 49 (2018), 215--228.

[31]

Wayne Piekarski and Bruce H. Thomas. 2002. The Tinmith System: Demonstrating New Techniques for Mobile Augmented Reality Modelling. Aust. Comput. Sci. Commun. 24, 4 (Jan. 2002), 61--70. http://dl.acm.org/citation.cfm?id=563997.563994

[32]

Ivan Poupyrev, Desney S Tan, Mark Billinghurst, Hirokazu Kato, Holger Regenbrecht, and Nobuji Tetsutani. 2001. Tiles: A Mixed Reality Authoring Interface. In Interact, Vol. 1. 334--341.

[33]

Eduardo Quintana and Jesus Favela. 2013. Augmented reality annotations to assist persons with Alzheimers and their caregivers. Personal and ubiquitous computing 17, 6 (2013), 1105--1116.

[34]

Patrick Renner and Thies Pfeiffer. 2017. Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 186--194.

[35]

Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2017. Construction of a Benchmark for the User Experience Questionnaire (UEQ). IJIMAI 4, 4 (2017), 40--44.

[36]

Matthias Schröder and Helge Ritter. 2017. Deep Learning for Action Recognition in Augmented Reality Assistance Systems. In ACM SIGGRAPH 2017 Posters (SIGGRAPH '17). ACM, New York, NY, USA, Article 75, 2 pages.

Digital Library

[37]

Hartmut Seichter, Julian Looser, and Mark Billinghurst. 2008. ComposAR: An intuitive tool for authoring AR applications. In Proceedings of the 7th IEEE/ACM international symposium on mixed and augmented reality. IEEE Computer Society, 177--178.

Digital Library

[38]

Cledja Karina Rolim da SILVA. 2016. Design guidelines for generating augmented reality instructions. (2016).

[39]

Benjamin Strenge, Ludwig Vogel, and Thomas Schack. 2018. Individualized cognitive assistance by smart glasses for manual assembly processes in industry. Technische Unterstützungssysteme, die die Menschen wirklich wollen (2018), 399.

[40]

Thomas S Tullis and Jacqueline N Stetson. 2004. A comparison of questionnaires for assessing website usability. In Usability professional association conference, Vol. 1. Minneapolis, USA.

[41]

Daniel W Turner III. 2010. Qualitative interview design: A practical guide for novice investigators. The qualitative report 15, 3 (2010), 754--760.

[42]

Xin Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4, 1 (2016), 1--22.

[43]

Stefan Werrlich, Austino Daniel, Alexandra Ginger, Phuc-Anh Nguyen, and Gunther Notni. 2018. Comparing HMD-based and Paper-based Training. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 134--142.

[44]

Stefan Werrlich, Phuc-Anh Nguyen, and Gunther Notni. 2018. Evaluating the training transfer of Head-Mounted Display based training for assembly tasks. In Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference. ACM, 297--302.

Digital Library

[45]

Stefan Werrlich, Kai Nitsche, and Gunther Notni. 2017. Demand Analysis for an Augmented Reality Based Assembly Training. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17). ACM, New York, NY, USA, 416--422.

Digital Library

[46]

Dennis Wolf, Daniel Besserer, Karolina Sejunaite, Matthias Riepe, and Enrico Rukzio. 2018. cARe: An Augmented Reality Support System for Dementia Patients. In The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings. ACM, 42--44.

Digital Library

[47]

Dennis Wolf, John J Dudley, and Per Ola Kristensson. 2018. Performance Envelopes of In-Air Direct and Smartwatch Indirect Control for Head-Mounted Augmented Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 347--354.

[48]

Jürgen Zauner, Michael Haller, Alexander Brandl, and Werner Hartmann. 2003. Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures. In Proceedings of the 2Nd IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '03). IEEE Computer Society, Washington, DC, USA, 237-. http://dl.acm.org/citation.cfm?id=946248.946783

Digital Library

[49]

J Zhu, SK Ong, and AYC Nee. 2013. An authorable context-aware augmented reality system to assist the maintenance technicians. The International Journal of Advanced Manufacturing Technology 66, 9--12 (2013), 1699--1714.

Cited By

Ghasemi YBayro AMacDonald JJeong HReynolds JNam C(2024)Embedding Spatial Augmented Reality in Culinary Training: A Comparative Evaluation of sAR Kitchen and Video TutorialsIEEE Transactions on Learning Technologies10.1109/TLT.2023.333478817(765-775)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2023.3334788
Laviola EUva AGattullo M(2024)The minimal AR authoring approachComputers in Industry10.1016/j.compind.2023.104026154:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compind.2023.104026
Blattgerste JBehrends JPfeiffer T(2023)TrainAR: An Open-Source Visual Scripting-Based Authoring Tool for Procedural Mobile Augmented Reality TrainingsInformation10.3390/info1404021914:4(219)Online publication date: 3-Apr-2023
https://doi.org/10.3390/info14040219
Show More Cited By

Index Terms

Authorable augmented reality instructions for assistance and training in work environments
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Augmented reality action assistance and learning for cognitively impaired people: a systematic literature review
PETRA '19: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

Augmented reality (AR) is a promising tool for many situations in which assistance is needed, as it allows for instructions and feedback to be contextualized. While research and development in this area have been primarily driven by industry, AR could ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MUM '19: Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia

November 2019

462 pages

ISBN:9781450376242

DOI:10.1145/3365610

General Chair:
Fabio Paternò
CNR-ISTI, Italy
,
Program Chairs:
Giulio Jacucci
University of Helsinki, Finland
,
Michael Rohs
University of Hannover, Germany
,
Carmen Santoro
CNR-ISTI, Italy

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Deutsche Forschungsgemeinschaft

Conference

MUM 2019

MUM 2019: 18th International Conference on Mobile and Ubiquitous Multimedia

November 26 - 29, 2019

Pisa, Italy

Acceptance Rates

Overall Acceptance Rate 190 of 465 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
567
Total Downloads

Downloads (Last 12 months)77
Downloads (Last 6 weeks)5

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ghasemi YBayro AMacDonald JJeong HReynolds JNam C(2024)Embedding Spatial Augmented Reality in Culinary Training: A Comparative Evaluation of sAR Kitchen and Video TutorialsIEEE Transactions on Learning Technologies10.1109/TLT.2023.333478817(765-775)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2023.3334788
Laviola EUva AGattullo M(2024)The minimal AR authoring approachComputers in Industry10.1016/j.compind.2023.104026154:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compind.2023.104026
Blattgerste JBehrends JPfeiffer T(2023)TrainAR: An Open-Source Visual Scripting-Based Authoring Tool for Procedural Mobile Augmented Reality TrainingsInformation10.3390/info1404021914:4(219)Online publication date: 3-Apr-2023
https://doi.org/10.3390/info14040219
Ahmad AElaklouk AEdris ISalleh D(2023)Educator-oriented Authoring Tools to Develop Rich Educational Media: A Systematic Review2023 6th International Conference on Applied Computational Intelligence in Information Systems (ACIIS)10.1109/ACIIS59385.2023.10367320(1-6)Online publication date: 23-Oct-2023
https://doi.org/10.1109/ACIIS59385.2023.10367320
Tainaka KFujimoto YSawabe TKanbara MKato H(2023)Selection framework of visualization methods in designing AR industrial task-support systemsComputers in Industry10.1016/j.compind.2022.103828145(103828)Online publication date: Feb-2023
https://doi.org/10.1016/j.compind.2022.103828
Fleiner C(2022)Leveraging Human-Agent Collaboration for Multimodal Task Guidance with Concurrent Authoring CapabilitiesCompanion Proceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490100.3516477(138-142)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490100.3516477
Santos JNunes MPires MRocha JSousa NAdao TMagalhaes LJesus CSousa RLima RReis AOliveira E(2022)Generic XR game-based approach for industrial training2022 International Conference on Graphics and Interaction (ICGI)10.1109/ICGI57174.2022.9990346(1-8)Online publication date: 3-Nov-2022
https://doi.org/10.1109/ICGI57174.2022.9990346
Blattgerste JLuksch KLewa CPfeiffer T(2021)TrainAR: A Scalable Interaction Concept and Didactic Framework for Procedural Trainings Using Handheld Augmented RealityMultimodal Technologies and Interaction10.3390/mti50700305:7(30)Online publication date: 22-Jun-2021
https://doi.org/10.3390/mti5070030
Laviola EGattullo MManghisi VFiorentino MUva A(2021)Minimal AR: visual asset optimization for the authoring of augmented reality work instructions in manufacturingThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-08449-6Online publication date: 30-Nov-2021
https://doi.org/10.1007/s00170-021-08449-6
Uletika NHartono BWijayanto T(2020)Gamification in Assembly Training: A Systematic Review2020 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)10.1109/IEEM45057.2020.9309791(1073-1077)Online publication date: 14-Dec-2020
https://doi.org/10.1109/IEEM45057.2020.9309791
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents