research-article

Caring About the Human Operator: Haptic Shared Control for Enhanced User Comfort in Robotic Telemanipulation

Authors:

Giulia Matarese,

Marco Gabiccini,

Alessio Artoni,

Domenico Prattichizzo,

Paolo Robuffo Giordano,

Claudio PacchierottiAuthors Info & Claims

IEEE Transactions on Haptics, Volume 13, Issue 1

Pages 197 - 203

https://doi.org/10.1109/TOH.2020.2969662

Published: 01 January 2020 Publication History

Abstract

Haptic shared control enables a human operator and an autonomous controller to share the control of a robotic system using haptic active constraints. It has been used in robotic teleoperation for different purposes, such as navigating along paths minimizing the torques requested to the manipulator or avoiding possibly dangerous areas of the workspace. However, few works have focused on using these ideas to account for the user's comfort. In this article, we present an innovative haptic-enabled shared control approach aimed at minimizing the user's workload during a teleoperated manipulation task. Using an inverse kinematic model of the human arm and the rapid upper limb assessment (RULA) metric, the proposed approach estimates the current user's comfort online. From this measure and an a priori knowledge of the task, we then generate dynamic active constraints guiding the users towards a successful completion of the task, along directions that improve their posture and increase their comfort. Studies with human subjects show the effectiveness of the proposed approach, yielding a 30% perceived reduction of the workload with respect to using standard guided human-in-the-loop teleoperation.

References

[1]

D. J. Bruemmer, D. A. Few, R. L. Boring, J. L. Marble, M. C. Walton, and C. W. Nielsen, “Shared understanding for collaborative control,” IEEE Trans. Syst., Man, Cybern., A, Syst. Humans, vol. 35, no. 4, pp. 494–504, Jul. 2005.

Digital Library

[2]

B. Khademian and K. Hashtrudi-Zaad, “Dual-user teleoperation systems: New multilateral shared control architecture and kinesthetic performance measures,” IEEE/ASME Trans. Mechatronics, vol. 17, no. 5, pp. 895–906, Oct. 2012.

[3]

D. Gopinath, S. Jain, and B. D. Argall, “Human-in-the-loop optimization of shared autonomy in assistive robotics,” IEEE Robot. Autom. Lett., vol. 2, no. 1, pp. 247–254, Jan. 2017.

[4]

M. Selvaggio, F. Abi-Farraj, C. Pacchierotti, P. R. Giordano, and B. Siciliano, “Haptic-based shared-control methods for a dual-arm system,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 4249–4256, Oct. 2018.

[5]

V. Duchaine and C. M. Gosselin, “General model of human-robot cooperation using a novel velocity based variable impedance control,” in Proc. IEEE World Haptics, 2007, pp. 446–451.

[6]

H. Arai, T. Takubo, Y. Hayashibara, and K. Tanie, “Human-robot cooperative manipulation using a virtual nonholonomic constraint,” in Proc. IEEE Intl. Conf. Robot. Autom., 2000, pp. 4063–4069.

[7]

R. Rahal, F. Abi-Farraj, P. Giordano, and C. Pacchierotti, “Haptic shared-control methods for robotic cutting under nonholonomic constraints,” pp. 8151–8157, 2019.

[8]

C. Pacchierotti, et al., “Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance,” IEEE Trans. Autom. Sci. Eng., vol. 15, no. 1, pp. 290–306, Jan. 2018.

[9]

C. Passenberg, A. Glaser, and A. Peer, “Exploring the design space of haptic assistants: The assistance policy module,” IEEE Trans. Haptics, vol. 6, no. 4, pp. 440–452, Oct.–Dec. 2013.

Digital Library

[10]

D. A. Abbink, M. Mulder, and E. R. Boer, “Haptic shared control: smoothly shifting control authority?” Cognition, Technol. Work, vol. 14, no. 1, pp. 19–28, 2012.

Digital Library

[11]

M. Hong and J. W. Rozenblit, “A haptic guidance system for computer-assisted surgical training using virtual fixtures,” in Proc. IEEE Int. Conf. Syst., Man, Cybern., 2016, pp. 002 230–002 235.

[12]

A. M. Ghalamzan, F. Abi-Farraj, P. Robuffo Giordano, and R. Stolkin, “Human-in-the-loop optimisation: mixed initiative grasping for optimally facilitating post-grasp manipulative actions,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2017, pp. 3386–3393.

[13]

M. Mulder, D. A. Abbink, and E. R. Boer, “Sharing control with haptics: Seamless driver support from manual to automatic control,” Human Factors, vol. 54, no. 5, pp. 786–798, 2012.

[14]

C. Passenberg, R. Groten, A. Peer, and M. Buss, “Towards real-time haptic assistance adaptation optimizing task performance and human effort,” in Proc. IEEE World Haptics Conf., 2011, pp. 155–160.

[15]

S. A. Bowyer, B. L. Davies, and F. Rodriguez y Baena, “Active constraints/virtual fixtures: A survey,” IEEE Trans. Robot., vol. 30, no. 1, pp. 138–157, Feb. 2014.

Digital Library

[16]

M. Abayazid, C. Pacchierotti, P. Moreira, R. Alterovitz, D. Prattichizzo, and S. Misra, “Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering,” Int. J. Med. Robot. Comput. Assisted Surgery, vol. 12, no. 2, pp. 219–230, 2016.

[17]

J. Bimbo, C. Pacchierotti, M. Aggravi, N. Tsagarakis, and D. Prattichizzo, “Teleoperation in cluttered environments using wearable haptic feedback,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2017, pp. 3401–3408.

[18]

L. Meli, C. Pacchierotti, and D. Prattichizzo, “Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation,” Int. J. Med. Robot. Comput. Assisted Surgery, vol. 13, pp. 1809–1809, 2017.

[19]

C. Pacchierotti, Cutaneous Haptic Feedback in Robotic Teleoperation. Berlin, Germany: Springer, 2015.

[20]

F. Chinello, C. Pacchierotti, J. Bimbo, N. G. Tsagarakis, and D. Prattichizzo, “Design and evaluation of a wearable skin stretch device for haptic guidance,” IEEE Robot. Autom. Lett., vol. 3, no. 1, pp. 524–531, Jan. 2018.

[21]

Y. Li, V. Patoglu, and M. K. O'Malley, “Negative efficacy of fixed gain error reducing shared control for training in virtual environments,” ACM Trans. Appl. Perception, vol. 6, no. 1, pp. 3:1–3:21, 2009.

[22]

L. Marchal-Crespo, M. Bannwart, R. Riener, and H. Vallery, “The effect of haptic guidance on learning a hybrid rhythmic-discrete motor task,” IEEE Trans. Haptics, vol. 8, no. 2, pp. 222–234, Apr.–Jun. 2015.

Digital Library

[23]

B. Busch, G. Maeda, Y. Mollard, M. Demangeat, and M. Lopes, “Postural optimization for an ergonomic human-robot interaction,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2017, pp. 2778–2785.

[24]

A. G. Marin, M. S. Shourijeh, P. E. Galibarov, M. Damsgaard, L. Fritzsch, and F. Stulp, “Optimizing contextual ergonomics models in human-robot interaction,” in Proc. IEEE/RSJ Int. Conf. Intel. Robots Syst., 2018, pp. 1–9.

[25]

L. Chen, L. F. Figueredo, and M. R. Dogar, “Planning for muscular and peripersonal-space comfort during human-robot forceful collaboration,” in Proc. IEEE Int. Conf. Humanoid Robots, 2018, pp. 1–8.

[26]

L. Peternel, W. Kim, J. Babi, and A. Ajoudani, “Towards ergonomic control of human-robot co-manipulation and handover,” in Proc. IEEE-RAS Int. Conf. Humanoid Robot., 2017, pp. 55–60.

[27]

L. McAtamney and E. N. Corlett, “Rula: A survey method for the investigation of work-related upper limb disorders,” Appl. Ergonom., vol. 24, no. 2, pp. 91–99, 1993.

[28]

F. Chaumette and S. Hutchinson, “Visual servo control, Part I: Basic approaches,” IEEE Robot. Autom. Mag., vol. 13, no. 4, pp. 82–90, Dec. 2006.

[29]

M. Shimizu, W.-K. Yoon, and K. Kitagaki, “A practical redundancy resolution for 7 DoF redundant manipulators with joint limits,” in Proc. IEEE Int. Conf. Robot. Autom., 2007, pp. 4510–4516.

[30]

H. Seraji, “Configuration control of redundant manipulators: Theory and implementation,” IEEE Trans. Robot. Autom., vol. 5, no. 4, pp. 472–490, Aug. 1989.

[31]

M. Kang, H. Shin, D. Kim, and S.-E. Yoon, “Torm: Collision-free trajectory optimization of redundant manipulator given an end-effector path,” 2019, arXiv:1909.12517.

[32]

F. Abi-Farraj, C. Pacchierotti, O. Arenz, G. Neumann, and P. R. Giordano, “A haptic shared-control architecture for guided multi-target robotic grasping,” IEEE Trans. Haptics, to be published.

Digital Library

[33]

C. Pacchierotti, D. Prattichizzo, and K. J. Kuchenbecker, “Displaying sensed tactile cues with a fingertip haptic device,” IEEE Trans. Haptics, vol. 8, no. 4, pp. 384–396, Oct.–Dec. 2015.

Digital Library

[34]

S. G. Hart and L. E. Staveland, “Development of nasa-tlx (task load index): Results of empirical and theoretical research,” in Advances in psychology. Amsterdam, The Netherlands: Elsevier, 1988, vol. 52, pp. 139–183.

[35]

C. Pacchierotti, S. Sinclair, M. Solazzi, A. Frisoli, V. Hayward, and D. Prattichizzo, “Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives,” IEEE Trans. Haptics, vol. 10, no. 4, pp. 580–600, Oct–Dec. 2017.

Digital Library

[36]

D. Michel, A. Qammaz, and A. A. Argyros, “Markerless 3d human pose estimation and tracking based on rgbd cameras: an experimental evaluation,” in Proc. Int. Conf. Pervasive Technologies Related Assistive Environ., 2017, pp. 115–122.

[37]

E. Biryukova, A. Roby-Brami, A. Frolov, and M. Mokhtari, “Kinematics of human arm reconstructed from spatial tracking system recordings,” J. Biomechanics, vol. 33, no. 8, pp. 985–995, 2000.

Cited By

Darvish KPenco LRamos JCisneros RPratt JYoshida EIvaldi SPucci D(2023)Teleoperation of Humanoid Robots: A SurveyIEEE Transactions on Robotics10.1109/TRO.2023.323695239:3(1706-1727)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TRO.2023.3236952
Li GLi QYang CSu YYuan ZWu X(2023)The Classification and New Trends of Shared Control Strategies in Telerobotic Systems: A SurveyIEEE Transactions on Haptics10.1109/TOH.2023.325385616:2(118-133)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TOH.2023.3253856
Rahal RGhalamzan-E. AAbi-Farraj FPacchierotti CRobuffo Giordano P(2023)Haptic-guided grasping to minimise torque effort during robotic telemanipulationAutonomous Robots10.1007/s10514-023-10096-747:4(405-423)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1007/s10514-023-10096-7
Show More Cited By

Index Terms

Caring About the Human Operator: Haptic Shared Control for Enhanced User Comfort in Robotic Telemanipulation

Index terms have been assigned to the content through auto-classification.

Recommendations

User Evaluation of a Haptic-Enabled Shared-Control Approach for Robotic Telemanipulation
2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Robotic telemanipulators are already widely used in nuclear decommissioning sites for handling radioactive waste. However, currently employed systems are still extremely primitive, making the handling of these materials prohibitively slow and ineffective. ...
A haptic shared control algorithm for flexible human assistance to semi-autonomous robots
2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Autonomous as well as teleoperated robots find wide applications in various environments. Their capability to accomplish complex and dynamic operations can be significantly improved by fusing human intelligence with autonomous algorithms. In this paper, ...
Haptic Feedback Improves Human-Robot Agreement and User Satisfaction in Shared-Autonomy Teleoperation
2021 IEEE International Conference on Robotics and Automation (ICRA)
Shared autonomy teleoperation can guarantee safety, but does so by reducing the human operator’s control authority, which can lead to reduced levels of human-robot agreement and user satisfaction. This paper presents a novel haptic shared autonomy ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Haptics

IEEE Transactions on Haptics Volume 13, Issue 1

Jan.-March 2020

255 pages

ISSN:1939-1412

Issue’s Table of Contents

1939-1412 © 2020 Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 January 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Darvish KPenco LRamos JCisneros RPratt JYoshida EIvaldi SPucci D(2023)Teleoperation of Humanoid Robots: A SurveyIEEE Transactions on Robotics10.1109/TRO.2023.323695239:3(1706-1727)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TRO.2023.3236952
Li GLi QYang CSu YYuan ZWu X(2023)The Classification and New Trends of Shared Control Strategies in Telerobotic Systems: A SurveyIEEE Transactions on Haptics10.1109/TOH.2023.325385616:2(118-133)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TOH.2023.3253856
Rahal RGhalamzan-E. AAbi-Farraj FPacchierotti CRobuffo Giordano P(2023)Haptic-guided grasping to minimise torque effort during robotic telemanipulationAutonomous Robots10.1007/s10514-023-10096-747:4(405-423)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1007/s10514-023-10096-7
Kourtesis PArgelaguet FVizcay SMarchal MPacchierotti C(2022)Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future DirectionsIEEE Transactions on Haptics10.1109/TOH.2022.318986615:3(479-496)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1109/TOH.2022.3189866
Lin TKrishnan ALi Z(2022)Comparison of Haptic and Augmented Reality Visual Cues for Assisting Tele- manipulation2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9811669(9309-9316)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1109/ICRA46639.2022.9811669
Sakagami NSuka MKimura YSato EWada T(2022)Haptic shared control applied for ROV operation support in flowing waterArtificial Life and Robotics10.1007/s10015-022-00788-127:4(867-875)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s10015-022-00788-1
Yazdani ASabbagh Novin RBethel CPaiva ABroadbent EFeil-Seifer DSzafir D(2021)Posture Estimation and Optimization in Ergonomically Intelligent Teleoperation SystemsCompanion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3434074.3446350(604-606)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3434074.3446350
Yazdani ANovin RMerryweather AHermans T(2021)Is the Leader Robot an Adequate Sensor for Posture Estimation and Ergonomic Assessment of A Human Teleoperator?2021 IEEE 17th International Conference on Automation Science and Engineering (CASE)10.1109/CASE49439.2021.9551471(1946-1952)Online publication date: 23-Aug-2021
https://dl.acm.org/doi/10.1109/CASE49439.2021.9551471

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents