Multi-Modal Multi sensor Interaction between Human andHeterogeneous Multi-Robot System
Author: 
S M Al MahiAuthors Info & Claims
S M Al MahiAuthors Info & ClaimsPages 524 - 528
Abstract
I introduce a novel multi-modal multi-sensor interaction method between humans and heterogeneous multi-robot systems. I have also developed a novel algorithm to control heterogeneous multi-robot systems. The proposed algorithm allows the human operator to provide intentional cues and information to a multi-robot system using a multimodal multi-sensor touchscreen interface. My proposed method can effectively convey complex human intention to multiple robots as well as represent robots' intentions over the spatiotemporal domain. The proposed method is scalable and robust to dynamic change in the deployment configuration. I describe the implementation of the control algorithm used to control multiple quad-rotor unmanned aerial vehicles in simulated and real environments. I will also present my initial work on human interaction with the robots running my algorithm using mobile phone touch screens and other potential multimodal interactions.
References
[1]
Sanjay K. Boddhu, Matt McCartney, Oliver Ceccopieri, and Robert L. Williams. 2013. A collaborative smartphone sensing platform for detecting and tracking hostile drones, Tien Pham, Michael A. Kolodny, and Kevin L. Priddy (Eds.), Vol. 8742. International Society for Optics and Photonics, 874211.
[2]
Jesse Butterfield, Odest Chadwicke Jenkins, and Brian Gerkey. 2008. Multi-robot Markov random fields. In Autonomous Agents and Multi-Agent Systems (AAMAS).
[3]
Jessica R. Cauchard, Jane L E Kevin, Y Zhai, and James A. Landay. 2015. Drone & Me: An Exploration Into Natural Human-Drone Interaction. UbiComp '15 (2015), 361--365. arxiv: arXiv:1011.1669v3
[4]
Meghan Chandarana, Erica L. Meszaros, Anna Trujillo, and B. Danette Allen. 2018. Challenges of Using Gestures in Multimodal HMI for Unmanned Mission Planning. 175--182.
[5]
Christopher Crick, Graylin Jay, Sarah Osentoski, Benjamin Pitzer, and Odest Chadwicke Jenkins. 2011. Rosbridge: ROS for non-ROS users. In Proceedings of the 15th International Symposium on Robotics Research .
[6]
Danielle Cummings, Stephane Fymat, and Tracy Hammond. 2012. Sketch-based Interface for Interaction with Unmanned Air Vehicles Work-in-Progress. (2012). http://delivery.acm.org/10.1145/2230000/2223664/p1511-cummings.pdf
[7]
D Cummmings, S Fymat, and Tracy Hammond. 2012. RedDog: A Smart Sketch Interface for Autonomous Aerial Systems., pages21--28 pages.
[8]
Ramon A. Suarez Fernandez, Jose Luis Sanchez-Lopez, Carlos Sampedro, Hriday Bavle, Martin Molina, and Pascual Campoy. 2016. Natural user interfaces for human-drone multi-modal interaction. In 2016 Int. Conf. Unmanned Aircr. Syst. IEEE, 1013--1022.
[9]
Seng Keat Gan, Robert Fitch, and Salah Sukkarieh. 2012. Real-time decentralized search with inter-agent collision avoidance. In International Conference on Robotics and Automation (ICRA).
[10]
Jeongwoon Kim, David Hyunchul Shim, and James R Morrison. 2014. Tablet PC-based visual target-following system for quadrotors . J. Intell. Robot. Syst. Theory Appl., Vol. 74, 1--2 (2014), 85--95.
[11]
Pablo Lanillos, Seng Keat Gan, Eva Besada-Portas, Gonzalo Pajares, and Salah Sukkarieh. 2014. Multi-UAV target search using decentralized gradient-based negotiation with expected observation. Information Sciences, Vol. 282 (2014), 92--110.
[12]
Tayyab Naseer, Jurgen Sturm, and Daniel Cremers. 2013. FollowMe: Person following and gesture recognition with a quadrocopter. In IEEE Int. Conf. Intell. Robot. Syst. IEEE, 624--630.
[13]
Shayegan Omidshafiei, Christopher Amato, Miao Liu, Michael Everett, Jonathan P. How, and John Vian. 2017. Scalable accelerated decentralized multi-robot policy search in continuous observation spaces. In International Conference on Robotics and Automation (ICRA).
[14]
Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. 2009. ROS: an open-source Robot Operating System. In ICRA Workshop on Open Source Software.
[15]
Alessandro Renzaglia, Christophe Reymann, and Simon Lacroix. 2016. Monitoring the evolution of clouds with UAVs. In International Conference on Robotics and Automation (ICRA).
[16]
Christophe Reymann, Alessandro Renzaglia, Faycc al Lamraoui, Murat Bronz, and Simon Lacroix. 2017. Adaptive sampling of cumulus clouds with UAVs. Autonomous Robots (2017), 1--22.
[17]
Andrea Sanna, Fabrizio Lamberti, Gianluca Paravati, and Federico Manuri. 2013. A Kinect-based natural interface for quadrotor control. Entertain. Comput., Vol. 4, 3 (aug 2013), 179--186.
[18]
Ryan K Williams and Gaurav S Sukhatme. 2011. Cooperative multi-agent inference over grid structure Markov random fields. In Intelligent Robots and Systems (IROS).
Index Terms
- Multi-Modal Multi sensor Interaction between Human andHeterogeneous Multi-Robot System
Recommendations
Lexical Entrainment in Multi-party Human–Robot Interaction
Social RoboticsAbstractThis paper reports lexical entrainment in a multi-party human–robot interaction, wherein one robot and two humans serve as participants. Humans tend to use the same terms as their interlocutors while making conversation. This phenomenon is called ...
Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot
In this paper, we present our work in building technologies for natural multimodal human-robot interaction. We present our systems for spontaneous speech recognition, multimodal dialogue processing, and visual perception of a user, which includes ...
Joint Behavioural Control of Autonomous Multi-Robot Systems for Lead-Follower Formation to Improve Human-Robot Interaction
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsThe effective autonomous cooperation between unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) has led to an increase in research and development for improving human robot interaction, due to their high potential for achieving ...
Comments
Information & Contributors
Information
Published In
October 2018
687 pages
ISBN:9781450356923
DOI:10.1145/3242969
- General Chairs:
- Sidney K. D'Mello,
- Panayiotis (Panos) Georgiou,
- Stefan Scherer,
- Program Chairs:
- Emily Mower Provost,
- Mohammad Soleymani,
- Marcelo Worsley
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGCHI: Specialist Interest Group in Computer-Human Interaction of the ACM
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 October 2018
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
ICMI '18
Sponsor:
- SIGCHI
Acceptance Rates
ICMI '18 Paper Acceptance Rate 63 of 149 submissions, 42%;
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 355Total Downloads
- Downloads (Last 12 months)85
- Downloads (Last 6 weeks)11
Reflects downloads up to 28 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in