skip to main content
10.1145/3613904.3642345acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Dances with Drones: Spatial Matching and Perceived Agency in Improvised Movements with Drone and Human Partners

Published: 11 May 2024 Publication History

Abstract

As drones become interwoven in human activities, increasingly taking on tasks interpreted as creative and performative, such as choreographed light shows, there is emerging interest in understanding how drones and humans can perform together. Humans have different habits when performing with partners as opposed to solo. How do people adapt their behaviors and perspectives when improvising with robotic partners? To explore these questions, we conducted a study investigating dancer-drone interactions using a system of micro aerial vehicles designed to facilitate improvised solo and partnered dances. Through solo and tandem dances with one or two robots, we analyzed the performers’ perceived workflow from semi-structured interviews and quantified their movement patterns during the improvisation. We found that the dancers perceived drone movements through spatial metaphors like the ceiling and gravity, anthropomorphizing drones as props on a stage through position and generated sound. The dancers felt a greater connection in single-drone scenarios and showed heightened avoidance behavior in two-drone situations. Our work shows how a robotic system can energize human dancers to improvise individually and in pairs.

Supplemental Material

MP4 File - Video Preview
Video Preview
Transcript for: Video Preview
MP4 File - Video Presentation
Video Presentation
Transcript for: Video Presentation
MP4 File - Video Figure
this is the video for the paper chi24-453, containing the main contribution and how the dancers dance with the drones
ZIP File - Dataset
This supplemental material contains the codes for computer vision methods, trajectory, and control for the drones used in this paper. It also includes the interview materials for the 12 dancers in this research. # Computer Vision We use the code of a) and b) directly to get the position of the dancer and the spatial location in 3D. a) 3d\_position: https://github.com/Garfield-kh/PoseTriplet b) 2d\_pose: https://github.com/open-mmlab/mmpose We fine-tune a model to detect the position of the drones c) drone\_detection: https://github.com/facebookresearch/detectron2 The training and inferring scripts are provided in the detection folder. # Drone Control and Trajectory We use the open-source platform Crazyflie 2.1 as the drones in this work d) https://github.com/bitcraze the control code are provided as: \drone_control\dance_with_drone.py # Dancers' interview We also organized the interview material (strictly anonymous) in the file interview.pdf

References

[1]
Julie Akerly. 2015. Embodied flow in experiential media systems: a study of the dancer’s lived experience in a responsive audio system. In Proceedings of the 2nd International Workshop on Movement and Computing. 9–16.
[2]
Sarah Fdili Alaoui and Jean-Marc Matos. 2021. RCO: Investigating social and technological constraints through interactive dance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
[3]
Angelos Angelopoulos, Austin Hale, Husam Shaik, Akshay Paruchuri, Ken Liu, Randal Tuggle, and Daniel Szafir. 2022. Drone brush: Mixed reality drone path planning. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 678–682.
[4]
Andreas Aristidou, Efstathios Stavrakis, Panayiotis Charalambous, Yiorgos Chrysanthou, and Stephania Loizidou Himona. 2015. Folk dance evaluation using laban movement analysis. Journal on Computing and Cultural Heritage (JOCCH) 8, 4 (2015), 1–19.
[5]
Alisha Bevins and Brittany A Duncan. 2021. Aerial flight paths for communication: How participants perceive and intend to respond to drone movements. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. 16–23.
[6]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77–101.
[7]
Anke M Brock, Julia Chatain, Michelle Park, Tommy Fang, Martin Hachet, James A Landay, and Jessica R Cauchard. 2018. Flymap: Interacting with maps projected from a drone. In Proceedings of the 7th ACM International Symposium on Pervasive Displays. 1–9.
[8]
Yves Candau, Jules Françoise, Sarah Fdili Alaoui, and Thecla Schiphorst. 2017. Cultivating kinaesthetic awareness through interaction: Perspectives from somatic practices and embodied cognition. In Proceedings of the 4th International Conference on Movement Computing. 1–8.
[9]
Jessica R Cauchard, Jane L E, Kevin Y Zhai, and James A Landay. 2015. Drone & me: an exploration into natural human-drone interaction. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. 361–365.
[10]
Jessica R Cauchard, Kevin Y Zhai, Marco Spadafora, and James A Landay. 2016. Emotion encoding in human-drone interaction. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 263–270.
[11]
Kaixu Dong, Runze Ding, Songnan Bai, Xinyu Cai, and Pakpong Chirarattananon. 2023. Stabilizing Aerodynamic Dampers for Cooperative Transport of a Suspended Payload with Aerial Robots. Advanced Intelligent Systems (2023), 2300112.
[12]
Sara Eriksson, Åsa Unander-Scharin, Vincent Trichon, Carl Unander-Scharin, Hedvig Kjellström, and Kristina Höök. 2019. Dancing with drones: Crafting novel artistic expressions through intercorporeality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[13]
Sarah Fdili Alaoui. 2019. Making an interactive dance piece: Tensions in integrating technology in art. In Proceedings of the 2019 on designing interactive systems conference. 1195–1208.
[14]
Maiken Hillerup Fogtmann, Jonas Fritsch, and Karen Johanne Kortbek. 2008. Kinesthetic interaction: revealing the bodily potential in interaction design. In Proceedings of the 20th Australasian conference on computer-human interaction: designing for habitus and habitat. 89–96.
[15]
Jules Françoise, Yves Candau, Sarah Fdili Alaoui, and Thecla Schiphorst. 2017. Designing for kinesthetic awareness: Revealing user experiences through second-person inquiry. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 5171–5183.
[16]
Natalie Friedman, Kari Love, RAY LC, Jenny E Sabin, Guy Hoffman, and Wendy Ju. 2021. What robots need from clothing. In Designing Interactive Systems Conference 2021. 1345–1355.
[17]
P. Gemeinboeck and R. Saunders. 2018. Human-Robot Kinesthetics: Mediating Kinesthetic Experience for Designing Affective Non-humanlike Social Robots. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 571–576. https://doi.org/10.1109/ROMAN.2018.8525596
[18]
Petra Gemeinboeck and Rob Saunders. 2022. Moving beyond the mirror: relational and performative meaning making in human–robot communication. AI & SOCIETY 37, 2 (2022), 549–563.
[19]
Kehong Gong, Bingbing Li, Jianfeng Zhang, Tao Wang, Jing Huang, Michael Bi Mi, Jiashi Feng, and Xinchao Wang. 2022. PoseTriplet: Co-evolving 3D human pose estimation, imitation, and hallucination under self-supervision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11017–11027.
[20]
Edward T Hall. 1966. The hidden dimension. Vol. 609. Anchor.
[21]
Jeonghye Han and Ilhan Bae. 2018. Social proxemics of human-drone interaction: Flying altitude and size. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 376–376.
[22]
Viviane Herdel, Anastasia Kuzminykh, Andrea Hildebrandt, and Jessica R Cauchard. 2021. Drone in love: Emotional perception of facial expressions on flying robots. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–20.
[23]
Michaela Honauer, Danielle Wilde, and Eva Hornecker. 2020. Overcoming Reserve - Supporting Professional Appropriation of Interactive Costumes. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 2189–2200. https://doi.org/10.1145/3357236.3395498
[24]
Stacy Hsueh, Sarah Fdili Alaoui, and Wendy E Mackay. 2019. Understanding kinaesthetic creativity in dance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[25]
Tao Jiang, Peng Lu, Li Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen. 2023. RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose. arXiv preprint arXiv:2303.07399 (2023).
[26]
Elizabeth Jochum and Jeroen Derks. 2019. Tonight We Improvise! Real-time tracking for human-robot improvisational dance. In Proceedings of the 6th International Conference on Movement and Computing. 1–11.
[27]
Laewoo Kang, Steven J Jackson, and Phoebe Sengers. 2018. Intermodulation: improvisation and collaborative art practice for hci. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–13.
[28]
Pavel Karpashevich, Eva Hornecker, Michaela Honauer, and Pedro Sanches. 2018. Reinterpreting Schlemmer’s Triadic Ballet: interactive costume for unthinkable movements. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
[29]
Heesoon Kim and James A Landay. 2018. Aeroquake: Drone augmented dance. In Proceedings of the 2018 Designing Interactive Systems Conference. 691–701.
[30]
Melanie Kimmel and Sandra Hirche. 2017. Invariance control for safe human–robot interaction in dynamic environments. IEEE Transactions on Robotics 33, 6 (2017), 1327–1342.
[31]
Kazuhiro Kosuge. 2010. Dance Partner Robot: An engineering approach to human-robot interaction. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 201–201.
[32]
Joseph La Delfa, Mehmet Aydin Baytas, Rakesh Patibanda, Hazel Ngari, Rohit Ashok Khot, and Florian’Floyd’ Mueller. 2020. Drone chi: Somaesthetic human-drone interaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[33]
RAY LC. 2021. NOW YOU SEE ME, NOW YOU DON’T: revealing personality and narratives from playful interactions with machines being watched. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. 1–7.
[34]
RAY LC, Maurice Benayoun, Permagnus Lindborg, Hongshen Xu, Hin Chung Chan, Ka Man Yip, and Tianyi Zhang. 2021. Power chess: robot-to-robot nonverbal emotional expression applied to competitive play. In 10th International Conference on Digital and Interactive Arts. 1–11.
[35]
RAY LC, Sijia Liu, and Qiaosheng Lyu. 2023. IN/ACTive: A Distance-Technology-Mediated Stage for Performer-Audience Telepresence and Environmental Control. In Proceedings of the 31st ACM International Conference on Multimedia. 6989–6997.
[36]
Ray LC, Sihuang Man, Xiying Bao, Jinhan Wan, Bo Wen, and Zijing Song. 2023. " Contradiction pushes me to improvise": Performer Expressivity and Engagement in Distanced Movement Performance Paradigms. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1–26.
[37]
James Leach and Catherine J Stevens. 2020. Relational creativity and improvisation in contemporary dance. Interdisciplinary Science Reviews 45, 1 (2020), 95–116.
[38]
Yanheng Li, Lin Luoying, Xinyan Li, Yaxuan Mao, and Ray Lc. 2023. " Nice to meet you!" Expressing Emotions with Movement Gestures and Textual Content in Automatic Handwriting Robots. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 71–75.
[39]
Marc Lieser, Ulrich Schwanecke, and Jörg Berdux. 2021. Evaluating distances in tactile human-drone interaction. In 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN). IEEE, 1275–1282.
[40]
Weizhou Luo, Anke Lehmann, Hjalmar Widengren, and Raimund Dachselt. 2022. Where should we put it? layout and placement strategies of documents in augmented reality for collaborative sensemaking. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–16.
[41]
Robert Mahony, Vijay Kumar, and Peter Corke. 2012. Multirotor aerial vehicles: Modeling, estimation, and control of quadrotor. IEEE robotics & automation magazine 19, 3 (2012), 20–32.
[42]
Paul H Mason. 2012. Music, dance and the total art work: choreomusicology in theory and practice. Research in dance education 13, 1 (2012), 5–24.
[43]
Zachary McKendrick, Ori Fartook, Patrick Finn, Ehud Sharlin, and Jessica Cauchard. 2023. Waiting in the Wings: Drones in Live Performance. In Graphics Interface 2023-second deadline.
[44]
Yuko Nakano and Takeshi Okada. 2012. Process of improvisational contemporary dance. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 34.
[45]
Jimmy Or. 2009. Towards the development of emotional dancing humanoid robots. International Journal of Social Robotics 1 (2009), 367–382.
[46]
Roosa Piitulainen, Perttu Hämäläinen, and Elisa D Mekler. 2022. Vibing together: Dance experiences in social virtual reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18.
[47]
Helen Regan and Julia Howe. 2017. Video self-modelling: an intervention for children with behavioural difficulties. Educational Psychology in Practice 33, 1 (2017), 93–102.
[48]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015).
[49]
Amit Rogel, Richard Savery, Ning Yang, and Gil Weinberg. 2022. RoboGroove: Creating Fluid Motion for Dancing Robotic Arms. In Proceedings of the 8th International Conference on Movement and Computing. 1–9.
[50]
Johnny Saldana. 2015. The coding manual for qualitative researchers. SAGE.
[51]
Jürgen Scheible and Markus Funk. 2016. DroneLandArt: landscape as organic pervasive display. In Proceedings of the 5th ACM International Symposium on Pervasive Displays. 255–256.
[52]
Anselm Strauss and Juliet Corbin. 1998. Basics of qualitative research techniques. (1998).
[53]
McAngus N Todd, C Lee, and D O’Boyle. 2002. A sensorimotor theory of temporal tracking and beat induction. Psychological research 66, 1 (2002), 26–39.
[54]
Sanne Van Waveren, Rasmus Rudling, Iolanda Leite, Patric Jensfelt, and Christian Pek. 2023. Increasing perceived safety in motion planning for human-drone interaction. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 446–455.
[55]
Markus Waibel, Bill Keays, and Federico Augugliaro. 2017. Drone shows: Creative potential and best practices. Technical Report. ETH Zurich.
[56]
Ashley R. Williams. 2023. Some US cities are replacing 4th of July fireworks with environmentally friendly drones. https://edition.cnn.com/2023/07/02/us/drones-replace-july-fourth-fireworks-trnd/index.html
[57]
Nialah Jenae Wilson-Small, David Goedicke, Kirstin Petersen, and Shiri Azenkot. 2023. A Drone Teacher: Designing Physical Human-Drone Interactions for Movement Instruction. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 311–320.
[58]
Nialah Jenae Wilson-Small, Louisa Pancoast, Kirstin Petersen, and Shiri Azenkot. 2023. Exploring Human-Drone Collaboration Through Contact Improvisation. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 97–101.
[59]
Anna Wojciechowska, Jeremy Frey, Sarit Sass, Roy Shafir, and Jessica R Cauchard. 2019. Collocated human-drone interaction: Methodology and approach strategy. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 172–181.
[60]
Zeyi Yang. 2023. Food delivery by drone is just part of daily life in Shenzhen. https://www.technologyreview.com/2023/05/23/1073500/drone-food-delivery-shenzhen-meituan/
[61]
Alexander Yeh, Photchara Ratsamee, Kiyoshi Kiyokawa, Yuki Uranishi, Tomohiro Mashita, Haruo Takemura, Morten Fjeld, and Mohammad Obaid. 2017. Exploring proxemics for human-drone interaction. In Proceedings of the 5th international conference on human agent interaction. 81–88.
[62]
Massimiliano Zanoni, Michele Buccoli, Guglielmo Cassinelli, and Giorgio Rinolfi. 2020. Deep Music on Air. In Proceedings of the 9th International Conference on Digital and Interactive Arts (Braga, Portugal) (ARTECH 2019). Association for Computing Machinery, New York, NY, USA, Article 90, 4 pages. https://doi.org/10.1145/3359852.3359970
[63]
Qiushi Zhou, Cheng Cheng Chua, Jarrod Knibbe, Jorge Goncalves, and Eduardo Velloso. 2021. Dance and choreography in HCI: a two-decade retrospective. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
[64]
Zhuoming Zhou, Elena Márquez Segura, Jared Duval, Michael John, and Katherine Isbister. 2019. Astaire: A collaborative mixed reality dance game for collocated players. In Proceedings of the annual symposium on computer-human interaction in play. 5–18.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
May 2024
18961 pages
ISBN:9798400703300
DOI:10.1145/3613904
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. human-drone interactions
  2. improvisational dancing
  3. micro aerial vehicles

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

CHI '24

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)937
  • Downloads (Last 6 weeks)80
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media