11.1 Discussion
This research project investigates the previously unstudied phenomenon of intra-hug gestures during hugs between a human user and an autonomous robot. We found users to be positively interested in hugging a robot that can both respond to their gestures and proactively perform gestures on its own. The results support all six new of the design guidelines we proposed for hugging robots.
G4: Hug Initiation. First, our revised version of the fourth tenet states that when a hugging robot is the one initiating the interaction, it should autonomously initiate a hug when it detects a user in its personal space by inviting the user for a hug; it should then wait for the user to begin walking toward it before closing its arms to ensure a consensual and synchronous hugging experience, as done by HuggieBot 3.0. The previous evaluation of HuggieBot 2.0 did not find a statistically significant user preference between a hug initiated with a button press or a hug initiated via computer vision detection of the user’s approach [
13]. Users gave our new two-phase hug initiation method an average naturalness rating of almost a 7 out of 10. All users were able to initiate hugs with the robot after watching a simple video demonstration twice, without requiring the detailed instructions that were previously provided. Thus, we conclude that the new hug initiation method is an improvement over the previously tested methods. We also had pilot-tested an alternative three-step hug initiation method that was harder for users to master and received an average naturalness rating of only 4.62 out of 10 from 12 pilot participants. Our observations repeatedly indicate that consensual and synchronous hug initiation is indeed important. Although it still has room for improvement, rather than forcing the user to hurry up and wait, the new two-phase method more closely mimics how hugs occur between humans and lets the user decide when to start hugging the robot. Relevantly, Walters et al. [
76] found that both the voice used and previous experience interacting with the robot can have an effect on the mean approach distance to a robot. Mead and Matarić [
47] extensively worked on understanding interaction potential based on human-robot proxemics. These prior works also support the need for consensual and synchronous hug initiation highlighted by G4.
G7: Adaptation to User Height. The next new design guideline states that a good hugging robot should also perceive the user’s height and adapt its arm positions to contact the user at appropriate locations. We thus extended our platform’s perceptual capabilities beyond detecting a user’s approach so that HuggieBot 3.0 attempts to embrace the user at the proper height—not too high, and not too low. Our newly developed height adjustment system was created in direct response to user feedback about the inappropriateness of HuggieBot 2.0’s hand placement in the action-response elicitation study. The average rating for the appropriateness of HuggieBot 3.0’s hand placement’s was an 8.4 out of 10, with several users giving it the highest rating possible; these ratings indicate that the proposed approach usually succeeds at adjusting for user height. We also believe that the non-significant increases in the quality ratings for the rub and pat gestures (which we did not adjust other than the location of the robot’s left hand) can probably be attributed to this improved placement. In human-human touch interactions, there are relationship-specific maps of body regions where touch is considered appropriate [
66]; the areas where contact is allowed increase with the emotional bond to the toucher. Therefore, from human-human touch interaction research, we confirm that our robot’s hand placement on the user’s back is important to avoid taboo zones and ensure the comfort of all users, regardless of the relationship they associate with the robot. Thus, we conclude that users prefer hugging a robot that adjusts its arm placement to match their height.
G8: Gesture Perception. The next design guideline centers on enabling a hugging robot to accurately and reliably detect and classify user gestures applied to its torso in real time, regardless of the user’s hand placement. Both of our user studies demonstrated the excellent haptic sensing capabilities of the pressure sensor and microphone inside the inflated chamber of HuggieChest; simple signal-processing and machine learning techniques were able to detect and classify contacts very well, even when the pipeline was transferred to new sensing hardware and adapted with limited new training data. As our subjects used various hand positions on the robot (both arms below the robot’s arms, both above, or one above and one below), we found that this non-localized haptic sensing system works well regardless of user hand placement on the back chamber. Based on the average gesture detection accuracy of 86% for the 16 participants in the validation study, along with the positive opinions users shared when the robot responded to their gestures in both studies, we believe the results support the validity of this design guideline. In further exploring cases when the detection algorithm did not perform well, we found that users frequently performed the gesture in an unexpected or uncommon way; in some cases, these variations may have come from the user’s limited vocabulary for intra-hug gestures in English. As a surprising benefit, however, the perception pipeline was able to detect some rubs and pats performed at the same time as a squeeze, even though it had not been trained to do so.
G9: Fast Response. The ninth design guideline simply states that users like a robot that responds quickly to their intra-hug gestures. As seen in Figure
4, when a user performed an intra-hug gesture on the robot’s back, and the robot did not respond, users perceived this as a neutral robot behavior on average. In their written and verbal comments to the experimenter, users indicated they did not feel like the robot “understood” them, “knew [they were] there,” or “wanted to support/comfort [them].” Users clearly preferred when the robot indicated that it knew the user had performed an intra-hug gesture and responded quickly in some way. When we were piloting the validation study, we noticed that a technical error occasionally caused the sampling rate of the microphone and pressure sensor to drop to about half the normal value. We found that when users interacted with the robot in this condition, the delay was highly noticeable and detracted from the user experience. Several pilot subjects mentioned that “it felt like the robot was performing random actions at random intervals, not in response to anything I was doing.” During our final validation study, when the sampling rate issue was fixed, the robot responded quickly to user actions, and our subjects were delighted. They repeatedly performed the same gesture to experience the robot response again. Users would often comment to the experimenter during the hug itself, saying “it tapped me back!” (P4), or remarking after the hug that “every time I did an action, it noticed and did something back to me!” (P16) because they were so pleasantly surprised at the responsiveness of the robot. This user desire for a fast robot response time aligns with expert therapist opinions that robotic systems for perceiving social touch from humans also have strict timing requirements [
16]. A relevant common psychology study is called “infant response to still-face,” where a mother, who had been interacting and playing normally with her child, suddenly stops smiling and talking to her infant while the experimenters observe the child’s response [
70]. Children commonly become distressed when their mothers no longer respond, and can cry, become fussy, and grasp at themselves and their mothers, trying to get attention. The
delayed response from their mother significantly upsets them. Through the combination of results from our two user studies, as well as research from human-human interactions, we believe there is support for our ninth guideline suggesting a fast robot response to intra-hug gestures.
G10: Response Variety. The next design guideline states that hugging robots should adopt a gesture response paradigm that blends user preferences with slight variety and spontaneity. When starting this project, we believed that hugging robots should always reciprocate the same intra-hug gesture the user had performed. The results from the action-response elicitation study surprised us by showing that rote reciprocation is not expected and would not be perceived in a fully positive way. If users preferred gesture reciprocation, we would see a dark pink diagonal in Figure
4. Instead, we see a slight preference for a robot to respond to any user action with a squeeze. Speaking with our users showed us that they appreciate variety in robot responses. Something about the unpredictability of the response leads users to feel it is more “alive.” Users also mentioned that having the robot respond with the same action as the user performed feels “too mechanical,” because based on the input you know exactly what output you will receive. The results from the action-response elicitation study thus support this design guideline, as do the very positive user reactions to the resulting robot behavior algorithm tested in the validation study. We believe the slightly spontaneous robot hugging behavior enabled by our simple probabilistic behavior paradigm (Equation (
3)) succeeds at blending user preferences with spontaneity to reasonably match natural human exchanges of intra-hug gestures. The behavior algorithm’s tendency to prefer exploration versus exploitation can also easily be adjusted by changing the value of the exponent
\(m\) . Interestingly, most human-human research found that mimicry increases perception of another person [
21,
73]. The chameleon effect is a common phenomenon in which humans unconsciously mimic the gestures and facial expressions of an interaction partner to match their type of social expression and level of extroversion [
20]. In this instance, our users made it clear that explicit mimicry from the robot is not appreciated, but that they did want similar levels of support. This finding could be seen as similar to human interactions. In a social environment, it would be uncomfortable if a social partner was obviously copying you, but responding to gestures with similar levels of enthusiasm seems warranted.
G11: Proactive Robot Gestures. This design guideline states that hugging robots should occasionally provide unprompted proactive affective social touch to the user through intra-hug gestures. The findings of Chen et al. [
22] made us initially hypothesize that users would dislike robot-initiated affective social touch delivered via unprompted intra-hug gestures; their users reacted negatively when a robot attempted to comfort them by touch but did not mind functional contact from the robot. The findings of our two studies explicitly contradict this hypothesis and support G11. We were so surprised by these ratings during the action-response elicitation study that after the user had finished explaining their ratings, the experimenter asked the follow-up question, “so just to clarify, it did not bother you that you did nothing and the robot unprompted started rubbing/ patting/squeezing you?” Users confirmed that not only did they not mind this robot behavior, but they also
enjoyed and appreciated it. Users indicated that while in the other cases, the robot would respond to their gestures, here, they felt the robot was comforting them. In these cases, many users commented that they felt the robot’s emotions and feelings and that it cared more about them when it chose on its own to perform a gesture, rather than just responding. Although more work needs to be done to confirm this positive finding, it seems that appropriately framed robot-initiated affective touch may be key to creating robots that can provide good emotional support to human users.
How can we grapple with the seemingly conflicting findings between our work and Chen et al. [
22]? We believe these results are not as different as they may appear. The users in our studies
agreed to enter into a hug with a robot, so we believe they also felt at least partially responsible for initiating the affective touch that occurred during the resulting hug. Once this initial boundary is broken, we believe users are more receptive to proactive robot affective touch, for example, a rub, pat, or squeeze. Users in all of our studies have appreciated that HuggieBot
politely asked them for a hug, thereby allowing them to agree to this affective touch. Many users even responded affirmatively to the robot every time it asked the question, even though they knew it never listened to their answer. By changing the hug initiation method to be prompted by the robot lifting its arms for a hug and asking “Can I have a hug, please?” and then waiting with its arms outstretched for the user to approach, we further put the initiation of the affective social touch on the user, solidifying that it is their choice to enter the hug. We believe user initiation is key to acceptance of future social, affective touch from a robot. We therefore firmly believe G11’s statement that robots can evoke user feelings that the robot is alive and caring by occasionally providing unprompted affective touch to the user, as delivered by HuggieBot 3.0 through intra-hug gestures.
11.2 Limitations
While the research described in this article presents several key contributions to robotic hugging and broader social-physical HRI, we nevertheless acknowledge several limitations of our work.
The first limitation is the somewhat artificial methodology of our studies. We recognize the importance of conducting in-the-wild studies for human-robot interaction research. By conducting these studies in a laboratory environment, we have a self-selection bias of our participant pool. Only users who were interested in hugging a robot chose to participate in the reported studies. Unfortunately, due to the current COVID-19 pandemic, lab studies were the safest way to conduct research on hugging robots. We were able to screen participants for potential health risks and thoroughly sanitize the robot between subjects. For the validation study, we changed the robot’s introduction from verbal instructions from the experimenter to having the user watch a simple video of another user hugging the robot. This video introduction was meant to mimic how users would learn to use the robot in the wild. Once the COVID-19 crisis has ended in our region, we look forward to conducting a thorough in-the-wild study to see how many everyday people would and would not be interested in hugging a robot. Additionally, the COVID-19 pandemic also reduced the number of participants we could recruit for our second user study; the results from the validation study have lower statistical power (roughly 50%) than would typically be presented and thus should not be the only results taken into consideration.
Next, our refined fourth guideline is limited in that it addresses only situations where the robot is the one initiating the interaction. If the user is the one requesting the hug from the robot, the first half of the guideline should be ignored, and the robot should observe only the second half, which is “waiting for the user to begin walking toward it before closing its arms to ensure a consensual and synchronous hugging experience.”
Additionally, other researchers such as Chen et al. [
22] have shown the importance of context with respect to the acceptance of social touch. The reported studies did not use any specific context beyond the narrative descriptions provided by the experimenter. Future work could study HuggieBot in different contexts, such as a nursing home or shopping mall, and evaluate the effect each context has on user expectations and interpretations of the interaction.
We have also identified two main limitations of the Kinova JACO arms used in both HuggieBot 2.0 and 3.0. While we adjusted the hug initiation process to accommodate this limitation better, the Kinova JACO arms just cannot move fast enough to mimic the speed of a human’s arms closing. These arms were selected for safety reasons, and this speed limit was considered during that choice. After extensive user testing, we have found that the speed limit causes a real limitation on the naturalness of the user experience because subjects have to wait several seconds before the arms have fully closed around them. A second limitation of these arms is that repeated small movements of the first, second and third joints (shoulder lift, shoulder pan, and elbow flex, respectively) occasionally result in a sudden short but fast jerking movement, which startles the user. This phenomenon occurs only rarely during repeated rubs or pats. When this issue occurred during the two reported studies, we immediately commanded the robot to release the user, checked that they were okay, verified whether they wanted to continue, discounted the trial with the malfunction, and restarted the trial. Because the sudden motion is very small, this technical glitch never hurt a user. However, it is likely that it negatively affected some user ratings of HuggieBot as a whole, as well as the robot’s ability to perform rubs and pats.
This work is also limited by the fact that we have simplified the problem of gesture classification significantly by focusing on only four gestures. There are infinitely many gestures a person could choose to perform during a hug, and there are infinitely many ways they could perform each gesture. A user could even combine multiple gestures together. We chose to select a simple subset of four classic gestures and their combinations (e.g., squeeze-pat) as a first step into this new research area. We currently do not estimate the intensity with which users perform these gestures, nor do we measure the location where gestures are performed. Interesting future steps would be to measure the intensity and/or the location of user gestures to enable the robot to reciprocate gestures with an appropriate intensity and/or location on the user’s back.
Another limitation is that both of our studies asked the user to perform intra-hug gestures somewhat artificially. After placing their hands on the robot’s back, users had to wait for the robot’s arms to close fully before performing a gesture. This pause was used to collect baseline measurements for the microphone and pressure signals so that the real-time perception pipeline could determine what gestures the user subsequently performed. We found that many users naturally wanted to start performing the gesture immediately after beginning the hug, regardless of the robot’s arm movements. To collect data and then test our algorithm’s accuracy, we also asked users to perform only one gesture per hug, though they could perform the gesture repeatedly if they chose. This restriction was also somewhat unnatural. We found that many users naturally wanted to combine gestures. We added the natural hug scenarios in phase 1 and phase 3 of the validation study to address this limitation.
Our action-response elicitation study challenged users with the difficult task of separating the appropriateness of the robot’s response from the quality with which HuggieBot 2.0 performed the gesture. We had them explain their ratings to the experimenter to ensure they understood the distinction and were answering the question correctly. Nonetheless, the robot’s gesture quality probably affected other user ratings. Users experienced a similar challenge in the validation study, where we again asked them to separately rate the robot responses from the quality of the robot’s gestures; gesture quality also thus probably affected these results. Another limitation from the action-response elicitation study is that we did not ask users to rate the naturalness of the hug initiation process or the appropriateness of the robot’s hand placement. We had not realized these aspects of HuggieBot 2.0’s behavior would garner negative comments and thus need to be adjusted for the new version of the platform. Thus, we had to rely on written and verbal comments to evaluate the effects of these changes.
An additional limitation involves our evaluation of the user experience. Though we aimed at assessing it in an accurate manner and specifically collected data in multiple ways to facilitate comparisons, it is possible there were still problems. First, whenever collecting self-reported data, the questions will be subjected to the interpretation of the users, who may not have the same understanding [
58]. We used pilot testing to make our questions as clear and unambiguous as possible. We also did our best to conceal which aspects of the robot we were evaluating, so as to avoid the demand effect, where a participant tries to respond in a way to confirm or deny the hypothesis of a study [
55]. Though we acknowledge that a participant could have deduced what we were testing for, we do not think it is likely that participants responded untruthfully because we consciously conveyed equipoise throughout the experiment and because several of our findings did not match our initial hypotheses. Finally, as with any technology, there is the concern of the novelty effect, that users’ attitudes and preferences will wane over time [
41,
44]. We aimed at mitigating this effect by conducting long experiments and querying user opinions both before and after one and a half hours of robot hugs. Nevertheless, for almost all of our users, the reported study was their first experience interacting with HuggieBot, and for many of them it was their first time interacting with any robot. To better evaluate the influence of the novelty effect on user evaluations with HuggieBot, future studies should have users interact with HuggieBot over the course of many weeks or months.
Finally, having a robot that fully understands a human hug is very challenging. We acknowledge that the current version of our robot does not deliver on the full aspirational goal of a hugging robot. Rather, HuggieBot simulates a hug in a reasonably compelling way, and our data suggest that users enjoy the hug and can engage with the robot and relate to it as an autonomous being. However, in its current state, HuggieBot does not have an internal emotional model similar to humans, and thus it is not capable of engaging in the embodied emotional experience of a hug.