Understanding palm-based imaginary interfaces: the role of visual and tactile cues when browsing

Imaginary Interfaces are screen-less ultra-mobile interfaces. Previously we showed that even though they offer no visual feedback they allow users to interact spatially, e.g., by pointing at a location on their non-dominant hand.

The primary goal of this paper is to provide a deeper understanding of palm-based imaginary interfaces, i.e., why they work. We perform our exploration using an interaction style inspired by interfaces for visually impaired users. We implemented a system that audibly announces target names as users scrub across their palm. Based on this interface, we conducted three studies. We found that (1) even though imaginary interfaces cannot display visual contents, users' visual sense remains the main mechanism that allows users to control the interface, as they watch their hands interact. (2) When we remove the visual sense by blindfolding, the tactile cues of both hands feeling each other in part replace the lacking visual cues, keeping imaginary interfaces usable. (3) While we initially expected the cues sensed by the pointing finger to be most important, we found instead that it is the tactile cues sensed by the palm that allow users to orient themselves most effectively.

While these findings are primarily intended to deepen our understanding of Imaginary Interfaces, they also show that eyes-free interfaces located on skin outperform interfaces on physical devices. In particular, this suggests that palm-based imaginary interfaces may have benefits for visually impaired users, potentially outperforming the touchscreen-based devices they use today.