User-defined Swarm Robot Control

Lawrence H. Kim, Daniel S. Drew, Veronika Domova, Sean Follmer


A swarm of robots can accomplish more than the sum of its parts, and swarm systems will soon see increased use in applications ranging from tangible interfaces to search and rescue teams. However, effective human control of robot swarms has been shown to be demonstrably more diffcult than controlling a single robot, and swarm-specifc interactions methodologies are relatively underexplored. As we envision even non-expert users will have more daily in-person encounters with different numbers of robots in the future, we present a user-defned set of control interactions for tabletop swarm robots derived from an elicitation study. We investigated the effects of number of robots and proximity on the user’s interaction and found signifcant effects. For instance, participants varied between using 1-2 fngers, one hand, and both hands depending on the group size. We also provide general design guidelines such as preferred interaction modality, common strategies, and a high-agreement interaction set.

%## Video %<iframe width="560" height="315" src="" frameborder="0" allowfullscreen></iframe>


H1: Number of robots will affect how users interact.

Researchers have shown that the number of robots can signifcantly alter how people perceive the robots when viewing their motion [36] or being touched by them. Researchers have also developed different ways to teleoperate or remotely control a swarm of agents such as leader-follower, selection and beacon control, and physicomimetics. Thus, we hypothesize that users will also adapt their interaction method for in situ control based on the number of robots.

H2: Proximity to the robot(s) will affect how users interact.

Literature in Human-Robot Interaction (HRI) has shown that humans perceive robots differently based on their proximity as well as prefer robots that exhibit proxemic behavior. Cauchard et al. have reported that when the robots were closer, users tended to use smaller motions. Thus, we also hypothesize that proximity to the robot(s) will change how users choose to interact with a swarm of robots.


Example Prompt


Interaction Modality

Taxonometric Breakdown


User-defined Interaction Set

The user-defned interaction set was generated by taking the most frequent interaction for each referent. If the same interaction was used for different referents thus creating confict, the interaction was assigned to the referent with the largest group.

Effects of number of robots + proximity to the robots

For each referent category, we compared its data with that of the remaining referents. For instance, for the robot selection category, we compared its data with that of referents not in the robot selection category.


Lawrence H. Kim, Daniel S. Drew, Veronika Domova, and Sean Follmer. 2020. User-defined Swarm Robot Control. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ‘20). Association for Computing Machinery, New York, NY, USA, 1-13. DOI: