RESEARCHES

A general interaction for conventional gesture recognition technology is one in which a machine views a human gesture, recognizes the gesture after that behavior is finished, and reacts accordingly. Here, a “rock-paper-scissors” (RPS) robot system that always wins is explained as an example of a system that predicts the result of unfinished human behavior from high-speed motion capture information and reacts based on the prediction.

To always win at the game of “rock-paper-scissors” with a human, a robot needs to satisfy the conditions that the robot recognize the status of the human’s fingers and present a winning hand shape within approximately 0.2 seconds from when the human begins to show his or her hand until the behavior is completed. For the robot to recognize the hand shape, the human is supposed to wear a black glove marked at three points. The robot makes its assessments of rock, paper, scissors from this three-point position information.

In our high-speed gesture robot, high-speed megapixel vision, based on iMVS-155 was used for viewing. For presentation of the gesture, a robot hand capable of bending and extending its fingers at high speed was used. The PC recognizes the human hand by processing the high-speed motion-capture information from the viewing section and controls the robot hand through a servo driver to present a winning hand.

The moving pictures below show an actual experiment for a game of “rock-paper-scissors” between the system and a human. Despite the human being very quick, the robot always won. Similarly, the robot may be set to always lose or to tie.