Smart Vision & Robotic Sensing

Professor, Robotics Laboratory
Smart Innovation Program, Graduate School of Advanced Science and Engineering
Hiroshima University
>> Research Contents
In order to establish high-speed robot senses that are much faster than human senses, we are conducting research and development of information systems and devices that can achieve real-time image processing at 1000 frames/s or greater. As well as integrated algorithms to accelerate sensor information processing, we are also studying new sensing methodologies based on vibration and flow dynamics; they are too fast for humans to sense.

HFR-video-based Honeybee Activity Sensing

This study realizes the tracking of honey bees flying in a natural scene by inspecting the periodic changes in the brightness at pixels around their wings that were flapping at hundreds of Hertz in a high-frame-rate video. The periodic changes in brightness are extracted by executing short-time Fourier transforms at all pixels, and the pixels detected as vibrational ones are identified as several labeled regions with connected component labeling, corresponding to multiple honeybees; the trajectories of the honeybees are monitored with their time-frequency responses as their flight activities. We demonstrate the performance of the honeybee activity sensing when a 4 m x 4 m scene around a beehive was captured as 1024×1024 images at 2000 fps, in which one pixel corresponded to 4 mm; 12-14-mm-size flying honeybees were correctly tracked when their appearances were not as obvious in the images.








mp4 movie(1.8 MB)
input video (1000fps)
mp4 movie(0.3 MB)
extracted vibration region