Robot Moved at the 'Beep~' Sound... Development of Artificial Skin That Hears Sound

Ulsan National Institute of Science and Technology

[Asia Economy Reporter Kim Bong-su] An interface technology that controls robots using artificial skin capable of hearing sounds has been developed. This technology allows robots to distinguish the texture of materials through artificial skin and recognize sounds to execute commands. It can even mimic human movements exactly. It is attracting attention as a technology perfectly suited for the metaverse and avatar robot era.

Ulsan National Institute of Science and Technology (UNIST) announced on the 27th that a research team led by Professors Ko Hyun-hyeop (Department of Energy and Chemical Engineering) and Kim Jae-jun (Department of Electrical and Electronic Engineering) developed a human-machine interface that can recognize and transmit human movements, textures, and sounds to machines.

This is a human-machine interface method that intuitively transmits information to machines instead of pressing buttons or keyboards. The sensor is thin and attachable as artificial skin, making it applicable to various virtual reality (VR), augmented reality (AR), and Internet of Things (IoT) technologies. It is based on an artificial skin sensor that mimics the cochlear structure of the ear. The cochlear basilar membrane varies in thickness, width, and stiffness by region, enabling it to distinguish sounds by frequency. This principle was applied. Thanks to these characteristics of the sensor, it can transmit to machines not only low-frequency signals that repeat slowly like human movements but also high-frequency signals such as rapidly vibrating sounds and textures with a low signal-to-noise ratio.

The research team demonstrated application technologies such as avatar robot hand control technology and smart haptic gloves using this sensor. In a demonstration controlling an avatar robot hand by sound, the robot hand’s movements could be controlled by changing the frequency. Also, when a user wore the smart haptic gloves and moved, the avatar robot hand precisely followed the user’s hand movements, and it recognized the textures of eight different materials including glass, paper, and silk with 93% accuracy.

The developed sensor consists of multiple unit triboelectric sensors with varying thickness, porosity, and area, continuously attached like the cochlear basilar membrane. Additionally, the internal structure of the sensor was specially designed to improve pressure sensitivity up to eight times compared to conventional flat sensors. The recognition frequency bandwidth ranges from 45 to 9,000 Hz, allowing it to recognize all biological signals such as electrocardiogram signals (0.5 to 300 Hz), electromyogram signals (50 to 3,000 Hz), phonocardiogram signals (20 to 20,000 Hz), and voice (100 to 400 Hz). Even in noisy external environments, it can recognize human voices with 95% accuracy through machine learning, enabling its use as a microphone equipped with noise cancellation functionality.

This research was published on the 25th in Science Advances, a sister journal of Science, published by the American Association for the Advancement of Science (AAAS).