With the help of AI, researchers have overcome a major challenge in biomimetic robotics by developing a sensor that can glide over Braille text and read it accurately at twice the speed of a human. This technology can be integrated into robotic hands and prosthetics to provide fingertip sensitivity similar to that of humans.
Human fingertips are incredibly sensitive. They can convey details in objects as small as about half the width of a human hair, discern subtle differences in surface texture, and apply just the right amount of force to grasp an egg or a 20-pound (9 kg) bag of dog food. Without slipping.
As cutting-edge electronic skins begin to incorporate more and more biomimetic features, the need for human-like dynamic interactions, such as sliding, becomes more important. However, despite advances in soft robotics, replicating the sensitivity of human fingertips in robotic equivalents has proven difficult.
Researchers at the University of Cambridge, UK, have taken an approach that uses vision-based tactile sensors combined with AI to detect features at high resolution and speed, taking it one step closer to reality.
“The softness of human fingertips is one of the reasons we can grasp objects with the right amount of pressure,” said Parth Potdar, lead author of the study. “For robotics, softness is a useful characteristic, but it also requires a lot of sensor information, and obtaining both simultaneously is challenging, especially when dealing with flexible or deformable surfaces.”
Researchers have set themselves the difficult task of developing a robotic 'fingertip' sensor that can read Braille by sliding like a human finger. It's an ideal test. Because the dots of each representative character are placed closely together, the sensor must be very sensitive.
“There are existing robotic Braille readers, but they only read one letter at a time,” said study co-author David Hardman. “This is different from the way humans read.” “Existing robotic Braille readers operate in a static manner. Touch one letter pattern, read it, pull it from the surface to move it up, lower it to the next letter pattern, and so on. We want something more realistic and much more efficient.”
So the researchers created a robotic sensor with a camera mounted on the 'finger tip'. Recognizing that the sliding motion of the sensor causes motion blur, the researchers used a machine learning algorithm trained on a set of synthetically blurred real-world static images to 'deblur' the images. Once motion blur was removed, a computer vision model detected and classified each letter.
“This is a challenge for roboticists because there is a lot of image processing that needs to be done to remove motion blur, which is time-consuming and energy-consuming,” Potdar said.
Incorporating trained machine learning algorithms, the robot sensor can read 315 words per minute with 87.5% accuracy, twice the speed of human reading and Braille with approximately the same accuracy. The researchers say this is significantly faster than previous studies and that the approach can be scaled up to more data and more complex model architectures to achieve better performance at higher speeds.
“Considering that we used fake blur to train the algorithm, we were surprised at how accurate it was at reading Braille,” Hardman said. “We found a good balance between speed and accuracy, and this holds true for human readers as well.”
Although the sensor is not designed as an assistive technology, the researchers say its ability to read Braille quickly and accurately bodes well for developing robotic hands or prosthetics with sensitivity similar to human fingertips. They hope to scale the technology to the size of a humanoid hand or skin.
“Braille reading speed is a good way to measure the dynamic performance of a tactile sensing system, so our findings can be applied beyond Braille to applications such as surface texture or slip detection in robotic manipulation,” Potdar said.
The study was published in the journal IEEE Robotics and Automation CorrespondenceThe video below, produced by the University of Cambridge, explains how researchers developed a braille reading sensor.
Can a robot read Braille?
Source: University of Cambridge