Researchers have developed a robotic sensor incorporating artificial intelligence technology that can read Braille at a rate about twice as fast as most human readers.
A team of researchers at the University of Cambridge used machine learning algorithms to teach robotic sensors to quickly glide over lines of Braille text. The robot was able to read Braille at 315 words per minute with almost 90% accuracy.
Although robotic Braille readers have not been developed as assistive technology, the high sensitivity required to read Braille makes them an ideal test for developing robotic hands or prosthetics with sensitivity similar to human fingertips, the researchers say. Results are reported in the journal. IEEE Robotics and Automation Correspondence.
Human fingertips are very sensitive and help us gather information about the world around us. Our fingertips help us detect small changes in the texture of materials or know how much force to use when grasping an object. For example, picking up an egg without breaking it or picking up a bowling ball without dropping it.
Reproducing that level of sensitivity in a robotic hand in an energy-efficient manner is a huge engineering challenge. In Professor Iida Fumiya's laboratory at the Cambridge School of Engineering, researchers are developing solutions for this and other technologies that humans find easy to find but robots find difficult.
“The softness of human fingertips is one of the reasons why we can grasp objects with just the right amount of pressure,” said Parth Potdar, an undergraduate student at the Cambridge School of Engineering and first author of the paper. “For robotics, softness is a useful characteristic, but it also requires a lot of sensor information, and having both pieces of information simultaneously is tricky, especially when dealing with flexible or deformable surfaces.”
Braille requires high sensitivity because the dots of each representative letter pattern are so close to each other, making it an ideal test for robotic 'fingertips'. Researchers have developed a robotic Braille reader that uses off-the-shelf sensors to more accurately replicate human reading behavior.
“There are existing robotic Braille readers, but they only read one letter at a time, which is different from the way humans read,” said co-author David Hardman of the College of Engineering. “Existing robotic Braille readers work in a static way: touch one letter pattern, read it, lift it from the surface, move it, lower it to the next letter pattern, and so on. “I want something efficient.”
The robotic sensor used by the researchers is equipped with a camera 'at the tip of the finger' and reads information by combining information from the camera and sensor. “This is a challenge for roboticists because there is a lot of image processing that needs to be done to remove motion blur, which is time-consuming and energy-consuming,” Potdar said.
The team developed a machine learning algorithm that allows the robotic reader to 'blur' the image before the sensor attempts to recognize the character. They trained the algorithm on a set of sharp Braille images with a fake blur effect applied. After the algorithm learned how to deblur letters, it used a computer vision model to detect and classify each letter.
After the algorithm was integrated, the researchers tested the readers by quickly sliding them along a row of Braille letters. The robotic Braille reader can read 315 words per minute with 87% accuracy, which is twice as fast and more accurate than a human Braille reader.
“Considering that we used fake blur in the train algorithm, we were surprised at how accurate it was for reading Braille,” Hardman said. “We found a good balance between speed and accuracy, and this holds true for human readers as well.”
“Braille reading speed is a good way to measure the dynamic performance of a tactile sensing system, so our findings can be applied beyond Braille to applications such as surface texture or slip detection in robotic manipulation,” Potdar said.
In the future, the researchers hope to expand this technology to the size of a humanoid hand or skin. This research was partially supported by the Samsung Global Research Outreach Program.