New robotic sensor uses AI to read Braille at 2x the speed of humans

Human fingertips are highly sensitive, allowing us to gather detailed information about the texture of a material or the force required to handle objects, like picking up an egg delicately or gripping a bowling ball securely.

Replicating this level of sensitivity in a robotic hand presents a significant engineering challenge, particularly in achieving energy efficiency.

The research team from the University of Cambridge has developed a robotic sensor that incorporates artificial intelligence techniques to read Braille at speeds roughly double that of most human readers. Researchers used machine learning algorithms to teach a robotic sensor to quickly slide over lines of braille text. The robot could read the Braille at 315 words per minute at nearly 90% accuracy.

While the robot Braille reader wasn’t initially created as assistive technology, the researchers highlight that its high sensitivity, necessary for Braille reading, positions it as an excellent test platform for developing robotic hands or prosthetics with sensitivity levels comparable to human fingertips.

Co-author David Hardman from the Department of Engineering said, “There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read. Existing robotic braille readers work statically: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something more realistic and far more efficient.”

The robotic sensor employed by the researchers features a camera in its ‘fingertip’ and combines information from the camera and other sensors to read Braille. One significant challenge in this process is handling motion blur in the images, which requires extensive image processing, consuming time and energy.

To address this issue, the team developed machine learning algorithms that enable the robotic reader to “deblur” the images before attempting to recognize the Braille letters. The algorithms were trained on sharp images of Braille characters with artificial blur applied. After learning to deblur the letters, a computer vision model was employed to detect and classify each character.

Upon incorporating these algorithms, the researchers tested the robotic Braille reader by sliding it rapidly along rows of Braille characters. The results were impressive, with the robotic reader achieving a speed of 315 words per minute at an accuracy of 87%. This performance is twice as fast and nearly as accurate as a human Braille reader, showcasing the potential of advanced machine learning techniques in enhancing the capabilities of robotic systems.

Hardman said, “Considering that we used fake blur to train the algorithm, it was surprising how accurate it was at reading Braille. We found a nice trade-off between speed and accuracy, which is also the case with human readers.”

Journal Reference:

1. Parth Potdar et al. ‘High-Speed Tactile Braille Reading via Biomimetic Sliding Interactions.’ IEEE Robotics and Automation Letters (2024). DOI: 10.1109/LRA.2024.3356978