Gesture Reader Lets Blind Decipher Documents

//

If computer interfaces aren’t equipped with gesture-based technologies or screen readers, and documents don’t include Braille, a blind person can be limited in what material they can access.

That’s why researchers from the University of Maryland, Baltimore County have designed a new interaction system called “Access Lens,” a new form of technology that could potentially assist blind people with a more autonomous lifestyle. The system utilizes computer vision-based tracking that enables the blind to use gestures to access paper documents and other objects, such as bills, product packages, ATM interfaces, restaurant menus, charts and maps that don’t cater to the visually impaired.

NEWS: Retinal Implant Brightens Future For Blind

Access Lens uses a camera to scan documents or interfaces and software for recognizing text and tracking gestures. As text is cross checked against a dictionary file, a computerized grid overlays the document, to which the user runs their finger over as the system reads back content or provides directional voice commands.

“Since using gestures works well for exploring touch screen interfaces, why can’t we use similar gestures to explore objects in the real world?” said Shaun Kane, an assistant professor in the Department of Information Systems at the University of Maryland, who co-developed the system. “Access Lens uses a camera to track a user’s gestures and provide audio feedback about objects in the user’s environment.”

BLOG: World’s First Braille Smartphone Rises Up

Kane and his colleagues have created a wearable prototype in the form of a pendant camera and a mobile PC.

“Currently Access Lens works best with paper documents, but we are exploring ways to adapt it to other objects in the environment,” he added. “We’re also working on improving the hardware and making it more portable. In the future, we are hoping to have smaller versions of the hardware that could utilize a wearable camera.”

Kane and his fellow researchers presented their work at the CHI conference in Paris earlier this week.

Credit: Shaun Kane, University of Maryland, Baltimore County