An Ally for Access

Having to tap-tap-tap away to order a sandwich or check bags at a touchscreen kiosk can be annoying. For those who are blind or visually impaired, it can be excruciating.

Soon, instead of friends, family or even strangers bridging this technological chasm, a palm-sized portable robot could help. Toucha11y, created by a team including doctoral student Jiasheng Li and computer science Assistant Professor Huaishu Peng, can stick to and press buttons on a touchscreen, communicating with a user’s phone and allowing them to use accessibility features—like voiceover and zoom—to read the options.

Federal requirements that public kiosks include accessibility features haven’t solved the problem for many users, says Peng. Even ones that are technically compliant lack standard features, blind testers told him. “We thought, if there’s a universal way for [visually impaired] people to use these machines, we could lower the learning curve a bit.”

The current prototype looks like a tiny measuring tape with a camera, computer and suction cups. It takes photos to orient itself on the screen, then uses an extendable “tape” fitted with a touch probe to press buttons. The research team tested it on screens up to 37 inches wide and on six different interfaces, and users were able to complete tasks like ordering a bubble tea, specifying ice and sugar levels, in under 90 seconds.

In future iterations, Li and Peng plan to shrink Toucha11y (which combines “touch” and “accessibility”—the “11” stands for the letters between “a” and “y” in “accessibility”) to make it easier to carry in a purse or pocket, as well as add real-time menu-scanning capabilities.