Up until now the best way to bridge the communication gap between the deaf or hard of hearing and those that can hear has been through both parties learning sign language. But a new sign language translator that converts signs into spoken and written language — and vice versa — could open up a whole new range of communication, even for those who don’t understand sign language.
Developed by a team of researchers in China, the translator users a computer and a Kinect camera that recognizes signing gestures, then gives a spoken and written translation of English and Mandarin for the non-deaf. Conversely, the system can take a hearing person’s spoken words and translate them into accurate signs carried out by an on-screen avatar.
Researchers tout the translator as a cost-effective, efficient prototype that works in real time. As seen in this video, the developers envision this technology being used at doctors offices and other areas, like airport kiosks, that could provide deaf people with new communicative jobs and opportunities.
In effort to fine tune the translator’s algorithms and gather real world data, a group of teachers and students from Beijing Union University joined the project. According to Phys.org, a young woman named Dandan Yin told researchers the project fulfilled her childhood dream “to create a machine for people with hearing impairments.”
Microsoft Research Connections, the Chinese Academy of Sciences, Beijing Union University, and Microsoft Research Asia, all made crucial contributions to the project. Here’s hoping that researchers forge ahead with the prototype, as it could really open up a whole new world of possibilities for the estimated 360 million people worldwide who are deaf or hard of hearing.
Credit: Microsoft Research