New AI-powered app translates sign language into speech in real time

Read Article

A Netherlands-based start-up has developed an artificial intelligence (AI) powered smartphone app for deaf and mute people, which it says offers a low-cost and superior approach to translating sign language into text and speech in real time. The easy-to-use innovative digital interpreter dubbed as “Google translator for the deaf and mute” works by placing a smartphone in front of the user while the app translates gestures or sign language into text and speech.

The app, called GnoSys, uses neural networks and computer vision to recognise the video of sign language speaker, and then smart algorithms translate it into speech. Affordable and always available interpreter services are in huge demand in the deaf community. Every day thousands of local businesses around the globe face problems with providing their services to deaf, said Konstantin Bondar, Co-Founder & CTO of Evalk, the company which developed the app.

According to the National Deaf Association (NAD), 18 million people are estimated to be deaf in India. The app is expected to hit the Indian market next year, Roman Wyhowski Founder & CEO Evalk told PTI. The new application can find use in a B2B setting, where businesses who want to employ deaf and mute employees can use it to convey employee messages to the end consumer, according to the company.

It will help drive inclusivity at the workplace by removing communication barriers between the disabled and able, Wyhowski said. “Deaf people do not have that many options for communicating with a hearing person, and all of the alternatives do have major flaws. Interpreters aren’t usually available, and also could be expensive,” Wyhowski said. “Pen and paper approach is just a bad idea: it’s very uncomfortable, messy, time-consuming for both deaf and hearing person,” he said.

Messengers and texting, Wyhowski noted, are slightly better, but they do not solve the problem, which is translation, and do not offer easy confident and comfortable way to communicate. He said the translation softwares in the market are either slow or expensive, or rely on old technology which does not allow scaling to another markets outside country of origin.

“By comparison, we offer a compellingly fast, easy, comfortable and economical solution,” Bondar said. Showcased in the Netherlands recently, the app can be used on multiple devices such as smartphones, tablets, laptops, or PCs. It can translate as quick as the person speaks, translate any sign language and can be plugged into many products, such as video chat applications, AI assistants, etc.

The pocket interpreter for the deaf relies on superior new technology: AI and neural networks. All the translation happens in the cloud. It just requires a camera on the device facing the signing person, and a connection to the internet, said Bondar. “Using only camera on your device, our system provides the same services only an experienced sign language interpreter could do,” he said. “All the translation is done by algorithms so that means we can differentiate on price as well, offering an inexpensive translation service which is handy and gives a lot of benefits to both deaf people, and businesses and service providers,” Wyhowski added.

The service allows the user to pay for the day, for the week, or for the whole month costing one, four and 11 US dollars respectively, according to the company. The cost is USD 150 per month for small businesses, and USD 450 a month for medium and large-sized businesses. The plan, Wyhowski noted, is to have a free app, which has limited amount of phrases per month/week you can use, just enough to satisfy the basic communication needs of deaf people with low income.

Working in collaboration with India Accelerator, the start-up is in touch with NAD and is gathering sign language data for India, he said. “We have partnered with them already in order to know better how we can help deaf people in India,” Wyhowski said. “Plus we plan to get in touch with Indian government, in order to know if some of the government spending on deaf people can be used to provide them with digital interpreter service, which can lead to higher employment of the deaf, and social equality,” he said.

If you have an interesting article / experience / case study to share, please get in touch with us at [email protected]

Advertisement