Voiceit app on tablet with words " I love you"

Disability Innovations: Talkitt app helps disabled people use their voice

Disability Innovations is a blog series that gathers some of the most interesting new products and services that aim to make disabled people’s lives easier. We hope it will inspire more innovation in the disability field.

What is Talkitt?

Talkitt is a voice to voice app which aims to enable people with motor, speech, and language conditions to communicate freely and easily using their own voice. It works by interpreting an individual’s pronunciation of words into understandable speech. Talkitt recognises the user’s vocal patterns, translates words from any language and then speaks them aloud via an app.

Approximately 1.5% of the population in the western world has some form of difficulty communicating as a result of medical conditions including: Motor Neurone Disease, Cerebral Palsy, Stroke, Brain Damage and Autism. Current communication solutions include using eye and head tracking systems or using other body movements, but none of these enable the user to really communicate in the traditional sense, by using their voice. What makes Talkitt different is that it does not rely on expensive technology, simply a smartphone app and the user’s own voice. Talkitt wants to help increase participation in everyday activities, particularly when out and about and communicating with new people.

What’s behind the idea?

Talkitt’s Chief Executive (CEO) Danny Weissberg came up with the idea in Israel back in 2012 after his grandmother had a stroke that severely impaired her speech. As a software engineer himself, Danny wanted to come up with a solution to help his grandmother. The more he explored the issue and spoke to speech and occupational therapists about it, the more he was convinced that there was a need for this sort of solution.

In 2012 Voiceitt, the company behind Talkitt was launched as a joint venture between software engineers, technology officers and senior Occupational Therapists (OTs) to combine their technology background with the OTs experience and user insight. Inspired by Danny’s grandmother and working to use “technology for good”, Talkitt hopes to break down barriers between disabled people and their communities, and enable them to communicate and participate fully.

How does it actually work?

Talkitt is not your standard speech recognition app. The software works by creating a dictionary of sounds and their meanings, learning each individual’s way of speaking. First the user has to go through the calibration stage. This is when the app learns the user’s speech patterns by getting them to record a selection of set words and phrases dependent on their cognitive ability to create their personal dictionary. This dictionary helps the system to map what a person is saying to enable an accurate and personalised interpretation. Then the user can move on to the recognition stage where the app is able to interpret their individual pronunciation of words. The user speaks a word, it is associated with a word on the software, and the app speaks the interpretation. For example, the app can recognise the pronunciation of “o-ko-la” and the software will translate it to “chocolate”.

Diagram-of-how-talkitt-works

In techno terms, the approach is based on robust multi-domain signal processing, and an appropriate pruning of dynamic voice pattern classifier search space. Talkitt uses a smart system which uses machine learning so that the system continues to learn adaptively with the user over time to build and enhance the user’s personal dictionary. As the system is not language dependent but speaker dependent, there are no language restrictions as it works based on pattern recognition software. As it interprets vocal patterns, it can even interpret made up words or phrases such as an autistic child may use to communicate. Talkitt hopes that future developments will also enable the app to be used for degenerative conditions, by recording the user’s own voice to use later.

What’s next for Talkitt?

Talkitt is not yet on the market and is currently under development. They are testing the first release with users, working in partnership with disability charities across the globe. They are also in the process of collecting as many audio recordings as possible to help populate their audio recording database and inform the algorithm they are developing. The aim is to release a version one in early 2016. This will be a basic version of the system with a limited vocabulary for the user’s personal dictionary and will be able to interpret a few calibrated words. Version two is due to be released in 2017 and will incorporate the adaptive learning (without calibration) and continuous speech features as well as having an extended vocabulary.

Once released, Talkitt will run on a ‘freemium model’ with an initial period after the launch where it will be available for anyone to download for free. After that it will run on a monthly subscription fee of $20, around £12. The software can currently run on tablets and smartphones, but eventually they hope to offer it for wearable devices including smart watches and Google Glass. They also hope to integrate it into other devices such as a wearable necklace or wheelchair and browsers. Using this technology that is not currently found in mainstream speech engines to improve existing speech recognition technology. Ultimately, they hope that the data their app will gather in their speech database will help medical research centres and universities to further their research and understanding into neurological and cognitive diseases.

Talkitt has had many successes to date, including running a successful crowdfunding campaign which secured $87,000 (around £55,000) of funding to continue development and testing. They have also won some prestigious awards and competitions, including the Philips Innovation Fellows and Verizon Powerful Answers Competition, the Wall Street Journal Startup Showcase, Deutsche Telekom Innovation Contest and the Orange 4G Innovation Lab, to name but a few!

Why we love it!

What makes Talkitt really special is that unlike existing alternatives, it is a form of alternative communication that is based fully on the user’s voice. Everyone should have the chance to communicate in a natural way, and Talkitt aims to enable traditional communication using a person’s voice in a truly personalised way that’s not offered by text to speech systems. Talkitt will also offer an inexpensive alternative to traditional communication devices, and can cut waiting times as it can be downloaded instantly. The future looks bright for Talkitt, and we’re excited to see how this venture develops!

This blog is for information only. Scope does not endorse this product or service. We try to make sure our information is up to date and accurate at the time of publishing.

To tell us about a Disability Innovation, please email innovation@scope.org.uk