Tag Archives: app

Disability Innovations: Talkitt app helps disabled people use their voice

Disability Innovations is a blog series that gathers some of the most interesting new products and services that aim to make disabled people’s lives easier. We hope it will inspire more innovation in the disability field.

What is Talkitt?

Talkitt is a voice to voice app which aims to enable people with motor, speech, and language conditions to communicate freely and easily using their own voice. It works by interpreting an individual’s pronunciation of words into understandable speech. Talkitt recognises the user’s vocal patterns, translates words from any language and then speaks them aloud via an app.

Approximately 1.5% of the population in the western world has some form of difficulty communicating as a result of medical conditions including: Motor Neurone Disease, Cerebral Palsy, Stroke, Brain Damage and Autism. Current communication solutions include using eye and head tracking systems or using other body movements, but none of these enable the user to really communicate in the traditional sense, by using their voice. What makes Talkitt different is that it does not rely on expensive technology, simply a smartphone app and the user’s own voice. Talkitt wants to help increase participation in everyday activities, particularly when out and about and communicating with new people.

What’s behind the idea?

Talkitt’s Chief Executive (CEO) Danny Weissberg came up with the idea in Israel back in 2012 after his grandmother had a stroke that severely impaired her speech. As a software engineer himself, Danny wanted to come up with a solution to help his grandmother. The more he explored the issue and spoke to speech and occupational therapists about it, the more he was convinced that there was a need for this sort of solution.

In 2012 Voiceitt, the company behind Talkitt was launched as a joint venture between software engineers, technology officers and senior Occupational Therapists (OTs) to combine their technology background with the OTs experience and user insight. Inspired by Danny’s grandmother and working to use “technology for good”, Talkitt hopes to break down barriers between disabled people and their communities, and enable them to communicate and participate fully.

How does it actually work?

Talkitt is not your standard speech recognition app. The software works by creating a dictionary of sounds and their meanings, learning each individual’s way of speaking. First the user has to go through the calibration stage. This is when the app learns the user’s speech patterns by getting them to record a selection of set words and phrases dependent on their cognitive ability to create their personal dictionary. This dictionary helps the system to map what a person is saying to enable an accurate and personalised interpretation. Then the user can move on to the recognition stage where the app is able to interpret their individual pronunciation of words. The user speaks a word, it is associated with a word on the software, and the app speaks the interpretation. For example, the app can recognise the pronunciation of “o-ko-la” and the software will translate it to “chocolate”.

Diagram-of-how-talkitt-works

In techno terms, the approach is based on robust multi-domain signal processing, and an appropriate pruning of dynamic voice pattern classifier search space. Talkitt uses a smart system which uses machine learning so that the system continues to learn adaptively with the user over time to build and enhance the user’s personal dictionary. As the system is not language dependent but speaker dependent, there are no language restrictions as it works based on pattern recognition software. As it interprets vocal patterns, it can even interpret made up words or phrases such as an autistic child may use to communicate. Talkitt hopes that future developments will also enable the app to be used for degenerative conditions, by recording the user’s own voice to use later.

What’s next for Talkitt?

Talkitt is not yet on the market and is currently under development. They are testing the first release with users, working in partnership with disability charities across the globe. They are also in the process of collecting as many audio recordings as possible to help populate their audio recording database and inform the algorithm they are developing. The aim is to release a version one in early 2016. This will be a basic version of the system with a limited vocabulary for the user’s personal dictionary and will be able to interpret a few calibrated words. Version two is due to be released in 2017 and will incorporate the adaptive learning (without calibration) and continuous speech features as well as having an extended vocabulary.

Once released, Talkitt will run on a ‘freemium model’ with an initial period after the launch where it will be available for anyone to download for free. After that it will run on a monthly subscription fee of $20, around £12. The software can currently run on tablets and smartphones, but eventually they hope to offer it for wearable devices including smart watches and Google Glass. They also hope to integrate it into other devices such as a wearable necklace or wheelchair and browsers. Using this technology that is not currently found in mainstream speech engines to improve existing speech recognition technology. Ultimately, they hope that the data their app will gather in their speech database will help medical research centres and universities to further their research and understanding into neurological and cognitive diseases.

Talkitt has had many successes to date, including running a successful crowdfunding campaign which secured $87,000 (around £55,000) of funding to continue development and testing. They have also won some prestigious awards and competitions, including the Philips Innovation Fellows and Verizon Powerful Answers Competition, the Wall Street Journal Startup Showcase, Deutsche Telekom Innovation Contest and the Orange 4G Innovation Lab, to name but a few!

Why we love it!

What makes Talkitt really special is that unlike existing alternatives, it is a form of alternative communication that is based fully on the user’s voice. Everyone should have the chance to communicate in a natural way, and Talkitt aims to enable traditional communication using a person’s voice in a truly personalised way that’s not offered by text to speech systems. Talkitt will also offer an inexpensive alternative to traditional communication devices, and can cut waiting times as it can be downloaded instantly. The future looks bright for Talkitt, and we’re excited to see how this venture develops!

This blog is for information only. Scope does not endorse this product or service. We try to make sure our information is up to date and accurate at the time of publishing.

To tell us about a Disability Innovation, please email innovation@scope.org.uk

Disability Innovations: A global system of inclusive communication

Disability Innovations is a blog series that gathers some of the most interesting new products and services that aim to make disabled people’s lives easier. We hope it will inspire more innovation in the disability field.

What is BlueAssist?

BlueAssist is a free-to-use system of communication to help disabled people engage with their community. BlueAssist aims to help anyone for whom communication is hard, whatever the reason, to engage freely (and for free!) with those around them to communicate their needs and is a simple way for disabled people to ask for assistance when they are out and about. Write your questions on the BlueAssist Card or app, such as ‘Please can you tell me how to get to the station’ or ‘Please look at me when talking so I can lipread’. Present your card to a trained member of staff, or member of the public, to make your questions and requests known to them. They are then able to quickly and easily understand what is needed and know how best to support you.

What’s behind the idea?

When you’re out and about and you get lost, or encounter a problem you ask those around you for help. But if you have a difficulty communicating this may not be something you can easily do. People may not be able to understand what you’re asking for, or may even ignore you because they don’t know how to help. Research estimates that only 1% of those travelling by train who need assistance actually ask for it. The idea behind BlueAssist is to create a universally recognisable symbol, which disabled people can use to get help and support.

At present there is no comprehensive ‘Blue Badge’ equivalent for people with communication and learning difficulties. BlueAssist want to change this, so that every person who finds it hard to ask for things, or communicate when out and about can ask for help to be as independent as possible. The BlueAssist symbol is there to help the public understand that the person asking for help may not be able to communicate easily and to provide an effective and mutually beneficial system of communication.

The idea began in Belgium in 2000, where the original BlueAssist card was trialled with a group of people with learning disabilities to help them to travel independently and ask for help when needed. In 2013 the team from Belgium presented BlueAssist at the Autism Show in London, and Barbara Dewar, now Chief Executive Officer (CEO) of BlueAssist UK Ltd was so impressed with the system that she bought it back to the UK and launched a charity in the same name. The BlueAssist movement has now spread worldwide to the USA, Holland and Germany.

How does it work?

What started life as a simple card has now developed into a family of different apps to make the system more practical and easier to use, although the original BlueAssist card is still going strong, and available to download online for free! The BlueAssist Light app is an smartphone version of the original card, and can be used to show messages on screen. With the BlueAssist Light version you get access to two messages, one a general request for help which you can connect to an emergency telephone number, the second a message you can edit change as often as you want.

phone using the BlueAssist app
Phone using BlueAssist app

An important part of the BlueAssist ethos is that the basic system is free to anyone who wants to use it, and as such the BlueAssist Light app is free to download from Google Play or the Apple Store. However, the creators of BlueAssist have also developed a subscription based version of the app, which offers additional functions, such as multiple pages, and a range of messages and images. In addition to the full version of the BlueAssist app , they have also developed 3 additional apps, which can be both used together or in isolation.

The first is a Calendar app which uses visual cues to create clear daily plans and outline different activities and when they are due to happen. The second is a Photo phone book, which is a pictorial phone book, to create an easy way to call contacts. Numbers can also be barred at particular times, for example when staff go off shift, to help the user know when it is the right time to call. Finally the Photo Gallery app helps users not only to share their experiences but also to help sequence tasks. They have even be used to create albums for pictorial shopping lists, or visual cues for what to pack from swimming, and which order to put clothes on when getting dressed. All 4 apps plus cloud storage are available for a small monthly fee of £10.92.

Who are they working with?

Since its UK launch BlueAssist has received widespread support from disability organisations such as Mencap, The National Autistic Society, and has even been trialled at Scope’s own Beaumont College. They are also working with 28 train operators across the UK to adopt BlueAssist, including the Department for Transport and First TransPennine Express, who have launched the system with BlueAssist cards available at all their stations. Aside from transport, BlueAssist are also working with museums. And even the Houses of Parliament, to aid assistance and create ‘do’s and don’ts’ guidance sheets for staff, to help assist someone who presents a BlueAssist card.

The big dream for BlueAssist is to become a globally recognised symbol for anyone needing assistance to be able to confidently ask anyone around them for help and get it. They want the system to be simple and easy for everyone to use, not just trained staff, but also members of the public more widely. If disabled people feel confident to ask for help, and the public feel confident to give it, then everyone’s a winner!

Why we love it!

The social model of disability is really important to Scope, and we love how BlueAssist is helping to remove barriers in society that restrict communication for disabled people. Everyone needs help at some point, whether it is finding the right bus station or asking for directions, and if you don’t ask, you don’t get! BlueAssist both facilitates and promotes independence for disabled people, by giving them a really simple tool to help them communicate. Plus it may even help prevent some of those ‘awkward moments’ we saw as part of Scope’s End the Awkward campaign along the way!

This blog is for information only. Scope does not endorse this product or service. We try to make sure our information is up to date and accurate at the time of publishing.

To tell us about a Disability Innovation, please email innovation@scope.org.uk.

Disability Innovations: App lets volunteers lend their eyes to assist visually impaired people

Disability Innovations is a blog series that gathers some of the most interesting new products and services that aim to make disabled people’s lives easier. We hope it will inspire more innovation in the disability field.

What is Be My Eyes?

Be My Eyes is an iPhone app that connects visually impaired people with volunteer helpers from around the world via live video chat. The app allows sighted volunteers to ‘lend’ their eyes to a visually impaired person to assist them with tasks such as navigating their surroundings and identifying food labels. Hans Jørgen Wiberg, the app’s inventor says that “being visually impaired myself I know the challenges blind people face. It’s my hope that by helping each other as an online community, Be My Eyes will make a big difference in the everyday lives of blind people all over the world.”

What’s the idea?

From his personal experience, and his work for the Danish Blind Society, Hans Jørgen learned first hand that as resourceful and independent as people with visual impairments are, everyone needs help once in a while! As a non-profit app, Be My Eyes hopes to provide that help by creating a community all about contributing to and benefiting from small acts of kindness.

The idea behind the app is simple. To ‘ask for help’ a visually impaired user touches a button to request the assistance of a volunteer (the app itself is designed around users using iPhone VoiceOver technology).  The sighted volunteer then receives a notification for help which they can either ignore or accept. If they accept, the app uses live video technology to enable the visually impaired user to direct the camera on their smart phone to stream an image of the object they need to see to the sighted volunteer’s phone. The volunteer can then describe the objects or surroundings, and answer questions such as ‘when is the use by date on this carton of milk?’ Requests for help can currently only be made between 7am and 10pm and sighted volunteers can earn points and promotions to new levels of experience, the more people they help.

The power of the crowd

Not only does the app itself make the most of ‘the power of the crowd’ to help its visually impaired users, but it is also uses a network of volunteers to contribute to the technical side of things! To date, the app has already been ‘crowd translated’ into over 34 languages, (a Crowd sourcing platform which allows volunteers to contribute and approve translations online). The app itself is also ‘open source’, which means that the source code or how the app is built in coding is freely available online, for anyone to access, use, change or distribute. Be My Eyes hope that in doing so, more talented people will have the chance to improve the apps software and to continue to develop it by contributing with new features or fixing bugs.

The app is free to download on iOS and an android version is in development.  Be My Eyes has gathered over 160,000 sighted volunteers and 16,000 visually impaired users to date (that’s 10 volunteers to every user) and has made over 56,000 ‘helping connections’ in more than 80 languages.

Why we like it

We think that Hans Jørgen’s idea of a network of ‘good Samaritans’ to help others is a really interesting take on the idea of the ‘shared economy’. Be My Eyes has the potential to make a real difference in helping disabled people to live more independently by offering a new type of flexible support on hand, as and when required. We also love how it uses technology to create micro volunteering opportunities in such an easy, and informal way- you could even be ‘volunteering’ during your commute!

This blog is for information only. Scope does not endorse this product or service. We try to make sure our information is up to date and accurate at the time of publishing.

To tell us about a Disability Innovation, please email innovation@scope.org.uk.

Touch screen devices and disabled children

Guest post from Elvia Vasconcelos, Includer, North London

 

I first saw the potential of touch screen devices a few months ago when Mary, mum of Julian, a three-year-old with global development delays, handed us an Ipad for us to play. Both I and Julian were thrilled and excited when we saw it lighting up. Julian knew what to do and clicked on the application he wanted to use. I followed in wonder.

So what is an app?

“Application software, also known as an application or an ‘app’, is computer software designed to help the user to perform singular or multiple related specific tasks.” Wikipedia

They can be divided into Web apps and Mobile Apps. I will be referring to the later ones as they are designed to run on smart phones, tablet computers, portable media players and other personal digital assistants.

Out of curiosity, App was voted Word of the Year in 2010 by the American Dialect Society. In 2009 the word was Tweet, the word of the past decade was Google (as a verb) and in the 90s it was Web. There is no escaping it!

Potential of touch screen devices for disabled children

Julian is three and although he picks up on everything his mum says he can’t speak. It is still yet to be seen if he will be able to write in the conventional pen and paper way but when he started the Farm animals app he did spell. I was amazed at the control he had on the device and how intuitive it all felt. That was when I first realised the true potential of the ipad, and more generally touch screen devices for disabled children and how much of an impact they will have on special needs education.

The range of applications in special needs is wide and is constantly growing. A very popular one is AutoVerbal talking soundboard. It’s a text-to-speech program developed for non-verbal people with picture buttons that speak pre-programmed messages (such as “My name is Julian”) and it also has the type anything function that allows for more advanced users to carry on conversations. I found a lot of very enthusiastic reviews on the itunes website from people that have been using these apps with their autistic children and it is easy to see the correlation of these programmes with the already instated communication tools in educational settings such as the Pecs (pictures exchange system), Visual timetables and Makaton. I found a couple of websites very helpful in tackling the 30.000 plus apps available: www.SNApps4kids.com and oneplaceforspecialneeds.com.

Overall, apps can be divided into the areas of communication and speech; language and literacy; behaviour, schedules and social cues; cause/effect. There are also tones of games, musical activities and movies.

Also for medical purposes iBiomed is a mobile software application developed for parents of special needs children, to help in managing the complexities involved with their care. It is handy for all medical conditions, but is even more useful for: Autism, Asthma, Allergies, ADHD, Cerebral Palsy, Multiple Sclerosis, Diabetes, Alzheimer’s, Fibromyalga, Migraines, Depression, Bipolar, Schizophrenia, OCD. It’s a free app and there is also an online version if you don’t have any of the touch screen devices. It seems like a very good tool and I have already signed up for a test run.

Happy apping, finger tapping!