Reading together is one of the best parts of childhood, but for Deaf children with hearing parents, story time can be a challenge. The StorySign app set out to help by turning printed children’s books into signed stories.
How StorySign Works
Launched on December 3, 2018, StorySign is an Android app that uses image recognition and optical character recognition (OCR) to read the text on a book page. A friendly animated avatar named Star then signs the words in real time as pages are turned. Image recognition helps the app cope with awkward camera angles, and OCR improves its accuracy.
Who Was Behind It
StorySign was a collaboration between Huawei, the European Union of the Deaf, Penguin, and Aardman, the animation studio. The involvement of the European Union of the Deaf is important, because it connects the project to real Deaf community input.
Languages and Availability
At launch the app was available on Google Play and Huawei’s AppGallery in 10 European markets. It supported several sign language variants, including British, French, German, Italian, Spanish, Dutch, Portuguese, Irish, Belgian Flemish, and Swiss-German. It officially supported Android 6.0 and later and was optimized for Huawei phones with AI hardware, such as the Mate 20 Pro.
The Limits at Launch
It is worth being honest about the early limitations. At launch, StorySign supported just one book, Where’s Spot? by Eric Hill, and users needed a physical copy of it. One reviewer felt the app launched prematurely. More books were planned to be added one at a time, so it is best to check the current state of the app before recommending it to families.
What It Tells Us About Sign Language Technology
StorySign is an early example of technology aimed at sign language access, and it raises questions that remain open today. Who should be involved in building signing avatars? How natural can an avatar look and feel? And how do we make sure Deaf children learn from rich, human signing as well as apps? Many in the Deaf community argue that avatars should support, not replace, Deaf teachers, interpreters, and parents signing with their children.
The Takeaway
StorySign shows both the promise and the limits of sign language technology. It is a creative idea that made story time more accessible, and it is a useful case study for anyone following Deaf technology and early literacy.
Interested in International Sign or the global Deaf community? Follow OneDeafWorld.com for more coverage of IS, accessible technology, and events from around the world.