Sometimes it’s hard to understand what somebody is saying when they’re wearing a mask – often, that problem is already an issue when the listener is reliant on lip-reading or is hard of hearing or even deaf.
However, problems like that lead to great solutions and ideas, like this wearable captioning device Kevin Lewis (@_phzn) has shown on Twitter.
Masks making it hard to understand people either audibly or because you rely on lip reading? Just got this hacked together – it displays my speech in real time with @DeepgramAI! pic.twitter.com/lPu4CZboIk
— Kevin Lewis (@_phzn) January 4, 2022
“The mask presents some interesting challenges – firstly, some people struggle to hear me with one of these on, but also some people rely on lip-reading and without the ability to lip-read, they will struggle to communicate with me,” says Lewis.
“When I go to events in the next few months or even a year, I’ll definitely be wearing a mask.
“So, I’ve built this. This is a little screen, powered by Deepgram speech recognition API that is presenting what I’m saying in real time, it’s going into this little mic here. And while I still need to work on the kind of packaging of it, I think this is going to be super useful.”
This wearable captioning device absolutely rules. It essentially provides TV-style close captioning through an on-person device.
Lewis is a senior developer advocate for Deepgram, an end-to-end AI speech recognition service that’s focused on providing superfast transcripts in realtime. If these videos are any indicator, Deepgram is not messing around. It’s used by wide range of organisations, including NASA, Tethr and Khoros.
As requested by a bunch of you there is now live translation built into this @DeepgramAI-powered wearable captions device! Next step will take a few days but it is the device casing so it can be more easily worn. Thanks for the kind words. Deepgram info at https://t.co/6NA2qQMVTT pic.twitter.com/SrEiFjFJCJ
— Kevin Lewis (@_phzn) January 6, 2022
It’s honestly quite remarkable how compact such a device is too, providing big, understandable captions on a pocket-mounted display at the chest of the user, near eye-level. It supposedly also only cost Lewis about £70 ($132.45) with readily-available parts, comprised of a Raspberry Pi and a Hyperpixel screen.
“Gonna make it more wearable and less ‘just about staying upright on my pocket’,” Lewis added on Twitter.
It’s even capable of translating languages in real-time, too (through iTranslate). That’s amazing. It’s not the first of its kind, but it’s definitely one of the first developed in this current form.
In the future, it’d be great to see this kind of reliable close-captioning at events, perhaps powered via phones and not reliant on a cloud-based API. There’s absolutely interest in this tech, with Twitter users praising the creativity of the device.
“My deaf wife would love this. I’m used to life with subtitles, so I’d love it too. Bravo,” one Twitter user wrote. “As someone who often wished for subtitles even before the pandemic, this is awesome,” wrote another.
It’s certainly an achievable reality on a wide scale, considering the Pixel 6 and the Pixel 6 Pro have offline speech-to-text software.
Technology rules.