Captioning phone calls for hard-of-hearing users

My hard-of-hearing mother is moving into a smaller place where she is going to need a cell phone that can provide captions for phone calls. Currently she has a land line device that uses a captioning service, but going forward she won’t have the ability to use that for long-distance calls. She doesn’t have a smart phone currently.

There are quite a few apps listed in the App Lounge that provide this functionality, one way or another, but one never knows for sure whether it will work until one tries it. Or how well it works in practice.

Has anyone used any captioning system with a device running /e/OS and had success? If so, what system did you use?

I’d love to recommend a de-Googled device to my mother, but I would love to hear confirmation that some system (any system) works.

Thanks in advance!

Jim Garrett
Boston area, USA

Regain your privacy! Adopt /e/OS the deGoogled mobile OS and online servicesphone

1 Like

After some days of research, allow me to report what I’ve found so far. TLDR: this is an area of unmet need that could probably be met with some effort, i.e., it’s an opportunity.

I haven’t found any free/libre/open-source Android tools. There’s a good FLOSS system for the desktop: https://github.com/abb128/LiveCaptions. I installed it and it appears to work very nicely. But it hasn’t been ported to Android. Computing power is a concern (because the first thing the app does is a performance test) so it might need to be optimized in some way.

In the proprietary world, it seems the predominant captioning system on Googled Android is Google’s. According to what I’ve read online, it’s among the best-performing systems. It’s built-in to Google’s Android and only needs to be turned on to be used in phone calls. Not surprisingly, there aren’t a lot of developers rushing to develop captioning phone apps when a respected participant’s is already installed. There are numerous apps that add captions to videos, or take notes from the audio of recordings. Thus, that small slice of Android that is de-Googled (i.e., us) is not served.

There seems to be something special about the phone app in Android. One app, CaptionCall, requires calls to be “forwarded” to the CaptionCall app.

Anyway, I installed every app I could find in the App Lounge that looked remotely applicable, and only one worked: Ava. This essentially captions whatever the microphone hears. But it doesn’t work directly with the phone app. I suppose one could start a call, send it to speaker phone, then go to Ava and watch the words flow by (yours and theirs). But this is a clumsy solution that could pose a challenge to people who are not that adept with their phone. Also, receiving a call could be uncomfortable: “Hold on while I turn on captioning!” (fiddle fiddle fiddle).

In short, I think currently there is no way to do first-class phone captioning on /e/OS. However, FLOSS software exists in the world (if it is efficient enough to run on a mobile device), and the phone app is itself FLOSS, so in principle there is a visible way forward to add general live captioning as well as captioned phone calls, if developers choose to pursue it.

I suggest that this is a task worthy of pursuing.

1 Like

thanks for assembling your research. You could post it to the gitlab backlog for it to ever be picked up.

There’s atm one related issue, but isn’t your use-case of real-time call-transcription (mostly an abandoned thread).

You could ask Ava if they considered any kind of Dialer integration or releasing their own?