Voice to Text feature using Open AI

As I was reading the Murena’s terms of use I discovered that the voice to text feature introduced with e/OS/ 3.0 is using the Open AI API. That means our voice is sent to Open AI so they can translate it to text.
Murena is anonymising the data but I think it is still someting we should know. We can give sensitive informations in those messages. And we are probably participating in the training of Open AI LLM models.
Open IA is a big tech company. It is not a transparent company, their models are not open source, we don’t know what they do with our data.
Users of Murena should be aware of that so I created this topic to be albe to discuss it with other users.

19 Likes

It’s really a shame that eOS use such a service whereas Murena promoted its services as “private, sustainable and fully open-source”. OpenAI doesn’t respect users privacy, exploits humans and natural resources in an unsustainable way and of course isn’t open-source.

Especially that voice to text can be made locally with efficient opensource app like this one (it use OpenAI models but on your phone):

10 Likes

I would like to add that HeliBoard (keyboard) in combination with Whisper provide a fully offline experience. Whisper needs to download the Whisper models from Hugging Face (~ 435 MB). This is the only use of the internet permission. Then voice recognition works completely offline (albeit a bit bumpy and not for all languages). There is a fast and slow model. In any case, it would only work for more recent devices.

It would be interesting to see if the murena / e/OS folks could team up with Mistral (https://mistral.ai/, based in Paris, France, more inclined to open source). The hay build the Mistral 3.1 model, which works quite well. To the best of my knowledge, there is no plan to support STT.

6 Likes

reality — an open, secure, de-Googled system is not open, not secure, and not a de-Googled system!? This is just a new anøm disguised as open source. I wouldn’t be surprised if the Tor nodes in Advanced Privacy are routed through servers controlled by Murena. I trust /e/ Foundation OS and the Murena company less and less

P.S. Regarding ‘anøm’ it was a joke—or maybe not. We’ll see if (or when) the authorities come knocking at our doors
30

1 Like

I am also very disappointed by this implementation. This is not what /e/ once stood for. Even more upsetting as there are offline alternatives, open source or not, that would not undermine trust in /e/ to such an extent.

A stupid, unnecessary shot in the foot. And what for? I just don’t get it…

1 Like

Apparently, the developers are paid too little, or on the contrary, the developers were paid a lot by those who benefit from this … :grimacing:

Did they seriously implement this using the web API?! That runs against all the reasons why people use /e/ and risks alienating current and potential future users.

Anonymising the IP address the data is sent from will not change the fact that:

  1. Voice recordings are actually biometric data
  2. Other sources with the same voice will make it trivial to de-anonymise the recordings and associate them with an identity
  3. The content of the audio recording, including all personal data (including any information about other people than the actual user whose permission they do not have) is transmitted to an organisation in the United States (and thus subject to the Cloud Act) with a former NSA head on the board.

If Murena wants to get into the government market in Europe, this is a seriously bad idea.

As others have mentioned, FUTO Voice and FUTO Keyboard are using a local Whisper model and work very well, even on older low-end devices.

A much better option would be to collaborate with FUTO to include these into /e/ to replace the default keyboard.

10 Likes

And what’s more, this feature is for Premium Murena Workspace users, as they write in GitLab releases. I think the Murena slogan should be. “Your data is our business”

This is a very disappointing information!
If I want to send my data to the US, I can use stock Android again…

5 Likes

Thank you a lot for your positive and supporting comments about our new /e/OS Voice-to-text!

Regarding its implementation in /e/OS, I’d like to explain a few things to explain why we have chosen an OpenAI STT API to implement it and how it’s going to evolve in the future:

  1. What we have learned from our experimentations with STT models that run locally on the smartphone for speech recognition:
  • they work quite poorly, they make a lot of mistakes in voice recognition
  • they are not able to mix languages (i.e. you have to preset one language before use and parts of what you say that is not in this language - that happens all the time - are not recognized)
  • they take a huge amount of memory to run (in the magnitude of hundreds of MB) + CPU overhead
  1. At some point it became clear that offline STT was a no-go unless we wanted to offer a degraded UX to /e/OS users. So we have looked for alternatives to implement this service, with Privacy constraints in mind. It soon became clear that OpenAI Whisper or OpenAI new GPT-4 transcribe API was the best quality option we could offer:
  • it’s fast
  • recognition is very accurate
  • recognition can mix different languages to some degree
  • the transcribe API allows realtime transcription by default.
  1. To ensure Privacy protection, we have deciced to offer an anonymization proxy to the system that offers two benefits:
  • it makes transcriptions fully anonymous: server-side the API receives various audiostreams from our proxy, that cannot be associated to any spectific user.
  • it allowed us to use the QUIC protocol between the /e/OS smartphone and the proxy, to ensure that whatever network conditions are (which can vary a lot when moving using 3G/4G/LTE), the service can recover easily (which is not possible with TCP).
  1. What’s next? We aknowledge that this service is not totally perfect in term of Privacy protection, although it offers a decent level of protection, as it is anonymizing the streams. In any case, we’re looking at possible better alternative services that we will be able to use in the future to improve the current implementation. One alternative we could chose would be to implement our own instances of Whisper (which is open source software) but this is quite a big project that we cannot prioritize at the moment.

So we think that this /e/OS voice-to-text feature is super useful to send quick messages when walking for instance, or driving a car to avoid typing it, which can be extremely dangerous, and that for this kind of usage has a decent level of Privacy protection.

Also, there is no obligation to use this service at all: people can chose other alternatives if they prefer them, and obviously, users who are dealing with very sensitive information should probably not be using ANY online voice-to-text service (but again, again and again, it’s never been the purpose of /e/OS and will never be to protect users targeted by a gov agency, or by a criminal organization).

Thank you for your positive support and suggestion!

Of course we welcome positive and viable suggestions how we can make this service better (in term of feature and in term of privacy).

25 Likes

Hi Gael,

Can you confirm that no data is ever sent to openAI, either directly or via your proxies, anonymised or otherwise, from our devices if we don’t use the feature? For example, but not limited to, handshaking, initial registration, device registration etc etc.

Thanks,

Dafdaf

4 Likes

Stop bullshiting around - as long you cannot proof this.
Your comment is not helpful for the discussion

6 Likes

Hello Gael, thanks for the clarification and background information regarding the OpenAI API usage and your proxies!

Two aspects:

I would like to read a response regarding this question from you, as well.
That is highly important and could give a lot of users a sort of relief.

Another aspect is:
You could make it - the implementation into /e/ - able to be a choice.

As I mentioned in another thread (there are several regarding this “issue”), either by making it a selection before or while the installation of /e/ occurs - or by having a first-boot assistant (more or less a simple GUI) which allows the removal of specific features by unchecking them (or even better: leaving them on the device by actively checking them, as such opt-in).

If they’re uninstalled on the first boot after the user selected what he/she wants to keep and use, that’s a good way due to less configuration-related hassle by later changes through the user which could be corrupted by a later uninstall (or dependencies to later installed apps not being there due to it).

Next to VTT, my suggestion would be to make “Find my device”, “Murena MGM”, “Health App” a choice of “keep or remove completely” on first boot.

Those would be the ones which cause the most discussions and up-stir in the forums from what I see. And none of those is a “must-have” for > 50% of the users, which should allow them to select if they want to keep/use them or if they want to have them removed, without using ADB and technical ways.

2 Likes

"Comments exist to express one’s opinion!
So why does a company with the slogan ‘your data is your data’ share your data with other companies

1 Like

Nothing against that but your prior comment…

“Apparently, the developers are paid too little, or on the contrary, the developers were paid a lot by those who benefit from this”

…is not appropriate

5 Likes

The voice-to-text feature works very well, no doubt about it. It’s very accurate and fast.

The fact that you choose OpenAI doesn’t really give a good signal to your users. We haven’t chosen /e/OS because it is the most advanced mobile operating system with a lot of cool features. We choose /e/OS because it is supposed to respect our privacy and is de-googled, and we accept that we are missing some features that other operating systems have, such as voice-to-text.

Choosing /e/OS is a political statement : not relying on big tech compagnies (and not only google). Instead of de-google we could say de-big-tech-compagnies. The problem is not only google obviously. Microsoft, Apple, Google, Meta, Open AI, Twitter… are kind of all the same, capitalist companies that think of their profits before thinking of their users or the planet.

What really concerns me is that you don’t seem to understand why that choice and the way it was made are problematics. It really feels like you tried to fool the users by not saying it at the event, not communicating about it at all. You waited for us to start questioning about it to give some explanation.
If you are making such a strong choice that really go against your principles, you should at least inform the users and letting them choose if they wanna use this feature or not.
You are giving our biometric data to Open AI without our consent, without letting us know. At least a popup should appear at the first use, explaining how it works in simple terms. Or a small ‘Powered by Open AI’ on a corner. At least.

This really lower my trust level on Murena. That’s a shame because I’m a very enthusiast user for many years. I have a premium Murena account and I also invest in it.

18 Likes

Would you also discard Startpage simply because it’s a proxy for the Google search engine and it doesn’t have a “Powered by Google” on a corner?

1 Like

Great making us aware but in the end it is ‘optin’, right? If you don’t like it don’t use it. Same goes for /e/OS in the end.

1 Like

Would you say the same if on your trusted de-googled and “Your data is your date” ROM-of-choice a new map app would be featured, full-bodied advertised, coming with a inconspicuous name, and only on closer inspection by users would be discovered that it is actually Google Maps? I doubt it.

That is the most disappointing thing for me for now. Not only have the users to find out themselves what is being foisted on them, we are also expected to get along with such an uncomprehending answer.

In a moment of bewilderment and incomprehension, I may have overshot the mark with my speculations about money in my “Plea”. However, I find this lack of understanding frightening and fear that it means nothing good for the future. My confidence regarding /e/, which I recently expressed elsewhere in this forum, is now close to zero.

Transparent communication looks different and would take place before, not after, such an implementation.

1 Like

I did find it out before, here on this link: Maps application
Here for the other default apps: /e/OS product description - a pro-privacy mobile operating system and cloud services
Because I didn’t trust them I wanted to see for myself what /e/OS is and how they can manage to have everything as stock Android has, as they claimed. And from then I knew, I couldn’t compare a billion dollar company with a small startup like endavour which wasn’t funded by millions from some profit driven investors. I knew they didn’t but they are trying as good as possible as they can. That was years ago, not sure how perception changed but documentation about apps is still accessable.

1 Like