Data leaks in Advanced Privacy and from DNS changes

[nanabanaman]

Continuing the discussion I started here. I think this warrants a separate thread.

In short, I noticed that the list of trackers caught by Advanced Privacy for an app diverges from that from Exodus, as listed in App Lounge. Genereally, the list in Advanced Privacy is quite a bit shorter.

My first test subject was the Drops language learning app, which is infested with quite a few trackers.

This is the list of trackers as listed by Exodus Privacy via App Lounge:

This is the list of trackers as caught by Advanced Privacy:

@tcecyk kindly suggested to use NextDNS as a tool for further investigation, which was a great tip. Here are the results of opening the app:

We can draw the following conclusions:

1. /e/ seems to report changes to the DNS settings to a Google domain that looks a lot like an analytics endpoint to me (the two connections on the bottom of the list). Unless this is somehow NextDNS using Google infrastructure to provide its services, this is bad and should be fixed.

2. The trackers that are caught by Advanced Privacy, such as Lean Plum, are not in the list of DNS queries. It stands to reason that the DNS queries for these get filtered by Advanced Privacy. In other words, what ultimately does end up in the NextDNS log are likely tracking connections that actually leak data.

3. Some trackers from the Exodus list are conspicuously absent from both AP and NextDNS. This means they are either false alarms, contacted directly based on an IP anddress instead of a domain or only get activated under certain conditions (say, Tencent Analytics might only get activated when the user has a Chinese IP to avoid scrutiny). This needs some further investigation.

4. Tracking information is likely being leaked to five different services by four different companies (Google with Firebase Logs/Crashlytics, Facebook, Revenuecat, Sentry.io) despite Advanced Privacy being active. All of these domains can be blocked without adverse effects on the functionality of the app.

I have checked the block list used internally by Advanced Privacy that @tcecyk pointed me to. Despite some of those endpoints being among the most common tracking domains (virtually every Play Store app connects to them), they are somehow omitted from the list of known trackers. facebook.net is another one, though it is not contacted in this example.

Given these results, as great as it is in principle, Advanced Privacy seems to give users a false sense of security at this stage.

For now, can anybody tell me where that e_trackers.json file is located on the device so I can adb push an edited version to my device for further research and testing? Is there maybe an additional JSON file where user entries can be added, so it won’t be overwritten by an update? If that’s not possible, I’ll probably replace AP with a firewall app as a workaround for now as right now, I’m not quite ready to mess with custom builds and flashing them onto a production system.

[Cabnedor]

Dns filter

[smu44]

From app/src/main/java/foundation/e/privacycentralapp/data/repositories/TrackersRepository.kt · main · e / os / Advanced Privacy · GitLab we can see the intent to use an external list for trackers.
And list/e_trackers.json · main · e / os / tracker-list · GitLab is actively maintained.

But I don’t know when/how this list is integrated into one’s device…

[Nicolas_Sas]

Note that Advanced Privacy is not only based on exodus privacy but it use also adaway:

Advanced Privacy

@anon88181694 Thanks for your feedback about “Advanced Privacy”. I found your comment very critical about this feature. I think it’s a first step and indeed, some things can/should be revised. However you say the following: It seems quite normal that the version of ORBOT used is 0.4.6 in the sense that the version 0.4.7 is only stable since April 27, 2022. I think they have been working on the code for a little longer than a month, hence the use of the second last stable version at the…

[jpmeuret]

Hello.
For the moment, I conservatively keep using Tracker Control for that purpose, waiting for AP’s maturity … but I have not compared TC’s filtering / blocking performances with other apps / methods.

[nanabanaman]

Indeed, the node.js script build_list.js · main · e / os / tracker-list · GitLab seems to handle building of the e_trackers.json file based on lists from AdAway and Exodus. The URLs for the sources are in lines 3 and 4 of the script:

const adawayUri = "https://adaway.org/hosts.txt"
const exodusUri = "https://reports.exodus-privacy.eu.org/api/trackers"

So I had a look at the contents of these source files. Both have human-readable syntax.

Shockingly, neither file lists the endpoints in question here:

facebook.net
graph.facebook.com
firebaseremoteconfig.googleapis.com
firebaselogging-pa.googleapis.com
firebase-settings.crashlytics.com
*.crashlytics.com
*.metric.gstatic.com
*.sentry.io
*.revenuecat.com

What’s so baffling is that except maybe the last two, pretty much every commercial Play Store app connects to these, which makes me wonder if it was a conscious decision by the upstream projects to omit them for some reason.

I think either /e/ should add its own list of additional URLs, or find a way to upstream these. All of these are services a user who makes a decision to use /e/ wouldn’t ever want their phone talking to.

[nanabanaman]

I’m not an Android developer and I find it a bit difficult to understand, what that code in TrackersRepository.kt does beyond loading the JSON and writing it to some kind of file.

Maybe the file gets written into an .apk, in which case it probably would be difficult to edit on the device without manually rebuilding all of AP.

[aibd]

@smu44

I don’t know when/how this list is integrated into one’s device…

From its position in the repos the .kt must be built into Advanced Privacy aka privacycenrtralapp

It was interesting to see the README for the tracker.list https://gitlab.e.foundation/e/os/tracker-list/-/blob/main/README.md. Participation seems invited.

[smu44]

I didn’t take enough time to dig into that, but I think that tracker URL are written in some kind of internal database.

Maybe the whole process should be like:

• internal database, delivered as a file with AP APK
• entries written to internal database
  (later)
• new database downloaded from eCloud
• new entries merged into internal database OR internal database overwritten with the downloaded file

[smu44]

@aibd

From its position in the repos the .kt must be built into Advanced Privacy aka privacycenrtralapp

Yes, I agree.
It was more about the scheduling of an update from eCloud (sorry if it wasn’t clear, this was only my first coffee ).

[aibd]

I phrased my observation badly by leading with your rhetorical question, for context Thanks for pointing out the source files.

I should have addressed @nanabanaman and said this is a good catch, and it would be great to see these items added.

I think that adding your additions (as you say @nanabanaman) on the device will be hard if my reading is right that the .json is baked in to the .apk with kotlin.

I think as it stands (and @smu44 suggests a better arrangement) that the suggestion of a commit as per the README is the method invited. (Implementation would at best at the next build, or self build.)

However I feel it is important enough the raise an issue against Advanced Privacy.

[nanabanaman]

From my perspective, it would be useful to have a JSON file on the file system on the phone where Advanced Privacy would read user entries from, in addition to the built-in database.

More frequent updates of the tracker database definitely sounds like a nice feature, whichever way the dev team would choose to implement it, be it eCloud or system updates or updates of Advanced Privacy via App Lounge.

@aibd: If I read that readme and the code correctly, the list is automatically generated from Exodus and AdAway using the build_list.js script. So we can’t just manually add something to the final e_trackers.json, commit it to that repo and be merry, as our changes would be overwritten with the auto-generated file the next time the tracker lists are generated.

Instead, the build_list.js script would need to be adapted to somehow merge custom items into the JSON file, either written directly into the code (which could become unwieldy quickly), or from a separate custom list of trackers in the same directory.

[smu44]

Allowing importing or editing the JSON file will imply embedding the syntax check code (actually in Node.js).
Anyway, it worth opening a Gitlab issue, as @aibd suggested

[Nicolas_Sas]

I’m really curious to see how IodéOS implements this list with an insane amount of different sources without running into problems.
The idea of a blocker only based on exodus (and adaway) seemed to me attractive in the simplicity of implementation but remains perhaps insufficient.
We could also compare AP’s exodus trackers to Blokada’s for example to see if the list is the same.

[petefoth]

@Nicolas_Sas

I’m really curious to see how IodéOS implements this list with an insane amount of different sources without running into problems.

Sadly, until they decide to open their proprietary system, we’ll never find out

[Nicolas_Sas]

Yes, I’m afraid so…
I read here that they were waiting to find the right economic/financial model before putting their source code in open access (which can be understood).

[nanabanaman]

The check script validate_json.js only seems to be used to verify that changes to the build_list.js script don’t break the output and generate valid JSON. Just reading JSON files can easily be done without node.js (thankfully so).

If a user breaks the syntay of their file, it would probably just result in a parser error, which could be displayed in an error message.

Also, thanks for pointing me to the bug tracker, I had been looking for that! I’ll file issues for both the AP list issue and the DNS change analytics leak as soon as I find time.

@Nicolas_Sas Blokada doesn’t seem to list these domains either, which I find quite surprising. I also remember having to add them manually to the Lockdown Apps firewall on various iOS devices, where they also do get hit a lot. So for some reason, they seem to be missing from common lists.

iodé’s sources seem to be mostly focused on ad blocking on the web and not so much in-app analytics – maybe that’s the reason.

I haven’t checked all their sources individually, but the ones I tried didn’t have the firebase URLs and the *.metric.gstatic.com URLs. Even their facbook-specific source from the list doesn’t seem to include graph.facebook.com and facebook.net.

Anyway, I’ll file the issues and we’ll see what the dev team thinks of it.

[Nicolas_Sas]

@nanabanaman

firebaselogging-pa.googleapis.com

This one is catched by DuckDuckGo Tracker Radar:

https://github.com/duckduckgo/tracker-radar/blob/main/domains/US/firebaselogging-pa.googleapis.com.json

Don’t know for the others on the list you gave

[nanabanaman]

Interesting that DuckDuckGo only blocks a certain URL instead of just the entire domain. If Googe bumps the revision of that Firelog API to v2, those contact attempts won’t be filtered unless someone manually adds them to that block list again.

Just a quick heads up to everyone with regards to the ticket: The issue tracker won’t let me sign up for now, I’ll have to go through staff to have one created for me. I’ll look into it as soon as I can find some time.

[nanabanaman]

Unfortunately, I haven’t yet heard back from the helpdesk with regards to the account creation problem on the issue tracker.

In the interest of getting this solved as quickly as possible, is there anyone here with a working bug tracker account who would be willing to file the two tickets? One being the trackers slipping through, and the other the possible connection to Google on DNS change?

Also, a quick addition: We should probably block *.facebook.net as well. The most common host seems to be connect.facebook.net, but there seem to be a myriad of others.