Block unwanted servers and websites using hosts files on any system

archje · May 4, 2020, 10:24pm

Yes @reinar, the directory /system/etc/hosts exists - but without files (empty).

Can we please continue corresponding tomorrow. It’s after midnight and I’m already sitting in front of the screen too long again. I should better go to sleep now.

reinar · May 4, 2020, 10:29pm

Of course @archje Good night

archje · May 5, 2020, 7:37am

Thanks @reinar, it’s a a well understood article. Interesting is also:

There are also some predefined block lists available that you may use in your hosts file: https://github.com/StevenBlack/hosts

Now I’m surfing with the pre-installed web browser G°°chrome on websites of the e.foundation, among others > What a Meizu phone is doing.

hosts_reinar_meizu-test_1 twrp_hosts

The file hosts contains only these two entries:

127.0.0.1       localhost
::1             ip6-localhost

DNS name resolution of my router (Cable > WLAN > Samsung Phone)

DNSv4 server 46.182.19.48 | digitalcourage e. V. (DNS-over-TLS)

alternative

DNSv4 server 80.241.218.68 | dismail.de (with DNS-over-TLS as well as advertising, tracker and malware filter)
DNSv6 server 2a02:2970:1002::18 | digitalcourage e. V. (DNS-over-TLS)

alternative

DNSv6 server 2a02:c205:3001:4558::1 | dismail.de (with DNS-over-TLS as well as advertising, tracker and malware filter)

@reinar, how does our experiment continue?

reinar · May 5, 2020, 11:08am

@archje Wow, this is already a very nice setup you got there !
Personnaly I use DNS over TLS on my browsers but not on my router since it does not support this kind of modications.

In your case, you need to create your own hosts file and to fill it with the things you want to block !

Let’s start with this basic instruction :

127.0.0.1 localhost
::1 ip6-localhost

127.0.0.1 google-analytics.com

To add it to your host file :

create a new file inside /sdcard : hosts.new
Fill the file with the instructions and save ! (You can fill the file directly on your phone or copy it from your computer)

Then, launch TWRP, mount /system and do this command :

mv /sdcard/hosts.new /system/etc/hosts

This will replace your hosts file.

Now you can try to go on google-analytics.com on your phone and check if it is effectively blocked !

archje · May 5, 2020, 12:40pm

Hej @reinar, this idea is so simple, so good!

I first tried it with my Wind°ws host machine. Without having changed the empty file hosts, the website google-analytics…com is not found by my system (host.jpg #1)

Page was not found

The connection with the server google-analytics.com failed.
[..]
Try again

Next I’ll try: https://support.google.com/. The web page opens immediately (hosts.jpg #2).

C:\Windows\System32\drivers\etc\hosts.[data file]

Now I enter in the file hosts: 127.0.0.1 support.google.com and save it.

Next, I clear browser cache and call the website again. This time I get an error message:
Error: Connection failed (host.jpg #3)

https://ecloud.global/s/XJCPdPB9GedGq8q

Since my knowledge of web networks is extremely limited, I will ask you once again in simple words: by this forwarding of the original hostnames to my internal IP addresse(s), no external contact to the addresses is established, @reinar?

reinar · May 5, 2020, 1:31pm

Well done @archje

You are absolutely right ! In your case when you are connecting to support.google.com, the computer send the request to itself instead of the internet. (127.0.0.1 is an address used when the computer want to talk to himself in simple words).

I see that your computer language is deutch, so I guess english is not your first language ?

If you want to see what really append on the network and learn about it I strongly suggest you to install a program called Wireshark on your computer

Here is the link : https://www.wireshark.org/

Try it !

archje · May 5, 2020, 1:39pm

Thank you! I’ill try it.

@reinar,
my idea is: Change the title of your thread so that our /e/ user friends don’t only think of Meizu & China and many more users read it: Block a Website simple; e.g. What a Meizu phone is doing

One more question: Which /e/ phone do you use?

reinar · May 5, 2020, 1:52pm

@archje Change the title seems like a good idea !

I suggest : Block unwanted servers and websites using hosts files on any system.

But I do not know how to change the title on discourse…

I am using an old nexus 5 as my main phone, /e/ is working very well on it actually, haven’t seen any bug.
I’m considering buying a galaxy S10 but /e/ does not support it and i am not sure that I want to root the phone or install TWRP to change stuff…

What do you think about that ? You are the Samsung expert here

Manoj · May 5, 2020, 1:54pm

Hi @reinar, changed the title of the post as required.

archje · May 5, 2020, 2:07pm

Block unwanted servers and websites using hosts files on any system

@reinar, my personal favorite of the Samsung’s supported by /e/ OS is the Galaxy S8 SM-G950F “dreamlte” and (S8+ SM-G955F “dream2lte”).

Why?
Like performance of the Galaxy S8 is more than adequate (at least as far as my needs are concerned)

Because an official /e/ OS version 9-pie is available for the S8.

(Because for the S8+ the /e/ OS version 9-pie will be available soon. The tests¹ are already running.)

andrelam · May 5, 2020, 2:09pm

I use this one, i even pay the guy once a year because i love the list, no it’s not me haha . I wrote a little program some time ago to push it to my phone to make my life easy. I would love to have this hosts file embedded in /e/. It works great, even with use of VPN, the hosts file is queried first.

reinar · May 5, 2020, 2:21pm

Hi @Manoj Thank you

@archje : Thank you for the details

Actually, there are multiple factors that can influence my decision but I think I need to wait a little bit more since I buy my last phone four years ago. Five years between each phone seems good for me in an ecological approach.

Manoj · May 5, 2020, 2:24pm

You can post this on Gitlab for the dev team to have a look at it. Once they figure out how to add it in or if they want to add it in this can be a part of the ROM.

archje · May 5, 2020, 2:39pm

Block unwanted servers and websites using hosts files on any system

There are also some predefined block lists available that you may use in your hosts file: https://github.com/StevenBlack/hosts

Another blocklist
https://www.reddit.com/r/oisd_blocklist/comments/dwxgld/dbloisdnl_internets_1_domain_blocklist/

exyna · May 5, 2020, 5:07pm

I strongly discourage you guys from solely using the host file to block websites, especially when you are doing this on your computer.

This has several reasons:

It’s not mandatory for browsers to use the system’s host file. Incase you have a misconfiguration, a file with a false extension or the program simply bypasses the OS resolver, the host file will be ignored. If you do not notice that, you will be fully exposed to tracking.
Following scenario: You visit the site ‘test.com’ by typing the address in your browser’s search bar. The webserver internally redirects you to ‘www.test.com’ as this is the common hostname used by the website. If you now block the site ‘test.com’ by adding an entry to your host file, you can’t be sure that it is actually blocked as your browser eventually has already cached the redirect to ‘www.test.com’ and will now connect you directly. Therefore connections are still possible although you allegedly blocked the hostname (but not the www. one).
Things may change with future updates. Who knows, maybe browser manufacturers will start to ignore the host file by default, especially when DoH is introduced more and more?
The host file has no use if you are using DoT/DoH in your browser as the local OS resolving is bypassed (which is the whole point of DoH). This is especially dangerous for Firefox users as Mozilla started to enabled DoH by default this year (not yet for all users as far as I know). If you are not aware of this, then suddenly also all your ad-blocking will be gone and all counter measures taken (Pi hole, etc.) are worthless as all DNS queries are encrypted (which generally is a good thing for privacy outside your network but not security). Android 9 also introduced DoT, so apps who connect to tracking services over DoT won’t be stopped by your host file.

As you should notice at this point, there are many things that can possibly go wrong and it’s so easy to have your data leaked here. I would always recommend you to block tracking websites as soon as possible which is in the browser directly (for example with uBlock,etc.). This is probably the safest variant with the least potential for data leaks.

A good solution for Android is the app NetGuard. It also relies on the system’s host file but acts like a local VPN. Therefore, all requests made have to be routed through the VPN interface which allows the app to gain full device-wide controll over all connections and decide on its own again wether the request should be allowed or forbidden.
Edit: I’m not sure though how NetGuard handles DoT connections on Android 9 and upwards. I don’t think that it can block those requests. So I would assume that you actually have to locally resolve those queries again. I haven’t made any research here as I’m still on Android 8, would be nice to know if somebody knows

If you are still looking for an OS-wide solution while maintaining privacy through DoH, I’d recommend you to use a local proxy like Stubby (https://github.com/getdnsapi/stubby) or dnscrypt-proxy (https://github.com/DNSCrypt/dnscrypt-proxy). Stubby is simply a local DNS resolver which accepts unencrypted DNS requests, encrypts them and then connects to a DoH server to read the IP address. So you have to do some adblocking before, either with firewall rules or another proxy.
Dnscrypt-proxy already features ad blocking so that’s maybe easier to set up. You could just feed the proxy your hostfile which will then block all malicious/ad-related domains before encrypting to DoH.
So with dnscrypt-proxy your data flow would look something like this:

Another tip for hostfiles: I’d recommend you to create your hostfiles depending on ASNs. If you visit this site here, you will get all IPs for the ASN 15169 which is Google LLC. That’s a good way to quickly block whole providers.

reinar · May 5, 2020, 9:25pm

Hello @exyna !

Yes, I understand your point of view, actually my first idea was to use the hosts file to block native trackers on stock Android roms.
It was more an example of the usage of hosts file.
The best idea is of course to rely on an external router or a proxy with DOH/DOT.
I have never heard about stubby, it looks interesting for a linux computer.

Can you tell more about this idea of ASN ?
How can we implement IP with netmasks in host files ?
I checked but I was thinking that it was only for DNS requests…
Are you using a firewall instead to block lists of IPs ?

exyna · May 7, 2020, 3:21pm

Check this wikipedia article for an in-depth explanation about autonomous systems: https://en.wikipedia.org/wiki/Autonomous_system_(Internet)
To summarize this a bit, an AS basically includes a range of IP nets that can be tied to the network of a specific organization (like Google, Facebook, etc.).
Each of these autonomous systems has a unique number which clearly identifies each AS. Therefore we can use the ASN to fetch all registered IP addresses allocated to an organization to block specific companies at once. A company can have multiple ASNs, have a look at https://www.ultratools.com/tools/asnInfo

You should keep in mind that AS blocking is a very harsh way to protect against tracking and it may lead to a lot of websites not working anymore. That’s why you should carefully decide which companies you block and which not.

Blocking IP addresses in the host file will not work as the hostfile is only there for associating IP addresses to hostnames. As we already have the IP addresses, you need to block them directly in your firewall. If you are running Linux, then you can do that by adding specific iptables rules:
iptables -A INPUT -s XX.XX.XX.XX/XX -j DROP
It’s probably a good idea to also look into ipset when you are blocking large amounts of IPs to prevent performance issues.

Of course it doesn’t make any sense to add these rules by hand as you then would be busy over the next years. You definitely have to automate that, also to keep your blocking lists updated as IP addresses may be added/removed from an AS. A quick search showed up the following script: https://github.com/CHEF-KOCH/ASN-blocking
I haven’t tried this script yet, so I can’t assure you that it will work. But you should get the idea with that - you can build iptable rules for specific ASNs. If you further automate that with a cron job, then you will always have up-to-date lists whilst having to spend no time on building lists.

reinar · May 9, 2020, 3:14pm

Thank you @exyna for your explanation !

Maybe that will make more sense in a corporation router, I will not block any ASN, but if I am under DDOS attack or something else

Also, the best way is still to choose our software in a smart way, so we do not have to block everything

I will definitely set up a dnscrypt server at home, this looks mandatory anyway.

The annoying part is that we have to set up a server running constantly at home.

smu44 · May 9, 2020, 3:27pm

Could the /e/ Foundation help, providing a DNSCrypt server ?
There is a Docker image available : https://www.dnscrypt.org/#dnscrypt-server

system · June 3, 2020, 9:29am

This topic was automatically closed after 30 days. New replies are no longer allowed.