maemo.org - Talk - [SCRIPT] - Ad-Blocking

maemo.org - Talk (https://talk.maemo.org/index.php)

- SailfishOS (https://talk.maemo.org/forumdisplay.php?f=52)

- - [SCRIPT] - Ad-Blocking - Hosts file method (https://talk.maemo.org/showthread.php?t=94627)

[SCRIPT] - Ad-Blocking - Hosts file method

Read this page one the wiki if you don't know what I'm talking about (the page talk about ad-blocking on N900 but the hosts file method works with nearly every OS ever created).

And maybe a look here shuld be useful too.

So first of all you need root acces, so type: devel-su

Backup your current hosts file with:

Code:

cp /etc/hosts /etc/hosts.base

hosts.base will be used by the script to create the new hosts file so do not delete it!

Now create a file (example: vi update-hosts) with these commands:

Code:

#!/bin/bash



URL=http://winhelp2002.mvps.org/hosts.txt

TEMP=/etc/hotst.tmp

BASE=/etc/hosts.base

HOSTS=/etc/hosts



rm -f $TEMP 2> /dev/null

curl $URL | grep ^0.0.0.0 | cut -d "#" -f 1 | grep -v localhosts >> $TEMP && rm -f $HOSTS && cat $BASE $TEMP | sed -e "s/^\s*//g" -e "s/\s*$//g" -e "/^\s*$/d" -e "/^$/d" >> $HOSTS

rm -f $TEMP 2> /dev/null

The third line of commands goes from curl to >> $HOSTS, it's only 1 line with a lot of commands... I know... this looks bad but it's a safe way to menage the hosts file without break anything.

Make it executable, writable and readable only by root with:

Code:

chown root:root update-hosts

chmod 770 update-hosts

Execute it with (or without devel-su if you have are the root)

Code:

devel-su ./update-hosts

Run this file to update the hosts file. "hosts.tmp" will be created to store the file downloaded from http://winhelp2002.mvps.org/hosts.txt. The script automatically remove the temp file and every comment or usless strings and characters in the file to reduce the size (not so much, ~100KB).

To restore the original hosts file type:

Code:

rm -f /etc/hosts

mv /etc/hosts.base /etc/hosts

Btw this ad-blocking method is very simple and works only with the well know ads... but...
http://i1.kym-cdn.com/photos/images/...047-38imqt.jpg
You can test it by yourself on your Jolla with these sites:
1) http://ads-blocker.com/testing (partially works)
2) http://thepcspy.com/blockadblock/ (works!)
3) http://simple-adblock.com/faq/testing-your-adblocker/ (do not work)

Re: [SCRIPT] - Ad-Blocking - Hosts file method

First of all thanks for this.

It might be a good idea to evaluate the contents for anything else than lines with localhost addresses 127.0.0.1 and ::1. Otherwise you are trusting the provider of hosts.txt file that it does not contain any redirects to malicious proxy sites.

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Quote:

Originally Posted by Manatus (Post 1461128)

Every reference to localhosts from that providere is replaced byt the original Jolla hosts settings (that why I use the file hosts.base).

The provider of that hosts file is really famous and everyone download the hosts file from him, but if anyone whuold like to check for malicious redirect fell fre to do it :D and report your progres here pls.

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Quote:

Originally Posted by Vento (Post 1461129)

I apologize if I'm wrong here, as I'm not very proficient with unix tools. I may miss some sed and regex magic...

But what I see is that you grep all the lines that mention localhost. That is fine. But ipv4 record can consist of more than that such as

Code:

112.112.112.112 im.malicious.com localhost.localdomain localhost

And currently you would insert that into the newly created host-file.

I think you need first to drop any lines that contain any other IP than 127.0.0.1 and ::1.

And also 127.0.0.1 must not be in the form #127.0.0.1 in the end of line like this (on one line although visible here on two):

Code:

112.112.112.112 im.malicious.com localhost.localdomain localhost #127.0.0.1

Code:

112.112.112.112 im.malicious.com #localhost.localdomain #localhost #127.0.0.1

EDIT: added second example line. I haven't checked yet what would happen, though.

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Aaaand... I stand corrected. You are right. :)

Lines formed in the way I suggested get cleaned away properly.

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Quote:

Originally Posted by Manatus (Post 1461141)

Aaaand... I stand corrected. You are right. :)

Lines formed in the way I suggested get cleaned away properly.

Actually thanks to your exemples I've done a little fix. Thank you :D

Re: [SCRIPT] - Ad-Blocking - Hosts file method

I'm correcting myself again. Try hosts.txt (all # on the same line as IP):

Code:

127.0.0.1 im.not.malicious

112.112.112.112 im.malicious.com #localhost.localdomain #localhost #127.0.0.1

112.112.112.113 im.malicious.com localhost.localdomain localhost #127.0.0.1

111.111.111.114 colocalhosted #127.0.0.1 #colocalhosted

111.111.111.115 evil.com  #127.0.0.1

This resulted in a host-file:

Code:

127.0.0.1 localhost localhost.localdomain

::1 localhost6 localhost6.localdomain6

127.0.0.1 im.not.malicious

111.111.111.115 evil.com

The last line got through, only #127.0.0.1 was cut away.

The original hosts-file was:

Code:

127.0.0.1 localhost localhost.localdomain

::1 localhost6 localhost6.localdomain6

EDIT: Doing edits at the same time you answered. Previous was tested with original command (curl replaced with cat here):

Code:

cat hosts.txt | grep -v ^# | grep -v localhost | cut -d "#" -f 1 >> /etc/hosts.tmp && cat /etc/hosts.base /etc/hosts.tmp >> /etc/hosts && rm /etc/hosts.tmp && sed 's/^ *//; s/ *$//; /$/d' /etc/hosts

Re: [SCRIPT] - Ad-Blocking - Hosts file method

With:

Code:

cat hosts.txt | grep -v ^# | cut -d "#" -f 1 | grep -v localhost >> /etc/hosts.tmp && cat /etc/hosts.base /etc/hosts.tmp >> /etc/hosts && rm /etc/hosts.tmp && sed 's/^ *//; s/ *$//; /$/d' /etc/hosts

evil.com and now im.malicious.com from previous example still got through.

But with:

Code:

cat hosts.txt | grep -v "#" | grep -v localhost >> /etc/hosts.tmp && cat /etc/hosts.base /etc/hosts.tmp >> /etc/hosts && rm /etc/hosts.tmp && sed 's/^ *//; s/ *$//; /$/d' /etc/hosts

it works. Since #-char has no purpose (except in malicious intent here), you can grep the lines with them all away. :)

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Quote:

Originally Posted by Manatus (Post 1461157)

With:

Code:

cat hosts.txt | grep -v ^# | cut -d "#" -f 1 | grep -v localhost >> /etc/hosts.tmp && cat /etc/hosts.base /etc/hosts.tmp >> /etc/hosts && rm /etc/hosts.tmp && sed 's/^ *//; s/ *$//; /$/d' /etc/hosts

evil.com and now im.malicious.com from previous example still got through.

But with:

Code:

cat hosts.txt | grep -v "#" | grep -v localhost >> /etc/hosts.tmp && cat /etc/hosts.base /etc/hosts.tmp >> /etc/hosts && rm /etc/hosts.tmp && sed 's/^ *//; s/ *$//; /$/d' /etc/hosts

it works. Since #-char has no purpose (except in malicious intent here), you can grep the lines with them all away. :)

But in this way you delete every line that contains a comment or localhost, even if the word localhost is placed in the comment.

Re: [SCRIPT] - Ad-Blocking - Hosts file method

Quote:

Originally Posted by Vento (Post 1461159)

But in this way you delete every line that contains a comment or localhost, even if the word localhost is placed in the comment.

Aww, blast! You are correct; hosts.txt from http://winhelp2002.mvps.org contains comments on the same line. Never paid attention to them before... :(

Example in the end of the file:

Code:

# [ZEDO]

0.0.0.0 zedo.com #[WebBug]

0.0.0.0 ads.zedo.com

0.0.0.0 c1.zedo.com

0.0.0.0 c2.zedo.com #[Tracking.Cookie]

0.0.0.0 c3.zedo.com