javanxd / ya-pihole-list

A bunch of useful pi-hole adlists and a auto updater.

Yet another Pi-hole list

Actually this project is not 'yet another'. I tried to improve what other projects missed. Also added an auto updater.

Supports the new Pi-hole 5 and above.

Quickstart

cd ~
git clone --depth=1 https://github.com/JavanXD/ya-pihole-list.git ya-pihole-list
cd ya-pihole-list
sudo chmod a+x adlists-updater.sh
sudo sh adlists-updater.sh

How to append your own adlists

1. Fork this project.
2. Add your own blocklist to the adlists.list.updater file.
3. Change the adListSource variable in the adlists-updater.sh file to your custom blocklist collection.

Enabling Raspbian and Pi-hole Auto Updater

Add the argument 1 to your sudo sh adlists-updater.sh 1 call.

Updating your adlists automatically

Unfortunately the Pi-hole Gravity script does not automatically update the adlists. Therefore this Updater was developed to do this job. However, it is very important to always use up-to-date blocklists to block the latest phishing sites. In addition, this project takes care to use only blocklists that are updated regularly.

1. Create a scheduled task to run the script:

   sudo crontab -e
   

2. Add this line to make it runs every 12 hour, but you can change it to whatever you like:

   0 */12 * * * sudo /home/pi/ya-pihole-list/adlists-updater.sh 1 >/dev/null
   

Screenshots

Other

Helpful links

• https://filterlists.com/ - A great search engine to find suitable blocklists.
• https://firebog.net/ - A collection of good blocklists.

License

Each converted / modified list file is licensed under the same license as the original list.

Disclaimer

This project is in no way affiliated with the core Pi-Hole project or organization. This project was created as a contribution to the community. Use at your own risk.

ya-pihole-list's Issues

Add YouTube ad-blocking

Hi,

Thanks for your addlist + updater.
Fantastic job.
Now my question, is it possible to add a YouTube ad-blocking list?

Thanks in advanced.
Jan.

Insert Query Error

There appears to be an issue with the insert query:

Info: Download latest adlists list to /home/pi/adlists.list.updater
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1620 100 1620 0 0 189 0 0:00:08 0:00:08 --:--:-- 363
Info: Adding content of /home/pi/adlists.list.updater into gravity database
Warning: Some warnings in table adlist in database /etc/pihole/gravity.db\n /ho me/pi/adlists.list.updater.tmp:1: expected 6 columns but found 5 - filling the r est with NULL
/home/pi/adlists.list.updater.tmp:1: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:2: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:2: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:3: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:3: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:4: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:4: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:5: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:5: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:6: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:6: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:7: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:7: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:8: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:8: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:9: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:9: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:10: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:10: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:11: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:11: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:12: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:12: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:13: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:13: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:14: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:14: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:15: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:15: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:16: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:16: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:17: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:17: INSERT failed: datatype mismatch
Info: Update Gravity because of the changes to the gravity.db
[i] Neutrino emissions detected...

Malware Domains no longer in operation

https://mirror1.malwaredomains.com/files/justdomains 404s. The project is no longer in operation.

According to https://riskanalytics.com/community/:

We have merged AutoShun and Malware Domains, and have integrated those technologies into ShadowNet. Through ShadowNet we are able to identify the latest threat intel gathered from around the globe, share learnings and best practices from our network of clients and partners and continually build those learnings into our products, resulting in the best, most powerful threat intel for your organization.

As of November 30th, 2020, both AutoShun and Malware Domains feeds will be shut off, and subscribers will no longer receive intel from those sources.
We will continue to make a free, non-commercial version of our threat intel, ShadowNet, while the enhanced full-threat intel feed will be licensed to our commercial partners.

It should probably be removed from the list.

updatescript updates whole system

Hi,

While having an up to date system is important, I really don't think it should be this scripts responsibility to perform the updates. Could you remove those lines, or at least make it overridable by setting some environment variable?

Changes to adlists.list.updater are ignored

I added a new blocklist to the file adlists.list.updater:

#DoH
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall.txt

But when I run adlists-updater.sh I do not see that list appear as a target in the script output. The last target that appears is the last line in your adlists.list.updater file:
Target: https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/dnscrypt/spy.txt

What do I need to do to make the script recognize my edits to the adlists.list.updater file?

Updated Pihole adlist format

The pihole adlist format now expects 10 columns and the INSERT breaks on the NULL values that get autofilled.
Line 73 in adlists-updater.sh needs to be updated to read:
echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Added by Updater\",,0,0,0" >> "${tmpFile}"

Add check for sqlite3 program [Feature Request]

Was running the script, but it wasn't updating the adlist. Starting debugging the logs/script and realized it was calling sqlite3 program, which is not installed on the system. Maybe add a check near the beginning for sqlite3 and error out if it is missing? It's not required to run Pi-hole.

Remove dbl.oisd.nl from your master list

I don't know if this is a temporary issue, or permanent, but there appears to be an issue with dbl.oisd.nl. May want to remove that from your master list.

Adlist:

# Ads, Mobile Ads, Phishing, Malvertising, Malware, Spyware, Ransomware, CryptoJacking and some Telemetry, Analytics, Tracking
https://dbl.oisd.nl/

Log:

  [i] Target: https://dbl.oisd.nl/
  [✗] Status: https://dbl.oisd.nl/ (410)
  [✗] List download failed: no cached list available