britannic / blacklist Goto Github PK
View Code? Open in Web Editor NEWBlacklist and Adware Blocking for the Ubiquiti EdgeMax Router
License: Other
Blacklist and Adware Blocking for the Ubiquiti EdgeMax Router
License: Other
http://www.hostsfile.org/Downloads/hosts.txt
contains
127.0.0.1 www.crunchbase.com
which is an obvious false-positive.
I've added it to my local whitelist using:
configure
set service dns forwarding blacklist exclude www.crunchbase.com
commit;save;exit
It's possible I am misunderstanding the instructions, and I'm hoping to get some clarification.
When I globally include a host/domain or add a new source, is it supposed to appear in the /config/user-data/blacklist.cfg file? It never does for me although if I try and add it again, it shows that it already exists. If I were to reboot the router without it showing in the file, will it remain?
Also, at what point does the blacklist.cfg file get backed up? Is it only when the package is removed? I ask since the instructions mention deleting the file, and I couldn't tell if it was because the blacklisting was disabled or the entire package deleted, or if it backed it up at some other time. After I've added a new host/domain or a new source, the blacklist.cfg file hasn't been backed up.
I appreciate the efforts you've made with this program! Thanks.
Would it be possible to have blocked requests redirected to a URL or DNS entry, such as blocked.mysite.com? This way should a user click a google sponsored link (for example) instead of the page turning white because the traffic was sinkholed, an internal web server could display a page letting them know what happened.
Currently, there is no way (that I can tell) to add a source file as a whitelist. This would save time manually entering commonly whitelisted domains, such as annudeepND's whitelist.
Add obsolete source OSID Mobile Blocklist
It would be nice to have support for ERPro-8 / ER-8
Every time I try to use this command "set service dns forwarding blacklist include ****" or " set service dns forwarding blacklist domains include *****" to blacklist a website that have p0rn ads. The adblocking stop working when I reset the device, not sure what is wrong so I had to reinstall the .deb package to make it working again. I tried even hard reset the device and configure it from scratch same problem
My device is a ER-X with 1.10.8
First I tried the cli commands to disable and then a complete uninstall.
Cleared browser cache, tried different browsers on different computers.
No Ad block extensions are installed.
But, Ads are still being blocked. I even restarted my router.
I needed to do some testing but I can't do that if I'm unable to disable it.
ERL-3
Linux ubnt 3.10.107-UBNT #1 SMP Mon Oct 1 11:07:12 UTC 2018 mips64 GNU/Linux
Thanks !
sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key | sudo apt-key add -
ubnt@EdgeRouter-4:~$ sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
I'm not sure is it add successfully, so that i tried to download it manually but show below error information.
ubnt@EdgeRouter-4:~$ sudo curl -L -k https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head>
<title>Error: 404 Not Found</title>
<style type="text/css">
html {background-color: #eee; font-family: sans;}
body {background-color: #fff; border: 1px solid #ddd;
padding: 15px; margin: 15px;}
pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
</style>
</head>
<body>
<h1>Error: 404 Not Found</h1>
<p>Sorry, the requested URL <tt>'https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key'</tt>
caused an error:</p>
<pre>Not found: '/britannic/debian-repo/master/blacklist/public.key'</pre>
</body>
</html>
So how can I ensure "public.key" had been add successfully by the CLI you provide?
My approach is use web browser to download keyfile, so that upload by SCP then use "sudo apt-key add public.key"
ubnt@EdgeRouter-4:~$ sudo apt-key add public.key
OK
But it looks has some error msg:
ubnt@EdgeRouter-4:~$ sudo apt-get update
Ign:1 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch InRelease
Ign:2 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch Release
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Ign:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Ign:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Ign:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Ign:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Ign:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Ign:3 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main all Packages
Err:4 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main mips Packages
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Ign:5 https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch/main Translation-en
Reading package lists... Done
W: The repository 'https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public stretch Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public/dists/stretch/main/binary-mips/Packages server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
E: Some index files failed to download. They have been ignored, or old ones used instead.
hello!
first, thank you so much for this work. it really rocks! I've sent to your paypal a 🍺 for that good work 😉
i wanted to give you a couple of heads up in care you are not aware.
thanks!
Hi..
First at all thanks; works perfect!!!
But I have a little problem; everytime the router reboots the script does not start and I have to uninstall and then reinstall....
¿Any sugestion?
Hi,
I have installed blacklists on my router and I am not clear about two aspects:
how can I check if the service is active and working?
I saw the advertising on YouTube continues to be there (less frequent than before) how can I check?
Many Thanks
Hi ,
Thank you for your wonderful tool , It has really helped me to get rid of annoying ads in my PCs and other devices.
I do wonder if there is a way to get rid of ADs from youtube TV ap .
Regards
When fetching the debian repo, I get a 400 error.
CLI commands:
configure
set service dns forwarding blacklist hosts source myhosts file /config/user-data/antiad.txt
commit
Logs:
NOTI[001]21:51:22.197: Starting blacklist update...
CRIT[002]21:51:25.199: No internet access, aborting blacklist update!
Is it possible to include TLDs as well? I've tried creating a new domain file with some problematic TLDs and it seems that they get dropped. (They are never downloaded/created to the /etc/dnsmasq.d/ folder.) The hosts file I created and played with works without any problems.
I went to globally include a domain and got an error.
# set service dns forwarding blacklist include zefirka.net
The specified configuration node is not valid
Set failed
If I changed the command to # set service dns forwarding blacklist include domains zefirka.net
then it worked. Any ideas why it wouldn't work globally?
I recently updated my ERPoe-5 to EdgeOS v1.10.8 and found that blacklist was no longer doing its magic.
First, I tried configuring of a new domain:
$ configure
# set service dns forwarding blacklist domains exclude bing.com
The specified configuration node is not valid
Set failed
Second, I tried updating but was already on the current version.
$ sudo apt-get update && sudo apt-get upgrade edgeos-dnsmasq-blacklist
…
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Finally, tried reinstalling, which did the trick and also included my previously included domains.
Question: Should updating EdgeOS require reinstall of blacklist?
Hey there - Two items which require your expertise....
Looks like OISD changed their url again and as a result the base set of adblocks is significantly reduced.
Also - looks like apt-get is only pulling down 1.2.4.8 and manual intervention is required to get 1.2.4.9 installed. My guess is that it needs to be updated here?
Thread with details at the UI forums....
Cheers!
I installed to an EdgeRouter X using the instructions here.
me@ubnt:~$ show configuration commands | grep blacklist
set service dns forwarding blacklist disabled false
set service dns forwarding blacklist dns-redirect-ip 0.0.0.0
set system package repository blacklist components main
set system package repository blacklist description 'Britannic blacklist debian stretch repository'
set system package repository blacklist distribution stretch
set system package repository blacklist password ''
set system package repository blacklist url 'https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public/'
set system package repository blacklist username ''
set system task-scheduler task update_blacklists executable arguments 10800
set system task-scheduler task update_blacklists executable path /config/scripts/update-dnsmasq-cronjob.sh
set system task-scheduler task update_blacklists interval 1d
~$ tail /var/log/update-dnsmasq.log
NOTI[001]2023-01-26 05:31:13.591: Starting blacklist update...
INFO[002]2023-01-26 05:31:13.624: Checking for stale blacklists...
INFO[003]2023-01-26 05:31:15.849: Successfully restarted dnsmasq
NOTI[004]2023-01-26 05:31:15.850: Blacklist update completed......
~$ ls /etc/dnsmasq.d
README dnsmasq-dhcp-config.conf
This seems to indicate that the package in installed and blacklist update is running but it's not actually writing any files for dnsmasq
to process that blackhole DNS requests to bad hosts. Is there a way I can confirm that is the case? Are there other troubleshooting steps I should take?
[ service dns forwarding blacklist ]
NOTI[001]17:59:48.086: Starting blacklist update...
INFO[002]17:59:48.090: Removing stale blacklists...
INFO[003]17:59:48.162: blacklisted-subdomains: downloaded: 15
INFO[004]17:59:48.165: blacklisted-subdomains: extracted: 15
INFO[005]17:59:48.170: blacklisted-subdomains: dropped: 0
INFO[006]17:59:48.199: blacklisted-servers: downloaded: 2
INFO[007]17:59:48.203: blacklisted-servers: extracted: 2
INFO[008]17:59:48.206: blacklisted-servers: dropped: 0
INFO[009]17:59:48.302: global-whitelisted-domains: downloaded: 99
INFO[00a]17:59:48.304: global-whitelisted-domains: extracted: 99
INFO[00b]17:59:48.306: global-whitelisted-domains: dropped: 0
INFO[00c]17:59:48.355: whitelisted-servers: downloaded: 1
INFO[00d]17:59:48.358: whitelisted-servers: extracted: 1
INFO[00e]17:59:48.361: whitelisted-servers: dropped: 0
INFO[00f]17:59:48.365: Downloading domains source zeus
INFO[010]17:59:48.375: Downloading domains source NoBitCoin
INFO[011]17:59:48.378: Downloading domains source SmartTVblacklist
INFO[012]17:59:48.380: Downloading domains source malwaredomains.com
INFO[013]17:59:48.382: Downloading domains source notrackingDomains
INFO[014]17:59:48.385: Downloading domains source simple_tracking
INFO[015]17:59:56.374: simple_tracking: downloaded: 34
INFO[016]17:59:56.405: simple_tracking: extracted: 34
INFO[017]17:59:56.427: simple_tracking: dropped: 0
INFO[018]17:59:56.652: SmartTVblacklist: downloaded: 198
INFO[019]17:59:56.688: SmartTVblacklist: extracted: 175
INFO[01a]17:59:56.740: SmartTVblacklist: dropped: 23
INFO[01b]17:59:57.454: NoBitCoin: downloaded: 689
INFO[01c]17:59:57.473: NoBitCoin: extracted: 677
INFO[01d]17:59:57.496: NoBitCoin: dropped: 12
INFO[01e]18:00:14.422: malwaredomains.com: downloaded: 26885
INFO[01f]18:00:14.441: malwaredomains.com: extracted: 26881
INFO[020]18:00:14.448: malwaredomains.com: dropped: 4
INFO[021]18:00:42.422: Downloading hosts source githubSteveBlack
INFO[022]18:00:42.424: Downloading hosts source YoutubeAdBlockList
INFO[023]18:00:42.425: Downloading hosts source notracking
INFO[024]18:00:42.426: Downloading hosts source YoutubeBlockList
INFO[025]18:00:42.429: Downloading hosts source sysctl.org
INFO[026]18:00:42.434: Downloading hosts source openphish
INFO[027]18:00:44.465: YoutubeBlockList: downloaded: 69
INFO[028]18:00:44.488: YoutubeBlockList: extracted: 69
INFO[029]18:00:44.523: YoutubeBlockList: dropped: 0
INFO[02a]18:00:49.713: openphish: downloaded: 2558
INFO[02b]18:00:49.716: openphish: extracted: 1136
INFO[02c]18:00:49.744: openphish: dropped: 1422
INFO[02d]18:01:03.362: YoutubeAdBlockList: downloaded: 8882
INFO[02e]18:01:03.376: YoutubeAdBlockList: extracted: 8848
INFO[02f]18:01:03.378: YoutubeAdBlockList: dropped: 34
INFO[030]18:01:13.825: sysctl.org: downloaded: 20567
INFO[031]18:01:13.846: sysctl.org: extracted: 17576
INFO[032]18:01:13.847: sysctl.org: dropped: 2991
INFO[033]18:01:28.178: notracking: downloaded: 39080
INFO[034]18:01:28.180: notracking: extracted: 22628
INFO[035]18:01:28.183: notracking: dropped: 16452
INFO[036]18:01:29.625: githubSteveBlack: downloaded: 42353
INFO[037]18:01:29.689: githubSteveBlack: extracted: 27137
INFO[038]18:01:29.694: githubSteveBlack: dropped: 15216
NOTI[039]18:01:32.742: Total entries found: 141432
NOTI[03a]18:01:32.745: Total entries extracted 105278
NOTI[03b]18:01:32.748: Total entries dropped 36154
ERRO[03c]18:01:32.753: ReloadDNS():
error: fork/exec /bin/bash: cannot allocate memory
Everything worked well out of the box.
I've only added a couple of extra sources.
After adding https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt i started getting an error while trying to commit
In the instructions to add back the githubSteveBlack host source, I get an error on an ER4 v2.0.9-hotfix.6.
Found in the README.md.
https://github.com/britannic/blacklist#release-1248-april-9-2023
In the configuration menu, one command fails.
# set service dns forwarding blacklist hosts source githubSteveBlack description '"Blacklists adware and malware websites"'
Cannot use the double quote (") character in a value string
Value validation failed
Set failed
Just omitting the double quote characters, the command is successful.
# set service dns forwarding blacklist hosts source githubSteveBlack description 'Blacklists adware and malware websites'
[edit]
Thank you for the software, slightly smoother life.(results may vary)
hello there!
sometimes i get some legit content blocked when im navigating from my mobile phone while connected to the wireless.
im wondering if I could output some logs in order to troubleshoot blacklisting to be able to make exceptions. because in the computer browser certainly i can enable developer tools and see the domains being blocked but in the mobile phone im not sure how to proceed since some content might be inside an app. any ideas?
thanks!
Is there a way to set responses for blacklisted domains to be an NXDOMAIN
(Domain does not exist) response instead of a NOERROR
response with an A
record pointing to dns-redirect-ip
?
If not, can a new feature be added to switch to this behavior instead?
Lots of folks are posting problems with the ERX units and it's a definite result of those two primary lists, OISD and Steve Black, growing so large.
Here are the default results from package installs:
July 2022: Total entries extracted 136104
Today: Total entries extracted 234118
I think it makes sense to remove one of them and then update the FAQ with a special ERX/ERX-SFP section with the commands that users can copy/paste if they want to remove/add the other.
Removing Steve Black / Keeping OISD
Total entries extracted 59691
Removing OISD / Keeping Steve Black
Total entries extracted 194322
Also, as an aside, I was curious about the simple_tracking list with the 34 domains. It appears that while the disconnect.me site and browser extension hasn't been updated in quite a while, they do have a json file that gets updated more frequently.
What's even stranger is that when I remove it from the default package, I get a "Total entries extracted 234360," and putting it back the number goes back down to 234118. Just thought, I pass that one.
Thanks again for all your work on this! Cheers!
https://github.com/britannic/blacklist/releases
Maybe delete v1.2.5 or manually change it to v1.2.4.65 so it flows better or at the very least move 1.2.4.8 so it appears at the top?
I can't seem to get this working on Chromecast. I have created a custom list. It works on desktop but doesn't seem to be reflected on Chromecast.
Following the docs to exclude a domain.
configure set service dns forwarding blacklist domains exclude list-manage.com commit;save;exit
Domain still resolves to 127.0.0.1
Are redirects logged anywhere?
Where are excludes written to?
I installed this thing today and it doesn't want to run.
It says:
NOTI[001]17:35:01.819: Starting blacklist update...
CRIT[002]17:35:04.821: No internet access, aborting blacklist update!
Commit failed
I checked source code and seems like it checks wherever https://www.google.com is available. And it does indeed available from my router. Here's curl output:
$ curl -vvv https://www.google.com
* Rebuilt URL to: https://www.google.com/
* Trying 172.217.168.228...
* TCP_NODELAY set
* Connected to www.google.com (172.217.168.228) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
* start date: Jul 7 08:10:21 2020 GMT
* expire date: Sep 29 08:10:21 2020 GMT
* subjectAltName: host "www.google.com" matched cert's "www.google.com"
* issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56802eb8)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/7.52.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Sun, 26 Jul 2020 14:38:48 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: 1P_JAR=2020-07-26-14; expires=Tue, 25-Aug-2020 14:38:48 GMT; path=/; domain=.google.com; Secure
< set-cookie: NID=204=JMPs6P7nmTwzfIVDxOsEAPYoaoUjZTS_AdTtUrpogsAv124z-vxOF3xYn-eQzN-oGgiNxuywR67iYoxubXSHqYmvVBuueNjsX07ZG5qSR4cqoUvGrap8B4-VMxPrfmbcX29U7IvQfvlZX6yNW4B2DkFHwTk5KncpXn7ObOitYb0; expires=Mon, 25-Jan-2021 14:38:48 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding
<
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="uk"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){window.google={kEI:'eJUdX6vZEsuBkwW77674DA',kEXPI:'0,18167,183956,3,4,32,1151585,5662,731,223,5104,207,3204,10,1226,364,1499,611,206,383,246,5,304,1050,351,297,1046,1936,470,294,20,3,1051,90,193,117,72,84,49,412,112,135,174,272,3,1120585,1197717,303245,26305,13677,4855,32692,15247,867,19397,9287,9188,8384,4858,1362,284,9006,3023,4740,5,11028,2,1811,4998,7931,5297,2054,920,873,1217,9405,14527,4518,2777,919,2277,8,2796,1593,1279,2212,530,149,1103,840,517,1522,4258,312,1137,2,2063,606,2025,544,1231,520,1947,2229,93,328,1284,16,2927,2246,1820,1780,3227,1990,855,9,6066,6286,4455,641,2450,2458,1226,1742,4929,108,3407,908,2,941,2614,2397,1027,6443,218,10,612,1337,1098,3,346,230,970,865,373,3546,705,149,189,3313,2488,2252,1992,1997,83,1010,651,4,1528,17,1277,1010,1242,1,264,874,405,42,763,4,650,381,20,177,2310,1697,9,43,975,441,1425,459,116,759,468,212,3,599,3189,568,1973,86,3,1493,1787,1418,3,5,69,2615,2811,939,2,812,690,1968,2,243,627,509,2525,148,291,218,194,4,24,888,277,212,4,71,299,390,431,30,463,35,410,134,589,123,34,275,2,334,1479,113,2,151,504,205,2,215,63,174,44,7,318,566,845,650,221,29,142,308,219,589,8,650,997,763,1258,665,4,2,237,125,5784360,8801916,549,333,444,1,2,80,1,900,896,1,9,2,2551,1,748,141,59,736,563,1,4265,1,1,2,1017,9,305,3299,129,2,20,65,7,25,64,33,3502099,20458661',kBL:'aaAI'};google.sn='webhp';google.kHL='uk';})();(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var c;a&&(!a.getAttribute||!(c=a.getAttribute("eid")));)a=a.parentNode;return c||google.kEI};google.getLEI=function(a){for(var c=null;a&&(!a.getAttribute||!(c=a.getAttribute("leid")));)a=a.parentNode;return c};google.ml=function(){return null};google.time=function(){return Date.now()};google.log=function(a,c,b,d,g){if(b=google.logUrl(a,c,b,d,g)){a=new Image;var e=google.lc,f=google.li;e[f]=a;a.onerror=a.onload=a.onabort=function(){delete e[f]};google.vel&&google.vel.lu&&google.vel.lu(b);a.src=b;google.li=f+1}};google.logUrl=function(a,c,b,d,g){var e="",f=google.ls||"";b||-1!=c.search("&ei=")||(e="&ei="+google.getEI(d),-1==c.search("&lei=")&&(d=google.getLEI(d))&&(e+="&lei="+d));d="";!b&&google.cshid&&-1==c.search("&cshid=")&&"slh"!=a&&(d="&cshid="+google.cshid);b=b||"/"+(g||"gen_204")+"?atyp=i&ct="+a+"&cad="+c+e+f+"&zx="+google.time()+d;/^http:/i.test(b)&&"https:"==window.location.protocol&&(google.ml(Error("a"),!1,{src:b,glmm:1}),b="");return b};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};(function(){
document.documentElement.addEventListener("submit",function(b){var a;if(a=b.target){var c=a.getAttribute("data-submitfalse");a="1"==c||"q"==c&&!a.elements.q.value?!0:!1}else a=!1;a&&(b.preventDefault(),b.stopPropagation())},!0);document.documentElement.addEventListener("click",function(b){var a;a:{for(a=b.target;a&&a!=document.documentElement;a=a.parentElement)if("A"==a.tagName){a="1"==a.getAttribute("data-nohref");break a}a=!1}a&&b.preventDefault()},!0);}).call(this);
var a=window.location,b=a.href.indexOf("#");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf("#")&&a.replace("/search?"+c.replace(/(^|&)fp=[^&]*/g,"")+"&cad=h")};</script><style>#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}
</style><style>body,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}body{background:#fff;color:#000}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}.tiah{width:458px}</style><script nonce="/lWi4QVFfvu11uHY7tp53Q=="></script></head><body bgcolor="#fff"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}
if (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}
}
})();</script><div id="mngb"><div id=gbar><nobr><b class=gb1>Пошук</b> <a class=gb1 href="https://www.google.com.ua/imghp?hl=uk&tab=wi">Зображення</a> <a class=gb1 href="https://maps.google.com.ua/maps?hl=uk&tab=wl">Карти</a> <a class=gb1 href="https://www.youtube.com/?gl=UA&tab=w1">YouTube</a> <a class=gb1 href="https://news.google.com.ua/nwshp?hl=uk&tab=wn">Новини</a> <a class=gb1 href="https://mail.google.com/mail/?tab=wm">Gmail</a> <a class=gb1 href="https://drive.google.com/?tab=wo">Диск</a> <a class=gb1 href="https://www.google.com/calendar?tab=wc">Календар</a> <a class=gb1 style="text-decoration:none" href="https://www.google.com.ua/intl/uk/about/products?tab=wh"><u>Більше</u> »</a></nobr></div><div id=guser width=100%><nobr><span id=gbn class=gbi></span><span id=gbf class=gbf></span><span id=gbe></span><a href="http://www.google.com.ua/history/optout?hl=uk" class=gb4>Історія веб-пошуку</a> | <a href="/preferences?hl=uk" class=gb4>Налаштування</a> | <a target=_top id=gb_70 href="https://accounts.google.com/ServiceLogin?hl=uk&passive=true&continue=https://www.google.com/" class=gb4>Увійти</a></nobr></div><div class=gbh style=left:0></div><div class=gbh style=right:0></div></div><center><br clear="all" id="lgpd"><div id="lga"><img alt="Google" height="92" src="/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png" style="padding:28px 0 14px" width="272" id="hplogo"><br><br></div><form action="/search" name="f"><table cellpadding="0" cellspacing="0"><tr valign="top"><td width="25%"> </td><td align="center" nowrap=""><input name="ie" value="ISO-8859-1" type="hidden"><input value="uk" name="hl" type="hidden"><input name="source" type="hidden" value="hp"><input name="biw" type="hidden"><input name="bih" type="hidden"><div class="ds" style="height:32px;margin:4px 0"><div style="position:relative;zoom:1"><input class="lst tiah" style="margin:0;padding:5px 8px 0 6px;vertical-align:top;color:#000;padding-right:38px" autocomplete="off" value="" title="Пошук Google" maxlength="2048" name="q" size="57"><img src="/textinputassistant/tia.png" style="position:absolute;cursor:pointer;right:5px;top:4px;z-index:300" data-script-url="/textinputassistant/11/uk_tia.js" id="tsuid1" alt="" height="23" width="27"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var id='tsuid1';document.getElementById(id).onclick = function(){var s = document.createElement('script');s.src = this.getAttribute('data-script-url');(document.getElementById('xjsc')||document.body).appendChild(s);};})();</script></div></div><br style="line-height:0"><span class="ds"><span class="lsbb"><input class="lsb" value="Пошук Google" name="btnG" type="submit"></span></span><span class="ds"><span class="lsbb"><input class="lsb" id="tsuid2" value="Мені пощастить" name="btnI" type="submit"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var id='tsuid2';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}
else top.location='/doodles/';};})();</script><input value="AINFCbYAAAAAXx2jiCeJzAMo6jeBSHs8LueZBBy3otrf" name="iflsig" type="hidden"></span></span></td><td class="fl sblc" align="left" nowrap="" width="25%"><a href="/advanced_search?hl=uk&authuser=0">Розширений пошук</a></td></tr></table><input id="gbv" name="gbv" type="hidden" value="1"><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){var a,b="1";if(document&&document.getElementById)if("undefined"!=typeof XMLHttpRequest)b="2";else if("undefined"!=typeof ActiveXObject){var c,d,e=["MSXML2.XMLHTTP.6.0","MSXML2.XMLHTTP.3.0","MSXML2.XMLHTTP","Microsoft.XMLHTTP"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b="2"}catch(h){}}a=b;if("2"==a&&-1==location.search.indexOf("&gbv=2")){var f=google.gbvu,g=document.getElementById("gbv");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);</script></form><div id="gac_scont"></div><div style="font-size:83%;min-height:3.5em"><br><div id="gws-output-pages-elements-homepage_additional_languages__als"><style>#gws-output-pages-elements-homepage_additional_languages__als{font-size:small;margin-bottom:24px}#SIvCob{display:inline-block;line-height:28px;}#SIvCob a{padding:0 3px;}.H6sW5{display:inline-block;margin:0 2px;white-space:nowrap}.z4hgWe{display:inline-block;margin:0 2px}</style><div id="SIvCob">Мова Google: <a href="https://www.google.com/setprefs?sig=0_IBsa2kBLszcilzfqVvFCIA9gOaQ%3D&hl=ru&source=homepage&sa=X&ved=0ahUKEwir9evPkuvqAhXLwKQKHbu3C88Q2ZgBCAU">русский</a> </div></div></div><span id="footer"><div style="font-size:10pt"><div style="margin:19px auto;text-align:center" id="fll"><a href="http://www.google.com.ua/intl/uk/ads/">Реклама</a><a href="http://www.google.com.ua/intl/uk/services/">Рішення для бізнесу</a><a href="/intl/uk/about.html">Усе про Google</a><a href="https://www.google.com/setprefdomain?prefdom=UA&prev=https://www.google.com.ua/&sig=K_eoXbg-InyBfVaGDrwwyzOxBg4XM%3D">Google.com.ua</a></div></div><p style="font-size:8pt;color:#767676">© 2020</p></span></center><script nonce="/lWi4QVFfvu11uHY7tp53Q==">(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d="CSS1Compat"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log("","","/client_204?&atyp=i&biw="+a+"&bih="+b+"&ei="+google.kEI);}).call(this);})();(function(){var u='/xjs/_/js/k\x3dxjs.hp.en.XsBchCh7bLs.O/m\x3dsb_he,d/am\x3dAE-wOQ/d\x3d1/rs\x3dACT90oGpUq2u9G5OXekOjNEpL2rNntM8Ng';
setTimeout(function(){var b=document;var a="SCRIPT";"application/xhtml+xml"===b.contentType&&(a=a.toLowerCase());a=b.createElement(a);a.src=u;google.timers&&google.timers.load&&google.tick&&google.tick("load","xjsls");document.body.appendChild(a)},0);})();(function(){window.google.xjsu='/xjs/_/js/k\x3dxjs.hp.en.XsBchCh7bLs.O/m\x3dsb_he,d/am\x3dAE-wOQ/d\x3d1/rs\x3dACT90oGpUq2u9G5OXekOjNEpL2rNntM8Ng';})();function _DumpException(e){throw e;}
function _F_installCss(c){}
(function(){google.jl={dw:false,em:[],emw:false,lls:'default',pdt:0,snet:true,uwp:true};})();(function(){var pmc='{\x22d\x22:{},\x22sb_he\x22:{\x22agen\x22:false,\x22cgen\x22:false,\x22client\x22:\x22heirloom-hp\x22,\x22dh\x22:true,\x22dhqt\x22:true,\x22ds\x22:\x22\x22,\x22ffql\x22:\x22en\x22,\x22fl\x22:true,\x22host\x22:\x22google.com\x22,\x22isbh\x22:28,\x22jsonp\x22:true,\x22msgs\x22:{\x22cibl\x22:\x22Очистити пошуковий запит\x22,\x22dym\x22:\x22Можливо, ви мали на увазі:\x22,\x22lcky\x22:\x22Мені пощастить\x22,\x22lml\x22:\x22Докладніше\x22,\x22oskt\x22:\x22Транслітерація\x22,\x22psrc\x22:\x22Цей пошук видалено з вашої \\u003Ca href\x3d\\\x22/history\\\x22\\u003Eвеб-історії\\u003C/a\\u003E\x22,\x22psrl\x22:\x22Видалити\x22,\x22sbit\x22:\x22Пошук за зображенням\x22,\x22srch\x22:\x22Пошук Google\x22},\x22ovr\x22:{},\x22pq\x22:\x22\x* Curl_http_done: called premature == 0
* Connection #0 to host www.google.com left intact
22,\x22refpd\x22:true,\x22rfs\x22:[],\x22sbpl\x22:16,\x22sbpr\x22:16,\x22scd\x22:10,\x22stok\x22:\x22wfMIUFI6m4fKX-P_cYuvyDA5cg8\x22,\x22uhde\x22:false}}';google.pmc=JSON.parse(pmc);})();</script> </body>
Thanks very much for this: initial installation was a breeze, and I'm really looking forward to seeing how well it works over the next while.
One minor suggestion I have is with regard to the "How can I keep my USG configuration" FAQ entry:
.. I think it might help those coming to this for the first time if you add a bullet point noting that the config.gateway.json file is to be obtained from the gateway, but stored on the controller. This is of course what the Ubiquiti instructions say, but it's an easy detail to miss if you're just skim-reading, since all of the other work needed to install is just done on the gateway.
I followed the setup instructions on the github page and am seeing the deb repo 404:
$ sudo apt-get update && sudo apt-get install edgeos-dnsmasq-blacklist
Ign https://raw.githubusercontent.com britannic Release.gpg
Ign https://raw.githubusercontent.com britannic Release
Err https://raw.githubusercontent.com britannic/main mips Packages
The requested URL returned error: 404
Ign https://raw.githubusercontent.com britannic/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/britannic/main/binary-mips/Packages The requested URL returned error: 404E: Some index files failed to download. They have been ignored, or old ones used instead.
admin@Router:~$ sudo apt-get update && sudo apt-get install edgeos-dnsmasq-blacklist
Ign https://raw.githubusercontent.com britannic Release.gpg
Ign https://raw.githubusercontent.com britannic Release
Err https://raw.githubusercontent.com britannic/main mips Packages
The requested URL returned error: 404
Ign https://raw.githubusercontent.com britannic/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/britannic/main/binary-mips/Packages The requested URL returned error: 404E: Some index files failed to download. They have been ignored, or old ones used instead.
I went back to downloading and installing the deb file manually but there may be an issue with either documentation or the published repo file.
I went back to downloading and installing the deb file manually but there may be an issue with either documentation or the published repo file.
See corresponding UBNT forum post
Console message is as follows:
Is the certificate out of date?
ubnt@ubnt:~$ sudo curl -L https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/public.key | sudo apt-key add -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
Hi there,
Thanks for making this package, it's great to be able to have dns blacklisting right on my ER without any separate network hardware.
EdgeOS has a feature where any .deb files placed in /config/data/firstboot/install-packages
will be installed when the OS is first booted (ie. when upgrading from one release to another). This is a useful feature to install some packages that aren't included by default, and ensure they'll be available to the system when it is being configured.
Ideally, I could put the edgeos-dnsmasq-blacklist file here and the existing config that's being loaded would take effect. Unfortunately, some part of the process does not work, and the device hangs somewhere and never completes the boot process, and I lose access to it. Unfortunately, I can't provide many more details than that. If I had a spare device, I would console in and see what exactly is happening, but I'm unwilling to intentionally break my device at the moment.
If you're willing to investigate further, the script that kicks all this off is /etc/ubnt/ubnt-rcS/ubnt-rcS.sh
, and it attempts to install all packages with the following command:
DEBIAN_FRONTEND=noninteractive dpkg --force-confold --force-confdef -i "${pkgs[@]}" 2>&1 >> /var/log/ubnt-firstboot-pkg-install.log
I suspect that the auto-config setup portion of the Debian installation script isn't happy when being run this early in the boot process, but again, I have no other data, that's just a hunch. Oh, further info, this has happened to me at least twice, under 2.0.x releases, and most likely all with 1.7.7.x releases of blacklist.
At the very least, you could possibly add a warning in the readme about this, and how it might just be the best idea to re-install the package after the first boot has been completed.
Installation went smoothly, however I am unable to add exceptions to the blacklist, according to the README.
configure
set service dns forwarding blacklist exclude photos.app.goo.gl
commit
update-dnsmasq: NOTI[001]17:53:43.097: Starting blacklist update...
update-dnsmasq: INFO[002]17:53:43.100: Removing stale blacklists...
update-dnsmasq: INFO[003]17:53:43.106: excluded-global: downloaded: 1
update-dnsmasq: INFO[004]17:53:43.108: excluded-global: extracted: 1
update-dnsmasq: INFO[005]17:53:43.109: excluded-global: dropped: 0
update-dnsmasq: NOTI[006]17:53:43.111: Total excluded-global: 1, dropped: 0
update-dnsmasq: INFO[007]17:53:43.113: excluded-domains: downloaded: 0
update-dnsmasq: INFO[008]17:53:43.115: excluded-domains: extracted: 0
update-dnsmasq: INFO[009]17:53:43.116: excluded-domains: dropped: 0
update-dnsmasq: INFO[00a]17:53:43.118: excluded-hosts: downloaded: 0
update-dnsmasq: INFO[00b]17:53:43.119: excluded-hosts: extracted: 0
update-dnsmasq: INFO[00c]17:53:43.120: excluded-hosts: dropped: 0
update-dnsmasq: ERRO[00d]17:53:43.278: ReloadDNS():
error: Restarting DNS forwarder and DHCP server: configuration syntax check failed!
%!(EXTRA string=exit status 1)
Commit failed
Since the malwaredomains list has been stale and removed from the package, might be a good idea to remove/update the hyperlinks in the FAQ and/or replace it with a current list or replace the name with an obvious fake.
The No Coin Filter list is current, follows the same structure, and is still in the default package.
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt
Just updated to v1.10.10 of the ER firmware and tried to install adblocking via apt-get method but I'm getting the following:
Ign https://raw.githubusercontent.com wheezy Release.gpg
Ign https://raw.githubusercontent.com wheezy Release
Err https://raw.githubusercontent.com wheezy/main mips Packages
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Ign https://raw.githubusercontent.com wheezy/main Translation-en
W: Failed to fetch https://raw.githubusercontent.com/britannic/debian-repo/master/blacklist/dists/wheezy/main/binary-mips/Packages server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: noneE: Some index files failed to download. They have been ignored, or old ones used instead.
I was able to manually download ind install the .deb file, but only by passing a -k argument to the curl download, then letting dpkg do its thing.
I haven't really tested anything else at this point. Just informing of the issues...
Thanks for all the hard work!
Hi!
I was wondering if there might be a workaround for the following error. When I run update-dnsmasq
, I get the following output:
ubnt@edgerouter:~$ sudo /config/scripts/update-dnsmasq -v
NOTI[001]17:43:03.451: Starting blacklist update...
INFO[002]17:43:03.466: Checking for stale blacklists...
INFO[003]17:43:03.501: blacklisted-subdomains: downloaded: 15
INFO[004]17:43:03.503: blacklisted-subdomains: extracted: 15
INFO[005]17:43:03.504: blacklisted-subdomains: dropped: 0
INFO[006]17:43:03.519: blacklisted-servers: downloaded: 2
INFO[007]17:43:03.521: blacklisted-servers: extracted: 2
INFO[008]17:43:03.525: blacklisted-servers: dropped: 0
INFO[009]17:43:03.564: global-whitelisted-domains: downloaded: 99
INFO[00a]17:43:03.565: global-whitelisted-domains: extracted: 99
INFO[00b]17:43:03.566: global-whitelisted-domains: dropped: 0
INFO[00c]17:43:03.592: whitelisted-servers: downloaded: 1
INFO[00d]17:43:03.593: whitelisted-servers: extracted: 1
INFO[00e]17:43:03.594: whitelisted-servers: dropped: 0
INFO[010]17:43:03.598: Downloading domains source NoBitCoin
INFO[00f]17:43:03.598: Downloading domains source simple_tracking
INFO[011]17:43:03.598: Downloading domains source OISD
WARN[012]17:43:04.653: Unable to get response for https://dbl.oisd.nl/basic/
INFO[013]17:43:04.682: simple_tracking: downloaded: 34
INFO[014]17:43:04.683: simple_tracking: extracted: 34
INFO[015]17:43:04.684: simple_tracking: dropped: 0
INFO[016]17:43:04.787: NoBitCoin: downloaded: 720
INFO[017]17:43:04.788: NoBitCoin: extracted: 714
INFO[018]17:43:04.789: NoBitCoin: dropped: 6
ERRO[019]17:43:04.803: Get "https://dbl.oisd.nl/basic/": x509: certificate has expired or is not yet valid: current time 2022-12-13T17:43:04+01:00 is after 2021-09-30T14:01:15Z
NOTI[01a]17:43:04.805: Total entries found: 871
NOTI[01b]17:43:04.806: Total entries extracted 865
NOTI[01c]17:43:04.807: Total entries dropped 6
INFO[01d]17:43:07.103: Successfully restarted dnsmasq
NOTI[01e]17:43:07.105: Blacklist update completed......
It seems that something is wrong with the dbl.oisd.nl/basic list, even though navigating to that URL seems to be fine in my browser.
Is there a way to ignore the certificate issue, so I can test of it works on my network?
EdgeMAX versions: v1.9.7+hotfix.4-v2.0.9-hotfix.2
replace hotfix.2 with hotfix.6
When I install this on my ER-X SFP I am getting this at the end of the install:
post-install: INFO[078]20:34:30.295: set service dns forwarding blacklist hosts source sysctl.org url http://sysctl.org/cameleon/hosts
post-install: INFO[079]20:34:30.511: set system task-scheduler task update_blacklists executable path /config/scripts/update-dnsmasq-cronjob.sh
post-install: INFO[07a]20:34:30.688: set system task-scheduler task update_blacklists executable arguments 10800
post-install: INFO[07b]20:34:30.921: set system task-scheduler task update_blacklists interval 1d
At least one interface must be configured for DNS forwarding parameter 'listen-on' or 'except-interface' but not both
[ system task-scheduler ]
Restarting periodic command scheduler: cronStopping periodic command scheduler: cron.
Starting periodic command scheduler: cron.
Commit failed
post-install: ERRO[07c]20:34:33.328: commit failed!
Saving configuration to '/config/config.boot'...
Done
post-install: INFO[07d]20:34:34.709: save
post-install: INFO[07e]20:34:34.896: end
post-install: INFO[07f]20:34:35.069: chgrp -R vyattacfg /opt/vyatta/config
This makes the install fail.
I get this message either way I install it.
Through apt-get or through dpkg
EdgeRouter ERLite-3, with EdgeOS 1.10.11
In situation when booting router and internet connenction is not fully ready yet (in my case caused by not ready DNS resolver) blacklist update fails:
NOTI[001]2020-11-21 12:21:34.815: Starting blacklist update...
CRIT[002]2020-11-21 12:21:37.816: No internet access, aborting blacklist update!
This is quite normal. The problem is, that existing blacklist configuration is not saved and router is wide open.
When running update (or waiting for cron/task scheduler to do that):
root@edge:/config/scripts# ./update-dnsmasq
Removing stale dnsmasq blacklist files, because no blacklist configuration has been detected
Still, no blackllist active. Addig -safe
switch changes nothig. Manually pointing to config file:
root@edge:/config/scripts# ./update-dnsmasq -f /config/config.boot
makes all updates:
root@edge:/config/scripts# tail /var/log/update-dnsmasq.log
INFO[022]2020-11-21 12:42:59.807: openphish: extracted: 3130
INFO[023]2020-11-21 12:42:59.808: openphish: dropped: 1430
INFO[024]2020-11-21 12:43:18.893: githubSteveBlack: downloaded: 57865
INFO[025]2020-11-21 12:43:18.894: githubSteveBlack: extracted: 43834
INFO[026]2020-11-21 12:43:18.895: githubSteveBlack: dropped: 14031
NOTI[027]2020-11-21 12:43:20.010: Total entries found: 126632
NOTI[028]2020-11-21 12:43:20.011: Total entries extracted 109280
NOTI[029]2020-11-21 12:43:20.011: Total entries dropped 17352
INFO[02a]2020-11-21 12:43:45.860: Successfully restarted dnsmasq
NOTI[02b]2020-11-21 12:43:45.860: Blacklist update completed......
However, the configuration is not active anyway. There is no blacklist
branch in show services dns resolver
.
The only method I foud to bring back backlist is full reconfiguration with sudo dpkg-reconfigure edgeos-dnsmasq-blacklist
. This means all custiomizations and exception lists go to waste.
Am I doing something wrong?
youtubeadsblacklist hasn't seen an update in nearly 2 years. When I leave this source in the config, Youtube ads are blocked – but, unfortunately, so are Youtube videos. Removing the source re-enables playback.
cc: @anudeepND
First of all, thank you for creating this application. It is really awesome to be able to use blocklists without any extra hardware.
I have recently been having issues with an USG. The application runs out of memory before completion.
I was using v1.2.4.2, and I tried upgrading to v1.2.4.5, but it didn't help. I tried rebooting and re-running the update task, but it ran out of memory again.
I don't know much about Go development, but I tried to do some debugging.
I was using the following command in each test case: time /config/scripts/update-dnsmasq -v
And while the script was running, I was watching memory usage in another session: watch free -h
total used free shared buffers cached
Mem: 483M 305M 177M 0B 24M 115M
-/+ buffers/cache: 165M 317M
Swap: 0B 0B 0B
NOTI[02f]22:05:01.208: Total entries found: 418129
NOTI[030]22:05:01.209: Total entries extracted 362205
NOTI[031]22:05:01.210: Total entries dropped 55924
ERRO[032]22:05:01.213: ReloadDNS(): error: fork/exec /bin/bash: cannot allocate memory
The least amount of free memory after Total entries dropped
, right before the error:
total used free shared buffers cached
Mem: 483M 437M 46M 0B 22M 108M
-/+ buffers/cache: 305M 178M
Swap: 0B 0B 0B
Time:
real 1m45.933s
user 1m55.940s
sys 0m19.680s
I've added 2 lines in main.go
in the latest master
branch:
6 │ "runtime/debug"
...
113 │ debug.SetGCPercent(20)
I've built with make mips
and SCP'd the executable to the USG.
It took a longer time to finish, but it never ran out of memory this way.
NOTI[02f]21:53:22.825: Total entries found: 418129
NOTI[030]21:53:22.852: Total entries extracted 362209
NOTI[031]21:53:22.854: Total entries dropped 55920
The least amount of free memory after Total entries dropped
:
total used free shared buffers cached
Mem: 483M 429M 54M 0B 24M 114M
-/+ buffers/cache: 290M 192M
Swap: 0B 0B 0B
Memory freed by GC:
total used free shared buffers cached
Mem: 483M 331M 152M 0B 24M 114M
-/+ buffers/cache: 193M 290M
Swap: 0B 0B 0B
Memory before completion:
total used free shared buffers cached
Mem: 483M 377M 106M 0B 24M 114M
-/+ buffers/cache: 238M 245M
Swap: 0B 0B 0B
Time:
real 3m20.654s
user 3m39.210s
sys 0m31.110s
I found notion.so was blacklisted:
/etc/dnsmasq.d/hosts.openphish.blacklist.conf:address=/www.notion.so/0.0.0.0
So I followed the readme, added the following to /etc/dnsmasq.conf
:
address=/www.notion.so/#
But then the following command fails:
$ sudo /etc/init.d/dnsmasq restart
[....] Restarting dnsmasq (via systemctl): dnsmasq.serviceJob for dnsmasq.service failed because the control process exited with error code.
See "systemctl status dnsmasq.service" and "journalctl -xe" for details.
failed!
sudo journalctl -xe
does not show anything.
Is this the right way to whitelist? Thank you!
Does it block something like this nslookup doubleclick.net 9.9.9.9
?
Does it block for IPv6 requests?
For some reason a recent update has prompted Slack to not be able to upload or download attachments, because it blocks files.slack.com.
After using little snitch to see what slack was accessing, I whitelisted files.slack.com, and the problem was fixed.
This is an easy fix, but it’s a very odd thing to block, and could really confuse some other people who are probably pestering Slack support about it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.