waterbywind / edgeos-bl-mgmt Goto Github PK

Previously the script worked well, however now I get the following error:

updBlackList.sh: line 322: syntax error near unexpected token `<'

updBlackList.sh: line 322: ` done < <(sed -e 's/#.$//g' -e '/^[[:space:]]$/d' ${fpUrlList})'

Any ideas?

Thanks again for this great solution. Works like a charm.

You might want to update the README to the "new" (2018) 1.0.4 version of iprange.
Version 1.0.3 is not in the pool anymore.

curl -O http://http.us.debian.org/debian/pool/main/i/iprange/iprange_1.0.4+ds-2_mips.deb
curl -O http://http.us.debian.org/debian/pool/main/i/iprange/iprange_1.0.4+ds-2_mipsel.deb

I am using 1.0.4+ds-2_mipsel on an ERX SFP - nothing broken.

Cheers

Can you provide steps to confirm that the new rulesets have been applied?

EdgeRouter PoE v1.9.7+hotfix.3

Including the https://isc.sans.edu/ipsascii.html blacklist results in the following error:

iprange: Invalid address 185.176.026.10018701290432019021320190417.
iprange: Cannot understand line No 56515 from /tmp/.BL/updBlackList.2-5TS8vD: 185.176.026.10018701290432019021320190417

http://blocklist.greensnow.co/greensnow.txt
http://charles.the-haleys.org/ssh_dico_attack_hdeny_format.php/hostsdeny.txt
http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt
http://talosintel.com/feeds/ip-filter.blf
http://www.binarydefense.com/banlist.txt
https://www.badips.com/get/list/any/2?age=30d

I have an ER-X and following the install options for IPRange I had an error:

package architecture (mips) does not match system (mipsel)
Errors were encountered while processing:
 iprange_1.0.3+ds-1_mips.deb

Checked out that repository and there was a mipsel out there so I downloaded that:
curl -O http://http.us.debian.org/debian/pool/main/i/iprange/iprange_1.0.3+ds-1_mipsel.deb

Just wondering if your script will use this version and wanted you to be aware of it.

I pretty new to this, but trying to commit the example fw rules gives me this?

sav[ firewall ipv6-name wan-dmz-6 ]
Firewall config error: 'action' must be defined

e[ firewall name wan-dmz-4 ]
Firewall config error: 'action' must be defined

[ firewall name wan-lan-4 ]
Firewall config error: 'action' must be defined
[ firewall name wan-self-4 ]
Firewall config error: 'action' must be defined

[ firewall ipv6-name wan-self-6 ]
Firewall config error: 'action' must be defined

[ firewall ipv6-name wan-lan-6 ]
Firewall config error: 'action' must be defined

Any idea why I might be seeing this error? I'm trying to modify this script as per: https://community.ubnt.com/t5/EdgeMAX/statuscake-auto-address-group-population/m-p/1509015#M102637

But I can't seem to get it to work. This is on edgeos 1.9.0.

Thanks!

Just want to start by saying the Blacklist Management project is awesome, and you're awesome for maintaining it! Just wanted to report an issue using the default lists FW-Blacklist-URLs, it seems parsing IPv6 is broken at this time. I had to remove each of the IPv6 lists, including BlockList.de which had a few IPv6 addresses, in order for updBlackList.sh to succeed. After removing each list which contained IPv6 addresses, I was sucessfully able to import the IPSet:
IPv4 blocklist items fetched: 23470, unique: 21522, final: 21522
Total IPv4 prefix length count (including hosts): 27
IPv6 blocklist items fetched: 0, unique: 0, final: 0
Total IPv6 prefix length count (including hosts): 0

Currently on EdgeRouter X v2.0.8 but also tried on 1.10.10

ubnt@ubnt:~$ sudo /config/scripts/updBlackList.sh Missing executable '/usr/bin/iprange'. Will not optimize IPsets Starting at 00:52:56 MST Wed 11 Dec 2019 Fetching 'https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt' Fetching 'https://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt' Fetching 'https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt' Fetching 'https://www.spamhaus.org/drop/drop.txt' Fetching 'https://www.spamhaus.org/drop/dropv6.txt' Fetching 'https://www.spamhaus.org/drop/edrop.txt' Fetching 'https://lists.blocklist.de/lists/all.txt' Fetching 'https://iplists.firehol.org/files/firehol_level1.netset' Fetching 'https://www.okean.com/chinacidr.txt' Processing block file list (IPv4): ' 00_www.team-cymru.org_fullbogons-ipv4.txt 01_www.team-cymru.org_fullbogons-ipv6.txt 02_rules.emergingthreats.net_emerging-Block-IPs.txt 03_www.spamhaus.org_drop.txt 04_www.spamhaus.org_dropv6.txt 05_www.spamhaus.org_edrop.txt 06_lists.blocklist.de_all.txt 07_iplists.firehol.org_firehol_level1.netset 08_www.okean.com_chinacidr.txt' Applying IPset (IPv4) Processing block file list (IPv6): ' 00_www.team-cymru.org_fullbogons-ipv4.txt 01_www.team-cymru.org_fullbogons-ipv6.txt 02_rules.emergingthreats.net_emerging-Block-IPs.txt 03_www.spamhaus.org_drop.txt 04_www.spamhaus.org_dropv6.txt 05_www.spamhaus.org_edrop.txt 06_lists.blocklist.de_all.txt 07_iplists.firehol.org_firehol_level1.netset 08_www.okean.com_chinacidr.txt' Applying IPset (IPv6) ipset v6.30: Error in line 2: Syntax error: cannot parse 1000::: resolving to IPv4 address failed FATAL: inet6 ipset restore failed: error 1

woops, wrong project

This is great work, and hugely helpful. Thank you for making this available to the community!

One of the sources contained a wrong IP (195.78.82.0/234 China)

which returned a error
ipset v6.23: Error in line 32227: Syntax error: '234' is out of range 0-32

This stops the whole process of updating.
Could there be a filter that would ignore such wrong entries?

Anyone looked at making this work on UDM/UDMPRO?

http://malc0de.com/bl/IP_Blacklist.txt

For some reason, adding the blacklist above inflates the generated IP set so that it cannot be created even when MaxElem is set to something ridiculous like a million entries or so.

Hi!

When i try to run the update script, I get this error - and I cant figure out why.

lundaa@LundaaDK-ER6P# sudo /config/scripts/updBlackList.sh

/config/scripts/updBlackList.sh: line 606: syntax error near unexpected token `then'

/config/scripts/updBlackList.sh: line 606: `trap atExit EXIT startup doFetch if [[ -n "${fwGroupNets4}" ]]; then'

[edit]

What to do ?

When running the update script, I get all the way past the download of the files and IPV4 ipset, then get this message. I am using an Edge Router 4.

Applying IPset (IPv6)
ipset v6.23: Error in line 2: Syntax error: cannot parse 1000::: resolving to IPv4 address failed
FATAL: inet6 ipset restore failed: error 1

To silence the following error message from sed, apply the patch below:
sed: character class syntax is [[:space:]], not [:space:]

Patch:

diff --git a/updBlackList.sh b/updBlackList.sh
index d04caca..246b9d0 100755
--- a/updBlackList.sh
+++ b/updBlackList.sh
@@ -475,7 +475,7 @@ doProcess4()
     cat ${flBlockList} | \
         sed -e '/^[;#]/d' \
             -e '/ERROR/d' \
-            -e '/[:\::]/d' \
+            -e '/[\:\::]/d' \
             -e 's/ .*//g' \
             -e 's#//.*##g' \
             -e 's/[^0-9,.,/]*//g' \

Not sure if anyone still maintains this, but there appears to be an issue pulling the Cisco Talos feed (https://www.talosintelligence.com/documents/ip-blacklist). My guess is the fact that the URL redirects to an Amazon S3 bucket is the issue.

USG has moved to a new labelling system for firewall groups - instead of using the actual name they use 'hashed names'. Would be great to fix this somehow...

More a question than an issue.
I have this setup as per the readme however it is not 100% clear to me, should the FW policies (e.g. WAN - LAN ) have interfaces defined?
same question for Nets4-BlackList etc, should they have a network defined? (currently empty)

Many thanks :)

I'm thinking of the specific case of a malware talking to its C&C server via UDP. These rules won't block the upload traffic. Given the growing number of ransomware with the goal of stealing private data, a upload-only UDP connection should be enough for them for their job.

Hi,

Do we need to actually specify which "Interfaces" for this Firewall Blocklist filtering method to apply?

Thanks for your help,
JR