libreqoe / libreqos

A Quality of Experience and Smart Queue Management system for ISPs. Leverage CAKE to improve network responsiveness, enforce bandwidth plans, and reduce bufferbloat.

LibreQoS is a Quality of Experience (QoE) Smart Queue Management (SQM) system designed for Internet Service Providers to optimize the flow of their network traffic and thus reduce bufferbloat, keep the network responsive, and improve the end-user experience.

Servers running LibreQoS can shape traffic for thousands of customers. On higher-end servers, LibreQoS is capable of shaping 50-80 Gbps of traffic.

Learn more at LibreQoS.io!

Sponsors

LibreQoS' development is made possible by our sponsors, the NLnet Foundation and Equinix.

LibreQoS has been funded through the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission’s Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101069594. Learn more at https://nlnet.nl/project/LibreQoS/

Equinix supports LibreQoS through its Open Source program – providing access to hardware resources on its Equinix Metal infrastructure. Equinix’ support has been crucial for LibreQoS to scale past 10Gbps for higher-bandwidth networks. Learn more about Equinix Metal here. Learn more about Equinix Metal here.

Support LibreQoS

Please support the continued development of LibreQoS by sponsoring us via GitHub Sponsors or Patreon.

Documentation / Get Started / System Requirements

ReadTheDocs

LibreQoS Chat

Our Zulip chat server is available at https://chat.libreqos.io/join/fvu3cerayyaumo377xwvpev6/.

Long-Term Stats (LTS)

Long-Term Stats (LTS) is an analytics service built for LibreQoS that revolutionizes the way you track and analyze your network. With flexible time window views ranging from 5 minutes to 1 month, LTS gives you comprehensive insights into your network's performance. Built from the ground-up for performance and efficiency, LTS greatly outperforms our original InfluxDB plugin, and gives you rapidly rendered data to help you maximize your network performance.

We provide a free 30-day trial of LTS, after which the rate is $0.30 USD per shaped subscriber. You can enroll in the 30-day free trial by upgrading to the latest version of LibreQoS v1.4 and selecting "Start Stats Free Trial" in the top-right corner of the local LibreQoS WebUI.

libreqos's Issues

I'm getting some errors and shapping is not working

dmesg

python3 ./LibreQoS.py

tc filter delete dev enp5s0f0
Error: Class doesn't support blocks.
We have an error talking to the kernel
tc filter delete dev enp5s0f0 root
Error: Specified class doesn't exist.
We have an error talking to the kernel
tc qdisc delete dev enp5s0f0 root
tc qdisc delete dev enp5s0f0
Error: Failed to find qdisc with specified handle.
tc filter delete dev enp5s0f0
Error: Class doesn't support blocks.
We have an error talking to the kernel
tc filter delete dev enp5s0f0 root
Error: Specified class doesn't exist.
We have an error talking to the kernel
tc qdisc delete dev enp5s0f0 root
Error: Cannot delete qdisc with handle of zero.
tc qdisc delete dev enp5s0f0
Error: Failed to find qdisc with specified handle.
This Network Interface Card has 24 queues avaialble.
./xdp-cpumap-tc/bin/xps_setup.sh -d enp5s0f0 --default --disable
./xdp-cpumap-tc/bin/xps_setup.sh -d enp5s0f0 --default --disable
./xdp-cpumap-tc/src/xdp_iphash_to_cpu --dev enp5s0f0 --lan
libbpf: Error loading BTF: Invalid argument(22)
libbpf: magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 504
str_off: 504
str_len: 1730
btf_total_size: 2258
[1] PTR (anon) type_id=2
[2] STRUCT xdp_md size=24 vlen=6
data type_id=3 bits_offset=0
data_end type_id=3 bits_offset=32
data_meta type_id=3 bits_offset=64
ingress_ifindex type_id=3 bits_offset=96
rx_queue_index type_id=3 bits_offset=128
egress_ifindex type_id=3 bits_offset=160
[3] TYPEDEF __u32 type_id=4
[4] INT unsigned int size=4 bits_offset=0 nr_bits=32 encoding=(none)
[5] FUNC_PROTO (anon) return=6 args=(1 ctx)
[6] INT int size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[7] FUNC xdp_program type_id=5
[8] STRUCT bpf_map_def size=24 vlen=6
type type_id=4 bits_offset=0
key_size type_id=4 bits_offset=32
value_size type_id=4 bits_offset=64
max_entries type_id=4 bits_offset=96
map_flags type_id=4 bits_offset=128
inner_map_idx type_id=4 bits_offset=160
[9] VAR map_ip_hash type_id=8 linkage=1
[10] VAR map_txq_config type_id=8 linkage=1
[11] VAR map_ifindex_type type_id=8 linkage=1
[12] VAR cpu_map type_id=8 linkage=1
[13] VAR cpus_available type_id=8 linkage=1
[14] INT char size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
[15] ARRAY (anon) type_id=14 index_type_id=16 nr_elems=4
[16] INT ARRAY_SIZE_TYPE size=4 bits_offset=0 nr_bits=32 encoding=(none)
[17] VAR _license type_id=15 linkage=1
[18] DATASEC license size=0 vlen=1 size == 0

libbpf: Error loading ELF section .BTF: -22. Ignored and continue.
TC workaround for mapname: map_ifindex_type map_fd:7

Added CPU:0 queue_size:2048

Added CPU:1 queue_size:2048

Added CPU:2 queue_size:2048

Added CPU:3 queue_size:2048

Added CPU:4 queue_size:2048

Added CPU:5 queue_size:2048

Added CPU:6 queue_size:2048

Added CPU:7 queue_size:2048

Documentation:

XDP: Lookup IPv4 and redirect to CPU hash

This program loads the XDP eBPF program into the kernel.

Use the cmdline tool for add/removing dest IPs to the hash

• Attached to device:enp5s0f0 (ifindex:6) prog_id:64

./xdp-cpumap-tc/src/xdp_iphash_to_cpu --dev enp5s0f0 --wan
libbpf: Error loading BTF: Invalid argument(22)
libbpf: magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 504
str_off: 504
str_len: 1730
btf_total_size: 2258
[1] PTR (anon) type_id=2
[2] STRUCT xdp_md size=24 vlen=6
data type_id=3 bits_offset=0
data_end type_id=3 bits_offset=32
data_meta type_id=3 bits_offset=64
ingress_ifindex type_id=3 bits_offset=96
rx_queue_index type_id=3 bits_offset=128
egress_ifindex type_id=3 bits_offset=160
[3] TYPEDEF __u32 type_id=4
[4] INT unsigned int size=4 bits_offset=0 nr_bits=32 encoding=(none)
[5] FUNC_PROTO (anon) return=6 args=(1 ctx)
[6] INT int size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[7] FUNC xdp_program type_id=5
[8] STRUCT bpf_map_def size=24 vlen=6
type type_id=4 bits_offset=0
key_size type_id=4 bits_offset=32
value_size type_id=4 bits_offset=64
max_entries type_id=4 bits_offset=96
map_flags type_id=4 bits_offset=128
inner_map_idx type_id=4 bits_offset=160
[9] VAR map_ip_hash type_id=8 linkage=1
[10] VAR map_txq_config type_id=8 linkage=1
[11] VAR map_ifindex_type type_id=8 linkage=1
[12] VAR cpu_map type_id=8 linkage=1
[13] VAR cpus_available type_id=8 linkage=1
[14] INT char size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
[15] ARRAY (anon) type_id=14 index_type_id=16 nr_elems=4
[16] INT ARRAY_SIZE_TYPE size=4 bits_offset=0 nr_bits=32 encoding=(none)
[17] VAR _license type_id=15 linkage=1
[18] DATASEC license size=0 vlen=1 size == 0

libbpf: Error loading ELF section .BTF: -22. Ignored and continue.
TC workaround for mapname: map_ifindex_type map_fd:7

Added CPU:0 queue_size:2048

Added CPU:1 queue_size:2048

Added CPU:2 queue_size:2048

Added CPU:3 queue_size:2048

Added CPU:4 queue_size:2048

Added CPU:5 queue_size:2048

Added CPU:6 queue_size:2048

Added CPU:7 queue_size:2048

Documentation:

XDP: Lookup IPv4 and redirect to CPU hash

This program loads the XDP eBPF program into the kernel.

Use the cmdline tool for add/removing dest IPs to the hash

• Attached to device:enp5s0f0 (ifindex:6) prog_id:67

./xdp-cpumap-tc/src/xdp_iphash_to_cpu_cmdline --clear
iphash_modify() IP:10.168.0.100 key:0x6400A80A TC-handle:0x0
key: 0x6400A80A

./xdp-cpumap-tc/src/tc_classify --dev-egress enp5s0f0
TC: Control program for tc_classify_kern.o

• When using --dev, loads TC-egress filter calling BPF program

• Config of map_txq_config, that control CPU to queue_mapping

• List current queue_mapping (txq) config via --list

Map filename: /sys/fs/bpf/tc/globals/map_txq_config

Dev:enp5s0f0 -- Loading: TC-clsact egress

• Run: tc qdisc del dev enp5s0f0 clsact 2> /dev/null

• Run: tc qdisc add dev enp5s0f0 clsact

• Run: tc filter add dev enp5s0f0 egress prio 1 handle 1 bpf da obj ./xdp-cpumap-tc/src/tc_classify_kern.o sec tc_classify

./xdp-cpumap-tc/src/tc_classify --dev-egress enp5s0f0
TC: Control program for tc_classify_kern.o

• When using --dev, loads TC-egress filter calling BPF program

• Config of map_txq_config, that control CPU to queue_mapping

• List current queue_mapping (txq) config via --list

Map filename: /sys/fs/bpf/tc/globals/map_txq_config

Dev:enp5s0f0 -- Loading: TC-clsact egress

• Run: tc qdisc del dev enp5s0f0 clsact 2> /dev/null

• Run: tc qdisc add dev enp5s0f0 clsact

• Run: tc filter add dev enp5s0f0 egress prio 1 handle 1 bpf da obj ./xdp-cpumap-tc/src/tc_classify_kern.o sec tc_classify

tc qdisc replace dev enp5s0f0 root handle 7FFF: mq
tc qdisc add dev enp5s0f0 parent 7FFF:1 handle 1: htb default 2
tc class add dev enp5s0f0 parent 1: classid 1:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 1:1 cake diffserv4
tc class add dev enp5s0f0 parent 1:1 classid 1:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 1:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:2 handle 2: htb default 2
tc class add dev enp5s0f0 parent 2: classid 2:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 2:1 cake diffserv4
tc class add dev enp5s0f0 parent 2:1 classid 2:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 2:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:3 handle 3: htb default 2
tc class add dev enp5s0f0 parent 3: classid 3:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 3:1 cake diffserv4
tc class add dev enp5s0f0 parent 3:1 classid 3:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 3:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:4 handle 4: htb default 2
tc class add dev enp5s0f0 parent 4: classid 4:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 4:1 cake diffserv4
tc class add dev enp5s0f0 parent 4:1 classid 4:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 4:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:5 handle 5: htb default 2
tc class add dev enp5s0f0 parent 5: classid 5:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 5:1 cake diffserv4
tc class add dev enp5s0f0 parent 5:1 classid 5:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 5:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:6 handle 6: htb default 2
tc class add dev enp5s0f0 parent 6: classid 6:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 6:1 cake diffserv4
tc class add dev enp5s0f0 parent 6:1 classid 6:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 6:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:7 handle 7: htb default 2
tc class add dev enp5s0f0 parent 7: classid 7:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 7:1 cake diffserv4
tc class add dev enp5s0f0 parent 7:1 classid 7:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 7:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:8 handle 8: htb default 2
tc class add dev enp5s0f0 parent 8: classid 8:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 8:1 cake diffserv4
tc class add dev enp5s0f0 parent 8:1 classid 8:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 8:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:9 handle 9: htb default 2
tc class add dev enp5s0f0 parent 9: classid 9:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 9:1 cake diffserv4
tc class add dev enp5s0f0 parent 9:1 classid 9:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 9:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:10 handle 10: htb default 2
tc class add dev enp5s0f0 parent 10: classid 10:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 10:1 cake diffserv4
tc class add dev enp5s0f0 parent 10:1 classid 10:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 10:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:11 handle 11: htb default 2
tc class add dev enp5s0f0 parent 11: classid 11:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 11:1 cake diffserv4
tc class add dev enp5s0f0 parent 11:1 classid 11:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 11:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:12 handle 12: htb default 2
tc class add dev enp5s0f0 parent 12: classid 12:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 12:1 cake diffserv4
tc class add dev enp5s0f0 parent 12:1 classid 12:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 12:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:13 handle 13: htb default 2
tc class add dev enp5s0f0 parent 13: classid 13:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 13:1 cake diffserv4
tc class add dev enp5s0f0 parent 13:1 classid 13:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 13:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:14 handle 14: htb default 2
tc class add dev enp5s0f0 parent 14: classid 14:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 14:1 cake diffserv4
tc class add dev enp5s0f0 parent 14:1 classid 14:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 14:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:15 handle 15: htb default 2
tc class add dev enp5s0f0 parent 15: classid 15:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 15:1 cake diffserv4
tc class add dev enp5s0f0 parent 15:1 classid 15:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 15:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:16 handle 16: htb default 2
tc class add dev enp5s0f0 parent 16: classid 16:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 16:1 cake diffserv4
tc class add dev enp5s0f0 parent 16:1 classid 16:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 16:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:17 handle 17: htb default 2
tc class add dev enp5s0f0 parent 17: classid 17:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 17:1 cake diffserv4
tc class add dev enp5s0f0 parent 17:1 classid 17:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 17:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:18 handle 18: htb default 2
tc class add dev enp5s0f0 parent 18: classid 18:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 18:1 cake diffserv4
tc class add dev enp5s0f0 parent 18:1 classid 18:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 18:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:19 handle 19: htb default 2
tc class add dev enp5s0f0 parent 19: classid 19:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 19:1 cake diffserv4
tc class add dev enp5s0f0 parent 19:1 classid 19:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 19:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:20 handle 20: htb default 2
tc class add dev enp5s0f0 parent 20: classid 20:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 20:1 cake diffserv4
tc class add dev enp5s0f0 parent 20:1 classid 20:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 20:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:21 handle 21: htb default 2
tc class add dev enp5s0f0 parent 21: classid 21:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 21:1 cake diffserv4
tc class add dev enp5s0f0 parent 21:1 classid 21:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 21:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:22 handle 22: htb default 2
tc class add dev enp5s0f0 parent 22: classid 22:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 22:1 cake diffserv4
tc class add dev enp5s0f0 parent 22:1 classid 22:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 22:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:23 handle 23: htb default 2
tc class add dev enp5s0f0 parent 23: classid 23:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 23:1 cake diffserv4
tc class add dev enp5s0f0 parent 23:1 classid 23:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 23:2 cake diffserv4
tc qdisc add dev enp5s0f0 parent 7FFF:24 handle 24: htb default 2
tc class add dev enp5s0f0 parent 24: classid 24:1 htb rate 1000mbit ceil 1000mbit
tc qdisc add dev enp5s0f0 parent 24:1 cake diffserv4
tc class add dev enp5s0f0 parent 24:1 classid 24:2 htb rate 125.0mbit ceil 500mbit prio 5
tc qdisc add dev enp5s0f0 parent 24:2 cake diffserv4
tc qdisc replace dev enp5s0f0 root handle 7FFF: mq
tc qdisc add dev enp5s0f0 parent 7FFF:1 handle 1: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 1: classid 1:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 1:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 1:1 classid 1:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 1:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:2 handle 2: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 2: classid 2:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 2:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 2:1 classid 2:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 2:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:3 handle 3: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 3: classid 3:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 3:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 3:1 classid 3:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 3:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:4 handle 4: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 4: classid 4:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 4:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 4:1 classid 4:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 4:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:5 handle 5: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 5: classid 5:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 5:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 5:1 classid 5:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 5:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:6 handle 6: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 6: classid 6:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 6:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 6:1 classid 6:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 6:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:7 handle 7: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 7: classid 7:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 7:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 7:1 classid 7:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 7:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:8 handle 8: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 8: classid 8:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 8:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 8:1 classid 8:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 8:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:9 handle 9: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 9: classid 9:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 9:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 9:1 classid 9:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 9:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:10 handle 10: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 10: classid 10:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 10:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 10:1 classid 10:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 10:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:11 handle 11: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 11: classid 11:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 11:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 11:1 classid 11:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 11:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:12 handle 12: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 12: classid 12:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 12:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 12:1 classid 12:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 12:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:13 handle 13: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 13: classid 13:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 13:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 13:1 classid 13:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 13:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:14 handle 14: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 14: classid 14:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 14:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 14:1 classid 14:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 14:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:15 handle 15: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 15: classid 15:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 15:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 15:1 classid 15:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 15:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:16 handle 16: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 16: classid 16:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 16:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 16:1 classid 16:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 16:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:17 handle 17: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 17: classid 17:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 17:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 17:1 classid 17:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 17:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:18 handle 18: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 18: classid 18:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 18:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 18:1 classid 18:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 18:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:19 handle 19: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 19: classid 19:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 19:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 19:1 classid 19:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 19:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:20 handle 20: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 20: classid 20:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 20:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 20:1 classid 20:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 20:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:21 handle 21: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 21: classid 21:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 21:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 21:1 classid 21:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 21:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:22 handle 22: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 22: classid 22:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 22:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 22:1 classid 22:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 22:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:23 handle 23: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 23: classid 23:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 23:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 23:1 classid 23:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 23:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.
tc qdisc add dev enp5s0f0 parent 7FFF:24 handle 24: htb default 2
Error: Exclusivity flag on, cannot modify.
tc class add dev enp5s0f0 parent 24: classid 24:1 htb rate 1000mbit ceil 1000mbit
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 24:1 cake diffserv4
RTNETLINK answers: Invalid argument
tc class add dev enp5s0f0 parent 24:1 classid 24:2 htb rate 125.0mbit ceil 500mbit prio 5
RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 24:2 cake diffserv4
Error: Exclusivity flag on, cannot modify.

Site_1
Download: 95 to 100 Mbps
Upload: 95 to 100 Mbps
tc class add dev enp5s0f0 parent 1:1 classid 3 htb rate 95mbit ceil 100mbit prio 3
tc class add dev enp5s0f0 parent 1:1 classid 3 htb rate 95mbit ceil 100mbit prio 3
RTNETLINK answers: File exists

AP_A
Download: 86 to 90 Mbps
Upload: 86 to 90 Mbps
tc class add dev enp5s0f0 parent 1:3 classid 5 htb rate 86mbit ceil 90mbit prio 3
tc class add dev enp5s0f0 parent 1:3 classid 5 htb rate 86mbit ceil 90mbit prio 3
RTNETLINK answers: File exists

  Moto G9 Plus
  Download:  2 to 3 Mbps
  Upload:    2 to 3 Mbps
  tc class add dev enp5s0f0 parent 1:5 classid 6 htb rate 2mbit ceil 3mbit prio 3
  tc qdisc add dev enp5s0f0 parent 1:6 cake diffserv4
  tc class add dev enp5s0f0 parent 1:5 classid 6 htb rate 2mbit ceil 3mbit prio 3

RTNETLINK answers: File exists
tc qdisc add dev enp5s0f0 parent 1:6 cake diffserv4
Error: Exclusivity flag on, cannot modify.
./xdp-cpumap-tc/src/xdp_iphash_to_cpu_cmdline --add --ip 10.168.0.100 --cpu 0 --classid 1:6
iphash_modify() IP:10.168.0.100 key:0x6400A80A TC-handle:0x10006
key: 0x6400A80A

Successful run completed on 02/06/2022 15:28:01
Program complete

Disable Offloading in Guest VM v1.0

1. Is the heading correct? Disable offloading in Guest vm or Just Disable offloading ?

2. while creating a symlink the command gives an error
   sudo systemctl service enable offloadOff.service
   Unknown command verb service.

if i run without the service works
sudo systemctl enable offloadOff.service

Install LibreQoS and dependencies v1.1

Install LibreQoS and dependencies

On running the below commands i get some warning

python3 -m pip install ipaddress schedule 'influxdb-client[ciso8601]' requests
sudo python3 -m pip install ipaddress schedule 'influxdb-client[ciso8601]' requests

WARNING: influxdb-client 1.26.0 does not provide the extra 'ciso8601'

Anything to be worried of?

Running as a service problem

Hello,

Main Problem: tried to run LibreQoS as a service ( but after reboot this is the status)

ubuntu@ubuntu:~$ sudo systemctl status LibreQoS.service
× LibreQoS.service
Loaded: loaded (/etc/systemd/system/LibreQoS.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2022-06-29 02:15:55 UTC; 25min ago
Process: 895 ExecStart=/usr/bin/python3 /home/ubuntu/LibreQoS/v1.1/scheduled.py (code=exited, status=200/CHDIR)
Main PID: 895 (code=exited, status=200/CHDIR)
CPU: 1ms

Jun 29 02:15:55 ubuntu systemd[1]: LibreQoS.service: Scheduled restart job, restart counter is at 5.
Jun 29 02:15:55 ubuntu systemd[1]: Stopped LibreQoS.service.
Jun 29 02:15:55 ubuntu systemd[1]: LibreQoS.service: Start request repeated too quickly.
Jun 29 02:15:55 ubuntu systemd[1]: LibreQoS.service: Failed with result 'exit-code'.
Jun 29 02:15:55 ubuntu systemd[1]: Failed to start LibreQoS.service.
ubuntu@ubuntu:~$

------- next

Maybe related problem: Manually running graph.py and graphBandwidth.py works (i think)
but graphLatency.py is stuck and stays in "Retrieving device statistics" and i have to stop it manually.

ubuntu@ubuntu:/LibreQoS/v1.1$ sudo python3 ./graph.py
Retrieving device statistics
Computing parent node statistics
Writing data to InfluxDB
Added 0 points to InfluxDB.
Graphs updated within 0 seconds.
ubuntu@ubuntu:/LibreQoS/v1.1$ sudo python3 ./graphBandwidth.py
Retrieving device statistics
Computing parent node statistics
Writing data to InfluxDB
Added 0 points to InfluxDB.
Graphs updated within 0.02 seconds.
ubuntu@ubuntu:~/LibreQoS/v1.1$ sudo python3 ./graphLatency.py
Retrieving device statistics
^Z
[1]+ Stopped sudo python3 ./graphLatency.py

------end

Backhaul level hierarchy?

I know there are other posts on this, but I didn't want to muddy them up.

Here's the challenge. A long chain of backhauls with multiple sites in the chain and of coarse, multiple APs per site.

Edge->backhaul_Edge-to-1_2Gbps->site1>backhaul_1-to-2_1Gbps->site2->backhaul_x-to-y_zMbps...

The challenge is that the first backhaul really would like an HTB type shaper that encompasses all downstream sites in the 'normal' routing path. Forget redundant paths here so not confusing topology to work around. Site shapers is nice, that could be used to accommodate a known backhaul limit. And 'undersizing' the site shaper could be used to reserve capacity for next hops manually. You could see this getting out of hand really quickly though.

So Sites{1..n} should all get some equal cut of backhaul 'Edge-to-2'
next hope, sites{2..n} should get some equal cut of backhaul '1-to-2'
and so on.

How rough would it be stacking up HTBs like this? I realize that this likely means that the top level HTB get's bound to a single CPU so would encourage getting the fastest cores possible to pull such a thing off, but I'm generally talking about wISP available backhauls here so ~2Gbps is up there in the top range. Not many of us can make short range mmWave 10Gbps radios work in our long links.

I'm imagining a link path csv
edge facing on the left
from site, to site, dl-capacity, ul-capacity (potentially min capacity, max capacity like the client list as well).
Then building this out documenting each link which could build either a stacked HTB (preferred) or just site shapers with those reserves for next hops calculated (somewhat functional)

no git in ubuntu minimal install

In instruction: https://github.com/rchac/LibreQoS/wiki/LibreQoS-v1.1-Installation-&-Usage-Guide-Physical-Server-and-Ubuntu-21.10

Instead of

git clone https://github.com/rchac/LibreQoS.git
sudo apt update
sudo apt install python3-pip clang gcc llvm libelf-dev
python3 -m pip install ipaddress schedule prettytable
sudo python3 -m pip install ipaddress schedule prettytable

Should be:

sudo apt update
sudo apt install python3-pip clang gcc llvm libelf-dev git nano
python3 -m pip install ipaddress schedule prettytable
sudo python3 -m pip install ipaddress schedule prettytable
git clone https://github.com/rchac/LibreQoS.git

Good thig is to have unattended-ubuntu-22.04-install.sh file for any of that kind of projects - this will help in future testing/developing.

Support for one-network-card card configuration, single card configuration, one NIC only

Is it possible to use LibreQoS on one network card using different VLANs or just use one interface for network conectivity?

I don't think xdp-project/xdp-cpumap-tc#7. xdp-cpumap-tc is what we use to redirect packets to the appropriate CPU quickly. If you follow this network design, you can have any VLANs desired on the Core router at least, and as they pass, routed, over the LibreQoS box, they'll be shaped correctly.

Now I have 1x40G QSFP+ card - here on the test machine - I thought that I could use vlan interface

ens1np0.100 
ens1np0.101

for networking.

The target for production is to:

• connect LibreQoS VM router to two different core switches,
• with two passthrough network interfaces and set some teaming (bridge,bond,lacp...)
• make the same on backup router with but with bigger path costs,

So I need six 1x40G cards or three 2x40G cards. (2x interfaces for BGP VM, 4x interfaces for LibreQoS)
Is that a problem? It depends.

This would results in:

• providing better ressistance from unpredictable events.
  (switch/QSFP cable/QSFP module serwer/QSFP switch module/more things failure),
• better maintance (and don't fear about upgrades),

I am just asking because I would like to make sure of design.png.

If we use OSPF (L3) and src/dst address (L3) - Can we use only one network card when the sumarized network throughoutput is less than 40% of total maximum sumarized (upload+download) throughoutput?

Is this sentence from README.txt:
"NIC must have two or more interfaces for traffic shaping." is a requirement or just recomendation?

On LibreQoS there is no NAT - so this is possible I think. Am I right?

Maybe we could use https://man7.org/linux/man-pages/man8/tc-vlan.8.html ?

Supporting Dynamic Network Topologies

From the docs and code, LibreQOS appears to have a model of one link between any two sites. (Correct me if I'm wrong about this!)

Because of weather, it is natural for wireless links to a) go down and come back, up or b) temporarily lose capacity

We have a number of sites that have two parallel links between them. Some of the links are active/active, while others are active/passive.

Example 1:

Site A <=> Site B

Link 1: AF60
Link 2: AF5x-HD

The OSPF route costs are configured to pass all traffic over Link 1 a long as it is available. When rain takes the link down, all traffic is routed over Link 2.

Of course Link 2 has less capacity and Link 1.

Example 2:

Site A <=> Site B

Link 1: AF24
Link 2: AF11

In this case, both radio links have similar capacity and we actively operate them in parallel. That is, both radio links carry traffic at all times.

Of course, severe rain storms occasionally take down the AF24 link and we lose 50% of the bandwidth capacity between the sites.

---

Questions:

1. What is the best way (if there is one) for manage two parallel links with LibreQoS?
2. In an ideal world, I would like to dynamically update the capacity of a Site (Sites.scv) when links go down and come back up. Presuming that if it is possible to track link state changes, it would be easy enough to update Sites.cvs and rerun LIbreQoS.

Bonus scenarios:

We actually have a number of sites that have multiple (non-parallel) paths back to the data center for redundancy.

DC --- Site A ---\
            |     Site C
DC --- Site B ---/

We also have cases like this:

DC --- Site A --- Site B --- Site C

Using HTB more efficiently with cburst and burst

I have been looking over my first major libreqos installation elsewhere and reviewing the code a bit. I am mostly familiar only with cake and with htb + fq_codel as in the sqm-scripts, and after looking over a ton of packet captures over a busy wireless network, I have a possible optimization for some (wifi) networks in mind, that I'd like (others!) to try out at a smaller scale.

HTB has a burst and cburst parameter that limit the size of how many packets can be dequeued from a given htb tier at a time. Our original use for it, in the SQM scripts, was to lighten the CPU load (and lower locking issues) on low end hardware. In the ISP case, as you scale, that problem could also become an issue, but more efficiently using the bandwidth within the packet aggregation that wifi does, is what I'm trying to describe today.

Basically wireless-n wifi can take about 240us to switch between stations and a txop can take up to 5.7ms, you can pack up to 42 packets or 64k into a single aggregate depending on your mcs rate. This gets hairier and hairier with ac and later (and I essentially advertise a TXOP in the beacon of 2ms or less)... sticking with the 802.11n model for points of illustration...

It's really inefficient to send 1 packet (which can take, 10us!) then switch. It's also very noticible with 20 stations, to be taking 5ms each, so a decent compromise is to aim for, say, 500us of multiplexing, figure out how many packets and data can fit into that at the mcs rate you have, and burst that amount out of the shaper.

So I am considering modifying all the classes to also include the burst and cburst parameters (without bothering too much to figure out what the right size is as yet), and if there is anyone out there willing to fiddle in a simplified scenario (one AP + 2-4 stations + cake on the the flent rtt_fair test) to show if even this simple change has benefit at high contention, or is daring enough to just deploy it, give it a shot.

shell('tc class add dev ' + interfaceA + ' parent ' + parentClassID + ' classid ' + str(minor) + ' htb rate '+ str(round(elemDownloadMin)) + 'mbit ceil '+ str(round(elemDownloadMax)) + 'mbit ' + ' burst 3028 cburst 3028 ' + ' prio 3')

PS As ghu is my witness I do not know what the differences really are vs a vs burst and cburst. Maybe cburst or burst alone is enough. Figuring out the right number to pack into an aggregate is hard, and perhaps on a good network, at short ranges, a much larger burst would show a demonstrable effect.

PPS Over here there's an 8 minute segment that explains the problems aggregation introduced into multi-station wifi multiplexing. We solved that by intelligently creating backpressure "at the line rate", and scheduling at the AP better, but if you are artificially limiting the bandwidth to the radios, accounting for aggregation in the shaper might help.

https://www.youtube.com/watch?v=Rb-UnHDw02o&t=1560

v1.0 install documentation

Hello

Noticed the following things in the install notes for v1.0 / Ubuntu 21.10
https://github.com/rchac/LibreQoS/wiki/LibreQoS-v1.0-Installation-&-Usage-Guide---Physical-Server-and-Ubuntu-21.10

Under Install LibreQoS and dependencies:

sudo apt-get install qemu-guest-agent

Maybe not needed unless you run Proxmox :)

Under Install and compile XDP-CPUMAP-TC

cd LibreQoS git submodule update --init cd /xdp-cpumap-tc/ git submodule update --init cd /src/ make

I had to go into the v1.0 subfolder to get the commands to run
cd LibreQoS cd v1.0 git submodule update --init cd /xdp-cpumap-tc/ git submodule update --init cd /src/ make

Under How to run LibreQoS, Running as a service

[Service] WorkingDirectory=/home/$USER/LibreQoS ExecStart=/usr/bin/python3 /home/$USER/LibreQoS/scheduled.py

I had to add the v1.0 subfolder to the path to get the daemon to start
[Service] WorkingDirectory=/home/$USER/LibreQoS/v1.0 ExecStart=/usr/bin/python3 /home/$USER/LibreQoS/v1.0/scheduled.py

ispConfig.py interfaces were swapped or documented incorrectly

Hi @rchac, I guess the documentation in the ispConfig.py related to the interfaceA and interfaceB is wrong.

interfaceA should be core, and interfaceB should be edge, or, you have to swap them in the LibreQoS.py code.

Please confirm. Thanks!

UISP Integration

Hi!

We chatted on Facebook about extending UISP Integration a bit. As promised, I threw together some code that works for our network. It's in Rust (since that's what I use 90% of the day, and my Python skills are really out-dated).

You can find the code here: https://github.com/thebracket/libre_qos_rs/tree/main/uisp_integration

It's similar to the existing integration, and I've tried to retain compatibility with 1.1 alpha.

The first time you run it, it walks the entire UISP tree and builds your network.json topology - with everything defaulting to 1Gbps speed. It knows about "other" devices, so those of us who use that to manage devices from other vendors are supported. It also doesn't care about device models; we have a whole bunch of older Ubiquiti equipment, and didn't want to type in a giant list of radio models!

Once the first run is done, you have network.json in the format shown in the repo. Shaper.csv is populated with customers. Two additional files appear: Sites.csv lists all the sites it found, and AccessPoints.csv lists all the APs it found. You can edit the speed limits listed in these files, and on the next run - the correct values will be placed in your topology.

The theory behind that decision is that topology changes, quite frequently in some cases. So it'd be good to have new stuff pulled in automatically, but we don't want to lose the speeds you've defined (there really is no good way to figure that out, since you might have load-balancing, ECMP, fail-overs, etc.)

Hope this is helpful!
Herbert

ecn stats would be interesting

Given your sample size, and probable proportion of osx/ios users, it should be around 2% of the drop stat.

LibreQoS & Layer3 (routed)

Hi Fam,

We've started investigating the possibility of running LibreQoS with routing rather than bridging.

So far, atleast in testing it "kind of works". For reasons that are unclear to us just yet it "sometimes" does not want to shape upload traffic (from user -> internet). It always shapes download (internet -> user).

Is there any obvious reason we've missed as to why routing would not work correctly all the time in the upload direction, but always works correctly in bridged mode?

Thanks fam!

Install InfluxDB v1.1

upon runing

wget https://dl.influxdata.com/influxdb/releases/influxdb2-2.1.1-xxx.deb

i get

--2022-03-07 14:47:55-- https://dl.influxdata.com/influxdb/releases/influxdb2-2.1.1-xxx.deb
Resolving dl.influxdata.com (dl.influxdata.com)... 52.84.45.49, 52.84.45.16, 52.84.45.36, ...
Connecting to dl.influxdata.com (dl.influxdata.com)|52.84.45.49|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2022-03-07 14:47:56 ERROR 404: Not Found.

Some other filtering ideas

https://github.com/heistp/qdisc-custom-classification

[v1.1] Missing python modules install for API

Need to do:
pip install flask_restful flask_httpauth waitress
to run API.

Please add this to wiki instruction (because I can not edit wiki).

Feature request Shape by network

I have noted in the Shaper.csv ipv4 is defined as single ip. ie the ip on the customer equipment.
Example if customer routers has 192.168.0.2 the the expected is 192.168.0.2
Could you incorporate a way of shaping by network like ipv6 (/64)?
like the ipv4 field would take in 192.168.0.0/30 covers 192.168.0.0-192.168.0.3

How to utilize multiple NIC

I have 2 pcie cards both cards have 2 sfp slots. How to utilize that ?
should i add all interface to bridge0 or create a new bridge and add other ports there.
Or it can be utilized seperately ?

Failover and fault tolerance

being able to do failover using a tech like VRPP would be good for fault tolerance.

Doing multi-machine load balancing is much harder.

Safeguard for running many simultaneus LibreQoS.py

LibreQoS needs safeguard for running many simultaneus LibreQoS.py this is important because many technicans can use CRM and add/change/delete QoS classes. We will do this in CRM integration lvl - but it's should be done here also.

I see this like:

1. LibreQoS.py --force-refresh-shapers (without parameters just prints the rules on stdoutput)
2. hardcode that check in code (and ommit this when --force-refresh-shapers is set),
3. API calls should also got some safeguard or queue,

I saw the enableActualShellCommands variable in ispConfig.py but when I set this I cannot pass all the things to XDP.

lacp over bonding interface

Hi
is it possible to run LibreQOS over bonding interface

config is:

bond0.0100 - uplink internet

bond.0xxx - ussr vlans

and user is connect over ppp connections

Lower CPU usage a bit

It's quite likely that on a decent sized network you'll have some unmapped IPs; the system handles these with the default queues - so that's ok. The xdp_cpumap setup defaults to logging every single one of them in the kernel debug stream, which is pretty inefficient.

Take a look at /sys/kernel/debug/tracing/trace_pipe:

cat /sys/kernel/debug/tracing/trace_pipe

You'll have to hit ctrl + c to stop it, since you're catting a pipe that doesn't have an end. If you have some unmapped IPs generating traffic, you'll see loads of the following:

          <idle>-0       [007] d.s3. 1997240.991310: bpf_trace_printk: (xdp) cant find default cpu_idx_lookup

          <idle>-0       [007] dNs3. 1997240.991348: bpf_trace_printk: (tc) Misconf: FAILED lookup IP:0x3c1fc340 ifindex_ingress:2 prio:0

The BPF program is writing every single one of them to the kernel debug stream. That's a fast operation, but unless you actually plan on reading this file it's pretty unnecessary (and you can figure out unmapped IPs pretty well with the latency handler).

You can not emit these messages:

1. Open src/tc_classify_kern.c in the xdp-cpumap project.
2. Find line 103, which will read #define DEBUG 1
3. Comment it out - so // #define DEBUG 1
4. Save and exit your editor
5. Run make to rebuild the BPF program
6. Restart your shaper to load the new program.

You'll no longer see a flood of debug messages in the kernel trace log, and CPU time (and more importantly cache) is no longer used up creating these messages and storing them in the kernel trace pipe.

(You can also edit Makefile and change CC := gcc to CC += gcc -O3 for a quick'n'dirty change to the original Makefile that enables maximum CPU optimization on the BPF program. You might get -Os (optimize for size) running faster - or have to use it if you extend the BPF program since they have a limit of 4096 bytes.)

Dynamic entries in Shaper.csv

Hi,
It would be great if we can add remove entries dynamically because our clients are not fixed and sync Shaper.csv with the LibreQos on the fly

Virtualbox Error

i'm trying to test LibreQoS in Virtualbox, i have tried changing interfaces and still getting this error.
anyone encountered this? tried the latest ubuntu server version and the preferred version

tested on virtualbox with images:
ubuntu-21.10-live-server-amd64
ubuntu-22.04-live-server-amd64

ubuntu@ubuntu:/LibreQoS/v1.1$ sudo python3 ./LibreQoS.py
tc filter delete dev enp0s8
tc filter delete dev enp0s8 root
Error: Parent Qdisc doesn't exists.
We have an error talking to the kernel
tc qdisc delete dev enp0s8 root
Error: Cannot delete qdisc with handle of zero.
tc qdisc delete dev enp0s8
Error: Failed to find qdisc with specified handle.
tc filter delete dev enp0s3
tc filter delete dev enp0s3 root
Error: Parent Qdisc doesn't exists.
We have an error talking to the kernel
tc qdisc delete dev enp0s3 root
Error: Cannot delete qdisc with handle of zero.
tc qdisc delete dev enp0s3
Error: Failed to find qdisc with specified handle.
This Network Interface Card has 1 queues avaialble.
./xdp-cpumap-tc/bin/xps_setup.sh -d enp0s8 --default --disable
./xdp-cpumap-tc/bin/xps_setup.sh: line 103: echo: write error: No such file or directory
./xdp-cpumap-tc/bin/xps_setup.sh -d enp0s3 --default --disable
./xdp-cpumap-tc/bin/xps_setup.sh: line 103: echo: write error: No such file or directory
./xdp-cpumap-tc/src/xdp_iphash_to_cpu --dev enp0s8 --lan
Traceback (most recent call last):
File "/home/ubuntu/LibreQoS/v1.1/./LibreQoS.py", line 253, in
refreshShapers()
File "/home/ubuntu/LibreQoS/v1.1/./LibreQoS.py", line 124, in refreshShapers
shell('./xdp-cpumap-tc/src/xdp_iphash_to_cpu --dev ' + interfaceA + ' --lan')
File "/home/ubuntu/LibreQoS/v1.1/./LibreQoS.py", line 24, in shell
proc = subprocess.Popen(commands, stdout=subprocess.PIPE)
File "/usr/lib/python3.9/subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: './xdp-cpumap-tc/src/xdp_iphash_to_cpu'
ubuntu@ubuntu:/LibreQoS/v1.1$

Installation break down

1. Install and compile XDP-CPUMAP-TC
   The v1.1 git submodule seems not to exist. https://github.com/rchac/LibreQoS/blob/main/.gitmodules
   any work around?

Install PPing + libtins v1.1

on changing Dir
cd build
then run

cmake ../ -DLIBTINS_BUILD_SHARED=0 -DLIBTINS_ENABLE_CXX11=1
-DLIBTINS_ENABLE_ACK_TRACKER=0 -DLIBTINS_ENABLE_WPA2=0
-DCMAKE_INSTALL_PREFIX=dirname $PWD

i get below errors of

-- Could NOT find Boost (missing: Boost_INCLUDE_DIR)
-- Disabling ACK tracking support
CMake Warning at CMakeLists.txt:184 (MESSAGE):
Disabling TCP stream custom data support as boost.any was not found

-- Could NOT find Boost (missing: Boost_INCLUDE_DIR regex)
CMake Warning at examples/CMakeLists.txt:27 (MESSAGE):
Disabling HTTP requests example since boost.regex was not found

-- googletest git submodule is absent. Run git submodule init && git submodule update to get it
-- Configuring done
-- Generating done
-- Build files have been written to: /home/manager/LibreQoS/v1.1/pping/libtins/build

I run
Run git submodule init && git submodule update to get it fixed googletest but the missing Boost_INCLUDE_DIR still at large

-- Could NOT find Boost (missing: Boost_INCLUDE_DIR)
-- Disabling ACK tracking support
CMake Warning at CMakeLists.txt:184 (MESSAGE):
Disabling TCP stream custom data support as boost.any was not found

-- Could NOT find Boost (missing: Boost_INCLUDE_DIR regex)
CMake Warning at examples/CMakeLists.txt:27 (MESSAGE):
Disabling HTTP requests example since boost.regex was not found

[question] Libre QoS - few questions before testing

1. Does it supports flat configuration configs with subscriber services parameters for example:
   { CUSTOMERIPS: 172.16.1.1, 172.16.1.2, DLRATE: 100M, UPRATE: 10M, DLCEIL: 200M, UPCEIL, 20M, DLBURST: 400M, DLBURSTTIME: 5sec,
   CUSTOMERIPS: 172.16.3.0/29, DLRATE: 100M, UPRATE: 10M, DLCEIL: 200M, UPCEIL, 20M, DLBURST: 400M, DLBURSTTIME: 5sec}

2. Can I change only one class?

3. Does Libre QoS got an API?

4. Does it supports HTB Offloading for filters (https://marc.info/?l=linux-netdev&m=160770071222418&w=2) ?

5. Does it supports counting per customer?

6. Is it possible to view class network usage in realtime?

7. Is NUMA [x] better for routers or not?

8. Is there any documentation?

LibreQoS.py - parameter for cleaning up all the queues

I see inside the code function clearPriorSettings (https://github.com/rchac/LibreQoS/blob/main/v1.1/LibreQoS.py#L96).

For debugging purposes - we need to clean all the queues sometimes.
It would be marvelous when we could run LibreQoS.py --clear-rules

I also check that
systemctl stop LibreQoS
dont clean tc rules (that is fine) and don't touch XDP

Install PPing + libtins v1.1 Guide

cd home/$USER/LibreQoS/v1.1/pping/ should be cd /home/$USER/LibreQoS/v1.1/pping/
after runing make
make

i get below error

cd /home/$USER/LibreQoS/v1.1/pping/
manager@libreqos:~/LibreQoS/v1.1/pping$ make
g++ -I/home/manager/src/libtins/include -std=c++14 -g -O3 -Wall -o pping pping.cpp -L/home/manager/src/libtins/lib -ltins -lpcap
pping.cpp:73:10: fatal error: tins/tins.h: No such file or directory
73 | #include "tins/tins.h"
| ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:9: pping] Error 1

tried to locate tins/tins.h and mine is here /home/manager/LibreQoS/v1.1/pping/libtins/include/

Where should i correct the dir path?

Reloading All QoS rules - don't drop packets for customers when reloading

It rips and replaces all classes at each run. For ~500 hosts thats usually just a few seconds. this is done because it gets very convoluted to track class ids when changing them on the fly.

0 Packet lost when commiting changes ?

I tested right now. On a network of 350 subs, I saw just 1 single packet lost during the refresh. I tested at an interval of 100ms. So not enough packet loss to cause perceptible issues on most networks.

Single packet lost per customer means many packets lost for the wole router :-)

In our Linux router we now we use tc batch means:

  -b, -b filename, -batch, -batch filename
              read commands from provided file or standard input and
              invoke them.  First failure will cause termination of tc.

We generate rules to files and then just load the file like

/sbin/tc -f -b /etc/router/qos/hfsc/down_class_current
/sbin/tc -f -b /etc/router/qos/hfsc/down_qdisc_current
/sbin/tc -f -b /etc/router/qos/hfsc/down_filter_nets_current
/sbin/tc -f -b /etc/router/qos/hfsc/down_filter_current

This is much more efficient. This is also useful as a startup rules for VM.

Maybe this performance change could be also implemented in LibreQoS?

A thought

Hi LibreQoS fam!

I've been pondering on the very high CPU loading we had at the very beginning of our initial test of LibreQoS and wondered if there might be improvements to be had in the default config.

In order for us to test (and i stress, test) shaping by Site or AP we had to route all of our traffic through the box; with only a handful of AP's and clients defined in the Shaper.CSV. During this phase our CPU loading was as high as 70% per core across all cores (once we got there). Once all clients were defined in the shaper.csv and had parent sites, this loading has dropped to a maximum of 30%. In both cases we were observing a touch over 11gig/sec running through the box; so if there was heavy shaping going on, it wasn't apparent.

I'm not very clear on the underlying cause; i thought initially its because XDP wouldn't touch those clients who were undefined, causing their traffic to touch the linux kernel, but idk - it's not my area of expertise. In any event given the fix for us was to define every user in shaper.csv; I wonder if in the configuration it would be useful to have a "testing" section (which can be commented out for production) or similar that gets the admin to specify the subnet's they're going to route through the box, an upper limit of what a client should be able to achieve in terms of UL/DL (this is per IP) and then create a queue for each IP within that subnet?

Or there may be a much simpler / sexier fix? Look forward to your thoughts!

XDP & HT Cores

This is more of a question than a issue.

We've observed that XDP doesn't seem to map queues to hyperthreaded cores. In Ubuntu they appear as physical CPU's but it just doesn't want to touch them. The CPU is a 18 core / 38 thread Intel Xeon Gold; only the first 18 cores seem to get used for this process with the balance remaining relatively idle.

Is this expected behavior or a potential bug?

smart queue management

I do not have an issue with y'all calling this "libreqos". It's a well known term in this field.

However, as it is actually a "smart queue management" system, also putting in a link to that might aid comprehension.

LibreQoS is a smart queue management (SQM) system[https://www.bufferbloat.net/projects/cerowrt/wiki/Smart_Queue_Management/] that allows ISPs to intelligently apply bandwidth rate limiting ...

PS (congrats on the ebpf support and cracking the 2gbps barrier!!!!)

Alternative Network Typologies

Hi,

I haven't deployed LibreQoS yet, but am exploring it w/ great interest after Dave Taht alerted me to it. I operate a WISP and after reviewing the docs & code suspect that our network topology might be outside of what LibreQoS can currently support. Granted I'm new to this so this so it may all be related to being a fq/LIbreQoS newbie.

Within our network, a given site may have one or more roles. (An upstream back-haul connection is a given for all sites.)

1. P2MP Broadcast
2. P2P relay to another site
3. Tenant service (think apartment building, multi-dwelling building or MDU)

We currently have multiple sites with all combinations of 1, 2, and 3.

Out data center provides services to all of our sites, and in our case only has P2P links (no P2MP).

Two things that are unclear to me from reading the LibreQoS docs and code.

A) Does LibreQoS support customers in an MDU? The current implementation seems to presume that all customers receive service via an AP, most likely a P2MP AP.
B) We provision customers in MDUs using PPPoE. While I can't imagine this makes a difference as long as we can specify the customer IP for shaping purposes.

Thanks

Running LibreQoS.py from other place like it's dir gives an error - small improvment needed

python3 /root/LibreQoS/v1.1/LibreQoS.py 
Traceback (most recent call last):
  File "/root/LibreQoS/v1.1/LibreQoS.py", line 243, in <module>
    refreshShapers()
  File "/root/LibreQoS/v1.1/LibreQoS.py", line 45, in refreshShapers
    with open('Shaper.csv') as csv_file:
FileNotFoundError: [Errno 2] No such file or directory: 'Shaper.csv'

root@rty-libreqos:~# ls -la /root/LibreQoS/v1.1/Shaper.csv
-rw-r--r-- 1 root root 564 Aug 31 12:07 /root/LibreQoS/v1.1/Shaper.csv

Script should:

• use config file declared in parameter --configShaper=Shaper.csv --configNetwork=network.json,
• if not is avaiable in parameter then use file from directory of the script,
• if no is avaiable print an error,

This would be small improvement because if we would like to run it's from crontab we have to use bash wrapper script.

Adding/Reloading single subscriber service parameters

tc qdisc/class/filter support action change - this give possibility to not reload whole ruleset - but only change selected user DL/UP parameters.

This function would reolace full reload function in most cases (generating 5k+ rules is always taking much time)

HTB: quantum of class 80015 is big. Consider r2q change.

When I run LibreQoS.py in dmesg I get an warning: HTB: quantum of class 80015 is big. Consider r2q change. is this important? (I don't use HTB since ~10 years)

v1.1 - doubled line

@rchac there is doubled line:

python3 -m pip install ipaddress schedule influxdb-client requests flask
sudo python3 -m pip install ipaddress schedule influxdb-client requests flask

This line:
sudo python3 -m pip install ipaddress schedule influxdb-client requests flask
is enough. Am I right?

AP Shaping

Hi, are you planning on adding shaping to AP side in future?

Install and compile XDP-CPUMAP-TC v1.1

cd /xdp-cpumap-tc/
-bash: cd: /xdp-cpumap-tc/: No such file or directory

the correct sytax

cd xdp-cpumap-tc/

same to cd /src/
correct cd src/

Unable to start

uname -a
Linux qos-box 5.13.0-40-generic #45-Ubuntu SMP Tue Mar 29 14:48:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Python script doesnt start, following error

/usr/bin/python3 LibreQoS.py

Traceback (most recent call last):
File "/home/uberadmin/LibreQoS/v1.0/LibreQoS.py", line 330, in
refreshShapers()
File "/home/uberadmin/LibreQoS/v1.0/LibreQoS.py", line 254, in refreshShapers
print(tabs + ' ', end='')
NameError: name 'tabs' is not defined

Subscriber - support two IP addresses (not in the same subnet)

Yup! just put 192.168.1.55/29 as the IPv4 in Shaper.csv for your customer.

The main thing is that LibreQoS not support getting different IPs from 192.168.1.2/24 and 192.168.2.2/24 and mark it as one flowid and limit into one queue :( Is it hard to implement this?

I suppose the best way to implement this would be to add a customer_ID field on Shaper.csv (in addition to the existing ID field). With that, if multiple entries in Shaper.csv share the same ID, they can be in the same queue.

This is the case for two main cases:

1. Subscriber have 2 publics IPs and one service,
2. Some LXC containers got two ip addresses from different VLANS and two gateways depending on conditions the routing is getting through one or second ip.

That makes sense. Let's work to add this functionality.

This is a blocker for us to use LibreQoS now for production.

v1.0 installation service start issue

i have done clean install for v1.0 all goes ok.
If i do one time run i get some missing
sudo python3 ./LibreQoS.py see attahced LibreQoS.py.txt not shapping happens

if i run as a service the service doesnt start exots with below error

manager@libreqos:/LibreQoS/v1.0$ sudo systemctl start LibreQoS.service
manager@libreqos:/LibreQoS/v1.0$ sudo systemctl status LibreQoS.service
× LibreQoS.service
Loaded: loaded (/etc/systemd/system/LibreQoS.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2022-03-04 19:25:06 UTC; 4min 43s ago
Process: 2864 ExecStart=/usr/bin/python3 /home/$USER/LibreQoS/v1.0/scheduled.py (code=exited, status=200/CHDIR)
Main PID: 2864 (code=exited, status=200/CHDIR)
CPU: 1ms

Mar 04 19:25:06 libreqos systemd[1]: LibreQoS.service: Scheduled restart job, restart counter is at 5.
Mar 04 19:25:06 libreqos systemd[1]: Stopped LibreQoS.service.
Mar 04 19:25:06 libreqos systemd[1]: LibreQoS.service: Start request repeated too quickly.
Mar 04 19:25:06 libreqos systemd[1]: LibreQoS.service: Failed with result 'exit-code'.
Mar 04 19:25:06 libreqos systemd[1]: Failed to start LibreQoS.service.

don't know shy i cant get this thing up and running well
LibreQoS.py.txt

VLANs in shaped traffic

Doing a bit of testing early this morning, I ran into an interesting one. I cut over from our current Preseem setup, which has tagged VLANs in the bridge traffic. (It's not ideal, but it's working around some Preseem issues with our proxmox setup; Preseem support and proxmox is not a great mix - long story, they logged in and broke everything a few times!)

This doesn't affect virtio interfaces. I had PCI pass-through to an actual Intel 10gb NIC. After banging my head against a wall for a bit, the solution turned out to be to add the following to your offloading configuration:

 ethtool -K (interface-out) rxvlan off
 ethtool -K (interface-in) rxvlan off

Replace (interface-out) and (interface-in) with the actual interface names, in my case enp1s0f0 and enp1s0f1 respectively.

Hope that helps!

FYI: BPF-based pping

Just wanted to note (since you added pping support) that we're working on a BPF-based version of pping. Being BPF-based it has lower overhead than the original libpcap-based implementation and it's explicitly targeting an always-on usecase for inline monitoring of latency.

https://github.com/xdp-project/bpf-examples/tree/master/pping

It's still a work in progress, but the current implementation is functional, and has an output format that is compatible with the original pping (and ppviz). There's also an opportunity to adapt the output format or operating mode to your needs; feedback welcome!

Cc @simosund as he's the one writing the code :)

Unable to get base installation working

Hey there, I have followed the instructions for a v1.0 installation. I've modified ispConfig.py with my relevant interfaces, but am receiving several errors.

Configuration: Ubuntu Server 21.10 running in a KVM/QEMU container. Both interfaces are of type virtio, but with no additional configuration in the hypervisor. This configuration is for testing, it will also be tested on a Proxmox and ESXi-based VM, as well as bare metal before going into hopeful production with any of those configurations.

Log output of running LibreQoS.py

Of note are:

• Errors in xps_setup.sh when trying to write to the relevant queue configuration file in /sys/class/net/[iface]/queues/[queue]/xps_cpus (Lines 11-12)
• Errors loading the BPF program into the kernel (Lines 13 and 55)
• Error configuring the mq qdisc (starts on line 102).

Manually trying to install the mq qdisc with tc qdisc replace dev enp1s0 root mq returns the same RTNETLINK answers: Operation not supported

Monitoring LibreQOS

Please advise if there is a way to monitor LibreQOS via a tool like LibreNMS or CheckMK? We have it nicely running inside our network and now I'd like to get our NOC to keep an eye on it...

[v1.1] LibreQoS.py: validation error

I've got flat design (only subscriber queues) but I would like to ommit tc single_lock bottleneck so I created 4 sites:

cat network.json

{
	"INTERDUO1":
	{
	  "downloadBandwidthMbps":5000,
  	  "uploadBandwidthMbps":5000,
	}
	"INTERDUO2":
	{
	  "downloadBandwidthMbps":5000,
  	  "uploadBandwidthMbps":5000,
	}
	"INTERDUO3":
	{
	  "downloadBandwidthMbps":5000,
  	  "uploadBandwidthMbps":5000,
	}
	"INTERDUO4":
	{
	  "downloadBandwidthMbps":5000,
  	  "uploadBandwidthMbps":5000,
	}
}

cat Shaper.csv

ID,AP,MAC,Hostname,IPv4,IPv6,Download Min,Upload Min,Download Max,Upload Max
,INTERDUO1,,Device 1,100.64.0.1,,25,5,155,20
,INTERDUO1,,Device 2,100.64.0.2,,25,5,105,18
,INTERDUO2,,Device 3,100.64.0.3,,25,5,105,18
,INTERDUO2,,Device 4,100.64.0.4,,25,5,105,18
,INTERDUO3,,Device 5,100.64.0.5,,25,5,105,18
,INTERDUO3,,Device 6,100.64.0.6,,25,5,90,15
,INTERDUO4,,Device 7,100.64.0.7,,25,5,155,20
,INTERDUO4,,Device 8,100.64.0.8,,25,5,105,18
,INTERDUO1,,Device 9,100.64.0.9,,25,5,105,18
,INTERDUO1,,Device 10,100.64.0.10,,25,5,105,18
,INTERDUO2,,Device 11,100.64.0.11,,25,5,105,18

python3 LibreQoS.py gives me

Traceback (most recent call last):
  File "/root/LibreQoS/v1.1/LibreQoS.py", line 243, in <module>
    refreshShapers()
  File "/root/LibreQoS/v1.1/LibreQoS.py", line 72, in refreshShapers
    network = json.loads(j.read())
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)

v1.1? IPv6?

rchac, I see your request over on the XDP page you've linked on getting IPv6 support there. Wondering if you had any luck? I'd consider pitching in a bit to make that happen. I'm running preseem now, which I'm happy enough with but I wouldn't mind spending that money elsewhere and I also like that you've got site/AP/sub shaping in place today and I could REALLY put that to work.