June 15, 2024

libreqos / cake

Cake (and earlier fq_codel) directs traffic on internet connections so that it works much better by reducing congestion by a lot. It’s really cool and pretty much just works. libreqos puts a gui on it.

I’m building this in Debian Bullseye, though it will certainly work on other things. This is a riff based on this. Get console and figure out which interfaces you want to use, don’t get this wrong or nothing will work!

The first thing I’ll do is get rid of the stupid newer names of interfaces like enp2s0, etc. by editing grub and making this line look like:

vi /etc/default/grub
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0"

then update grub and reboot:

apt install bridge-utils

When you reboot, you should now have names like eth0, eth1, etc. which will be tons easier to not screw up in this config.

libreqos cake

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:e0:67:10:91:88 brd ff:ff:ff:ff:ff:ff
    altname enp1s0
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:e0:67:10:91:89 brd ff:ff:ff:ff:ff:ff
    altname enp2s0
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:e0:67:10:91:8a brd ff:ff:ff:ff:ff:ff
    altname enp3s0
    inet brd scope global eth2
       valid_lft forever preferred_lft forever
    inet6 fe80::2e0:67ff:fe10:918a/64 scope link 
       valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:e0:67:10:91:8b brd ff:ff:ff:ff:ff:ff
    altname enp4s0

This shows my management interface is the third port from the right on my device, yours probably will be different.

make your /etc/network/interfaces look something like (note I assigned a bogus subnet for the bridge interface, which doesn’t do anything), change below to reflect your actual network, there’s 100% chance it won’t be like mine):

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo br0
iface lo inet loopback
# The primary network interface
allow-hotplug eth2
iface eth2 inet static
	# dns-* options are implemented by the resolvconf package, if installed
iface eth0 inet manual
	tc qdisc add dev eth0 root cake
iface eth1 inet manual
	tc qdisc add dev eth1 root cake
#Bridge setup
iface br0 inet static
	bridge_ports eth0 eth1
	#bridge_stp on

How we modify some properties of interfaces:

apt install ethtool

now create a couple scripts to modify interfaces:

vi /usr/src/offloadinterfaces.sh
  ethtool --offload eth1 gso off tso off lro off sg off gro off
  ethtool --offload eth2 gso off tso off lro off sg off gro off
vi /etc/systemd/system/offloadOff.service
chmod 664 /etc/systemd/system/offloadOff.service
chmod 744 /usr/src/offloadinterfaces.sh
systemctl daemon-reload
systemctl enable offloadOff.service
Created symlink /etc/systemd/system/default.target.wants/offloadOff.service → /etc/systemd/system/offloadOff.service.

Now install libreqos:

apt install build-essential python3-pip clang gcc gcc-multilib llvm libelf-dev git
python3 -m pip install ipaddress schedule influxdb-client requests flask flask_restful flask_httpauth waitress psutil binpacking
cd /usr/src/
git clone https://github.com/rchac/LibreQoS.git
cd LibreQoS/v1.2
git submodule update --init
cd xdp-cpumap-tc/
git submodule update --init
cd src
make[1]: Entering directory '/usr/src/LibreQoS/v1.2/xdp-cpumap-tc/libbpf/src'
make[1]: pkg-config: No such file or directory
  MKDIR    staticobjs
  CC       staticobjs/bpf.o
  CC       staticobjs/btf.o
  CC       staticobjs/libbpf.o
  CC       staticobjs/libbpf_errno.o
  CC       staticobjs/netlink.o
  CC       staticobjs/nlattr.o
  CC       staticobjs/str_error.o
  CC       staticobjs/libbpf_probes.o
  CC       staticobjs/bpf_prog_linfo.o
  CC       staticobjs/btf_dump.o
cd ../../
git clone https://github.com/pollere/pping.git
cd pping
git clone https://github.com/mfontanini/libtins.git
cd libtins
sudo apt-get install libpcap-dev libssl-dev cmake
mkdir build
cd build
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
git submodule init && git submodule update
Scanning dependencies of target tins
[  1%] Building CXX object src/CMakeFiles/tins.dir/address_range.cpp.o
[  2%] Building CXX object src/CMakeFiles/tins.dir/arp.cpp.o
[  4%] Building CXX object src/CMakeFiles/tins.dir/bootp.cpp.o
[  5%] Building CXX object src/CMakeFiles/tins.dir/crypto.cpp.o
[  7%] Building CXX object src/CMakeFiles/tins.dir/detail/address_helpers.cpp.o
[  8%] Building CXX object src/CMakeFiles/tins.dir/detail/icmp_extension_helpers.cpp.o
[  9%] Building CXX object src/CMakeFiles/tins.dir/detail/pdu_helpers.cpp.o
make install
[100%] Built target tins
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/pkgconfig/libtins.pc
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/cmake/libtins/libtinsConfig.cmake
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/cmake/libtins/libtinsConfigVersion.cmake
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/cmake/libtins/libtinsTargets.cmake
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/cmake/libtins/libtinsTargets-relwithdebinfo.cmake
-- Installing: /usr/src/LibreQoS/v1.2/pping/libtins/lib/libtins.a
cd ../..
vi Makefile (change line 3 to)
  LIBTINS = /usr/src/LibreQoS/v1.2/pping/libtins
g++ -I/usr/src/LibreQoS/v1.2/pping/libtins/include -std=c++14 -g -O3 -Wall -o pping pping.cpp -L/usr/src/LibreQoS/v1.2/pping/libtins/lib -ltins -lpcap
pping.cpp: In function ‘tsInfo* getTStm(const string&)’:
pping.cpp:175:19: warning: catching polymorphic type ‘class std::out_of_range’ by value [-Wcatch-value=]
  175 |     } catch (std::out_of_range) {
      |                   ^~~~~~~~~~~~
apt install influxdb

to see if it’s working do:

watch -n .1 tc -s qdisc show dev eth0
Every 0.1s: tc -s qdisc show dev eth0                                  hostname: Sat Oct 29 16:26:51 2022
qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 341732 bytes 1578 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0