June 20, 2021

tc fq_codel bufferbloat on bridge

tc, short for traffic control, can do all kinds of things like hook and manage bufferbloat issues, set bandwidth limits and lots of other stuff.

Here’s the setup I use for controlling bufferbloat across a couple bridge interfaces. This will give you a transparent bridge that will “automagically” manage package contention using fq_codel. Later, we’ll add MAC-based traffic limits. This assumes a topology like:

Laptop A ——— — — bridge fq_codel machine B —— laptop C
Machine D ——— — —|

Laptop A:

straight gigE interface

On server do:

vi /etc/network/interfaces
  enp3s0 mgmt interface
	enp2s0 bridge interface 1
	enp1s0 bridge interface 2
	br0 bridge for 1 and 2
  # The loopback network interface 
  auto lo br0 
  iface lo inet loopback 
  # The primary network interface 
  allow-hotplug enp3s0 
  iface enp3s0 inet static 
  iface enp1s0 inet manual 
	tc qdisc add dev enp1s0 root fq_codel 
  iface enp2s0 inet manual 
	tc qdisc add dev enp2s0 root fq_codel 
  # Bridge setup 
  iface br0 inet static 
	bridge_ports enp1s0 enp2s0 
	gateway 192.168.3

note: I still have to run this command later, will troubleshoot at some point (unless you have suggestions to make it work):

tc qdisc add dev enp1s0 root fq_codel

To start, my pings from Machine A to Laptop C were around 0.75 msec, then I flooded the link from Machine A to Laptop C using:

dd if=/dev/urandom | ssh user@ dd of=/dev/null

Then my pings went up to around 170 msec. Once I enabled fq_codel on the bridge machine B, my pings dropped to around 10 msec.

To make it start on boot:

crontab -e

# m h dom mon dow command
@reboot /usr/src/start.fq_codel.sh

vi /usr/src/start.fq_codel.sh

/sbin/tc qdisc add dev enp1s0 root fq_codel
/sbin/tc qdisc add dev enp2s0 root fq_codel

chmod 755 /usr/src/start.fq_codel.sh

it seems to be working under load (so far) though load will increase in the next 24 hours so I’ll watch it, but here’s what I get:

tc -s qdisc show dev enp2s0
qdisc fq_codel 8002: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
Sent 4273930406 bytes 3059139 pkt (dropped 0, overlimits 0 requeues 1269)
backlog 0b 0p requeues 1269
maxpacket 54504 drop_overlimit 0 new_flow_count 401 ecn_mark 0
new_flows_len 0 old_flows_len 0