ipv4 forwarding breaks bridges and veths

Question

I've successfully gotten the following to work:

ip netns add quarantine
ip link add eth0-q type veth peer name veth-q
ip link add br0 type bridge
ip link set veth-q master br0
ip link set br0 up
ip link set veth-q up
ip link set eth0-q netns quarantine
ip netns exec quarantine ip link set lo up
ip netns exec quarantine ip link set eth0-q up
ip netns exec quarantine ip address add 192.168.66.5/24 dev eth0-q
ip netns exec quarantine dnsmasq --interface=eth0-q --dhcp-range=192.168.66.10,192.168.66.50,255.255.255.0
ip link set eno1 master br0

This allows me to run an instance of dnsmasq without interfering with network-manager, and lets a device connecting through my default ethernet interface (eno1) get an IP in 192.168.66.0/24

I then decided to grant internet access, I did so:

ip address add 192.168.66.1/24 dev br0
iptables -A FORWARD -i wlp58s0 -o br0 -m state  --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -o wlp58s0 -o br0 -j ACCEPT
iptables -A FORWARD -j LOG
iptables -t nat -A POSTROUTING -o wlp58s0 -j MASQUERADE
sysctl -w net.ipv4.ipforward=1
sysctl -p

where wlp58s0 is my WiFi interface connected to my home WiFi. I also had to kill the dnsmasq described previously and replace it with:

ip netns exec quarantine dnsmasq --interface=eth0-q \
  --dhcp-range=192.168.66.10,192.168.66.50,255.255.255.0 \
  --dhcp-option=3,192.168.66.1 --dhcp-option=6,8.8.8.8

This way the device connected via eno1 knows to find the gateway and ask DNS queries to the Google DNS server 8.8.8.8.

All of this works perfectly fine, and after rebooting my machine, all the configuration is gone as expected, and things work consistently.

However: in an earlier attempt, I took advice found on the internet to enable packet forwarding, and instead of using sysctl, I did:

echo 1 > /proc/sys/net/ipv4/ip_forward

This had granted internet access after I had already connected my device on eno1 where it already had an IP.

But: after rebooting my machine, that ip forwarding setting had become persistent. Moreover: writing a 0 where I had written a 1 was not persistent. Worse: the initial setup (no internet access, just hand out IPs) was broken, my device on eno1 could not get an IP anymore from the configuration I described in the beginning. I used wireshark: requests for an IP could be seen on br0 but were gone from veth-q, even more peculiar: only IPv6 traffic could be seen on veth-q, the ipv4 traffic was entirely gone. Manually disabling IP forwarding by writing a 0 to /proc/sys/net/ipv4/ip_forward did nothing to help. Eventually I reinstalled my Linux distribution (Ubuntu) and took care of never using that echo command ever again and do things with sysctl which causes no problems.

Why did this happen ? It was a very strange and peculiar behaviour, because everything else with my computer seemed to be working just fine: I could get internet access, everything seemed to be back to normal, but that one interaction between the bridge and veth had been corrupted.

Any light shed on this would be greatly appreciated !

Answer 1

So, contrary to my initial thoughts, writing a 1 to /proc/sys/net/ipv4/ip_forward was not the problem.

The problem seems to be related to docker.

After disabling docker, I observe more normal behaviour from my bridges and virtual ethernets.

I will write in the comments to this answer whatever I can find out (what more precisely from docker causes the issue), but, at least I can safely say that manually writing a 1 to /proc/sys/net/ipv4/ip_forward was not the problem, which I guess technically solves my question.