3

Question: How do I route traffic though tun1 for a specific user?

What I've tried so far: I followed:

1: iptables forward all traffic to interface

sudo iptables -t nat -A POSTROUTING -m owner --uid-owner user1 -j SNAT --to-source 192.168.1.1

Result: The traffic doesnt reach tun1, which I can see in wireshark and "host google.com" gives no result.

2:

sudo iptables -t nat -A POSTROUTING -m owner --uid-owner test --out-interface tun1

Result: The traffic goes through default route, i.e. reaches internet without going through tun1 (vpn).

More information:

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: enx0c5b8f279a64: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 0c:5b:8f:27:9a:64 brd ff:ff:ff:ff:ff:ff
    inet 192.168.8.100/24 brd 192.168.8.255 scope global dynamic noprefixroute enx0c5b8f279a64
       valid_lft 53043sec preferred_lft 53043sec
    inet6 fe80::5f5a:e5de:ae93:e80b/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
9: tun1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 500
    link/none 
    inet 10.0.0.1/24 scope global tun1
       valid_lft forever preferred_lft forever
    inet6 fe80::c626:a226:a66c:2b0/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

ip route list
default via 192.168.8.1 dev enx0c5b8f279a64 proto dhcp metric 100 
10.0.0.0/24 dev tun1 proto kernel scope link src 10.0.0.1 
169.254.0.0/16 dev enx0c5b8f279a64 scope link metric 1000 
192.168.8.0/24 dev enx0c5b8f279a64 proto kernel scope link src 192.168.8.100 metric 100 
linkdown
Paulo Tomé
  • 3,782
james
  • 133
  • 1
    Variants of this question get asked pretty often. One simple solution to the underlying problem is to not attempt to route it by user, but instead create a second network namespace, and run all applications of this user (or, if it fits your use case, only some applications of one user, or some applications of multiple users) in this namespace. – dirkt Jan 26 '20 at 12:14

1 Answers1

8

I fully agree with dirk's remark: using a network namespace gets routing easier, without special cases. But here's an answer I hope is different than usual, which requires Linux kernel >= 4.10.

About the failing attempt: POSTROUTING as the name implies, is done after the routing decision was done. This chain can never alter a route. What should be used for locally initiated traffic is mangle/OUTPUT (and on nftables the special type route hook output chain). It will then still require interactions with the routing stack by using marks and to correct corner cases, probably also by using CONNMARK rather than only MARK. It's still difficult to get it working fine.

Anyway, here's a method not relying on iptables but on the network routing stack's uidrange feature which appeared in 2016 with kernel 4.10:

Add support for per-UID routing. It allows the administrator to configure rules such as:
# ip rule add uidrange 100-200 lookup 123. This functionality has been in use by all Android devices since 5.0. It is primarily used to impose per-app routing policies (on Android, every app has its own UID) without having to resort to rerouting packets in iptables, which breaks getsockname() and MTU/MSS calculation, and generally disrupts end-to-end connectivity commit, commit, commit

As the routing stack will pick immediately the correct route and thus the matching source IP, there's no need to SNAT this IP. There's still a minor "unclean" part required: it requires to use loose reverse path filter on the tun device, because while the targeted user gets a different routing table than other users, return traffic coming back from this interface doesn't belong to any user, so will not use this routing table.

In this example user1's UID will be 1234, and the arbitrary chosen table 1001234 for the alternate routes.

Copy/alter relevant main routes to table 1001234 and relax routing. Those routes and settings will have to be re-added each time the tunnel is re-established because the routes and settings are lost when the interface disappears and/or gets down:

ip route add table 1001234 10.0.0.0/24 dev tun1 src 10.0.0.1
ip route add table 1001234 default dev tun1 #no need of a gateway on a layer 3 interface
sysctl -w net.ipv4.conf.tun1.rp_filter=2

This is needed only once and will then affect immediately the user:

ip rule add uidrange 1234-1234 lookup 1001234

Do not add any iptables nat rules. Adequate firewalling rules are of course still required. Note that if the user queries local services (eg: a local DNS caching daemon available on 127.0.0.1:53), the queries made by the daemon will not use the tunnel, since the daemon has a different uid.

A.B
  • 36,364
  • 2
  • 73
  • 118
  • Thank you for this great and well described answer. Thank you, very very much! //Jenso – james Feb 03 '20 at 09:11
  • Great! Thanks. However, I think you mean L2 when "no need of a gateway"? – GOST Aug 17 '20 at 14:03
  • @GOST an L2 tunnel (usually called tapX rather than tunX) would require a gateway. Packets routed through a classical L3 tunnel (wireguard might behave a bit differently, to be checked) don't require a gateway: all packets are sent to the other side of the interface (rather than to the gateway available on the interface). It's still valid to specify a gateway though, it can even avoid to specify the interface then (but it will be set and used in the route), just not needed. – A.B Aug 17 '20 at 18:06
  • Great. Thanks. Can I ask you to add an example for tap (L2) tunnel like that: ip route add default via 10.0.0.1 dev tapX metric 10 table 1001234 and place it to you brilliant main answer? – GOST Aug 17 '20 at 19:48
  • @GOST can't use one's own address as gateway with an L2 interface (though it would have worked with an L3 tunnel), it's the same as not using a gateway. For example if one then did a connection to 8.8.8.8 (from uid 1234) this would generate an ARP request from 10.0.0.1 to 8.8.8.8, which wouldn't get an answer (since 8.8.8.8 isn't part of 10.0.0.0/24 on the other side). This would require a question where the gateway is known. It's not known here. If you have a problem in mind, just post an other question on UL SE. – A.B Aug 17 '20 at 21:41
  • I do understand it. The point is that some of readers don't. It is really good to have a complete example of uid-based policy routing together with routing table example both for L2 and L3 modes. Anyway, the policy explaination itself is good enough. – GOST Aug 17 '20 at 21:55