Splitting port/IP-bound traffic between interfaces

Question

Similar to this request (I think) I'd like to see traffic bound for a certain port number routed through one specific interface while traffic for a specific IP is routed through another. The server I'm trying to do this to has a fresh install of Ubuntu 14.04.2, so I've got a wide array of tools to accomplish the task. However, networking isn't something with which I've much experience, so I'm not sure where to start. =)

Thanks in advance!

EDIT:

Some details, because they matter.

me@server:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 54:9f:35:0a:6a:20 brd ff:ff:ff:ff:ff:ff
    inet 10.22.89.50/24 brd 10.22.89.255 scope global em1
       valid_lft forever preferred_lft forever
    inet6 fe80::569f:35ff:fe0a:6a20/64 scope link 
       valid_lft forever preferred_lft forever
(...)
6: p1p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0f:53:2d:4b:80 brd ff:ff:ff:ff:ff:ff
    inet 10.22.52.199/26 brd 10.22.52.192 scope global p1p1
       valid_lft forever preferred_lft forever
7: p1p2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0f:53:2d:4b:81 brd ff:ff:ff:ff:ff:ff
    inet 10.22.52.200/26 brd 10.22.52.192 scope global p1p2
       valid_lft forever preferred_lft forever

I'd like to route all traffic bound for 74.120.50.7 through p1p2 and all traffic bound for any IP on port 7047 through p1p1.

So in general, the sending side of a packet, on each hop, decides where to send it. Typically, that decision is based on destination IP address alone, but other attributes (source IP, source/destination port, etc.) are also possible. With the current almost no details in your question, I'm not even sure how it's different than the question you linked to—how does the answer there fall short? Unfortunately, "please tech me IP networking" is beyond the scope of what can reasonably be done in a [unix.se] answer... — derobert, May 07 '15 at 18:48
Agreed entirely, haha. The question I linked to didn't mention IP-based routing in the accepted answer because the OP didn't ask about it. However, I can at least say that my two interfaces have separate IP's, so perhaps the best answer is a regular static route in /etc/network/interfaces as well as port-based route like the one in the question I linked? — pdm, May 07 '15 at 18:52
Ok, I'm afk at the moment, but I'll wrote something up in a few hours if no one else has beaten me to it. Which everyone else is welcome to do, of course. — derobert, May 07 '15 at 19:08
I've added an answer, but the more I think about this—unless p1p1 and p1p2 are connected to different actual connections (e.g., different ISPs), it seems like there is probably a much better way to accomplish whatever you're trying to pull off. — derobert, May 07 '15 at 23:26

score 5 · Accepted Answer · answered May 07 '15 at 23:25

Ok: First, you have two interfaces with sequential IP addresses on the same subnet—if you're trying to spread load between them to get higher throughput (say, you want to turn two gigabit NICs into one 2 gigabit NIC), there are much better ways to do this! The general term would be "bonding". Search for "network bonding" or "NIC bonding" to find some stuff.

Presuming that's not what you're doing, the next thing to beware of is that the default under Linux is to consider IP addresses as bound to a host, not to a interface. There is a /proc entry to change this: /proc/sys/net/ipv4/conf/«INTERFACE»/arp_filter; set it to 1 to disable this behavior. You probably need to.

Finally, it's possible you just made up some sequential private IP addresses and used them in your question to redact your actual IPs, and the two interfaces aren't on the same subnet. In which case, ignore the above, but please in the future use different private subnets to avoid confusion.

Anyway, there are a couple of things that appear different in your situation vs. the question you linked to—the other stuff from those answers should apply.

If both these ports are on the same Ethernet network (e.g., plug into the same switch), then the switch(es) actually decides which port to send the traffic to. Unless you have expensive switches, they do this only based on destination MAC address. I'm sure someone makes a switch that can look at the TCP port number—but I bet it isn't cheap.
By default, the router sending data to your server is only going to care about the destination IP address. To get "all incoming traffic to 10.22.52.199 or 10.22.52.200 on port 7047 goes to 10.22.52.199" you're going to have to convince your router to do that... And that may not be easy.
Outgoing traffic to 74.120.50.7 should be fairly easy: ip route add 74.120.50.7/32 via «ROUTER» dev p1p2 src 10.22.52.200. That specifies the device to send it on and the source IP; I'm pretty sure that'll get it going out that interface. (/32 is a 255.255.255.255 netmask, i.e., only one address). You need to fill in the next-hop router for «ROUTER». (Probably called the "default gateway" somewhere).
Similarly, for the port number: in the extra routing table (from the linked question's answer) add in a dev p1p1 src 10.22.52.199. If the above works, this should too.

You can confirm traffic is flowing over the correct interfaces with, e.g., tcpdump -p -i INTERFACE

I will be trying this solution as soon as the server is on site. =) — pdm, May 08 '15 at 15:43
Of course a little bit of routing plus the tables answer from the other question worked out together to (apparently) create the solution I was looking for. So for that, thank you! But much more importantly, your answer taught me a healthy does of fundamentals that will allow me to make better decisions regarding networking in the future. Thank you! — pdm, May 15 '15 at 13:34

Splitting port/IP-bound traffic between interfaces

1 Answers1