Routing between two networks that have duplicate IP addresses

Question

On a linux box we have three network interfaces, they look like below

| CentOS 6 Server
| ---------> eth0 (DHCP (192.168.1.x) Default Gateway, connects to a wired internet, 
|----------> eth1 (IP : (10.165.11.139) GW to be used : (10.165.11.137), connects to a network A
|----------> eth2 (IP : (10.150.114.190) GW to be used: (10.150.114.191), connects to a network B

Problem here is that both network A and network B have nodes with same IP, example :

10.232.130.171
10.232.130.172
10.232.131.100

route-eth1 file looks like:

10.232.130.0/24 via 10.165.11.137
10.232.131.0/24 via 10.165.11.137

route-eth2 file looks like:

10.232.130.0/24 via 10.150.114.189
10.232.131.0/24 via 10.150.114.189

so pinging 10.232.130.171 will always route it thru eth1 and not eth2 tried with application which binds with interface (asterisk PBX), incoming connection from above IP work fine, but any response to it is sent via eth1, hence rejected.

Any pointers how to resolve this?

Your best option would be to fix things so that each physical network segment has a different IP subnet. Another option would be to bridge the two networks and run them as a single /23 subnet (10.232.130.0/23) instead of two /24s. — cas, Apr 12 '21 at 13:38
So for the example of 10.232.130.171, is that a single node reachable through two paths, is that two independant nodes not knowing each others having a symmetrical role (=> think LB), or two independant nodes not even having the same role (=> no idea on how one would choose to reach one rather than the other)? — A.B, Apr 12 '21 at 16:37
@cas, they are different networks and beyond my administration to make any changes in them — user1263746, Apr 12 '21 at 16:41
Then it's perhaps possible to handle traffic initiated by them (with a lot of tinkering, and probably not with CentOS6's stock kernel), but not to initiate traffic to them. — A.B, Apr 12 '21 at 16:41
one part of the tinkering requires conntrack zones, but it's available in kernel 2.6.34 not 2.6.32 — A.B, Apr 12 '21 at 16:46
routing between two networks that have duplicate IP addresses is almost as bad as having duplicate IPs on the one network. I think your only option is to explain to the admin(s) of the networks why packets can't be reliably routed between the two networks and recommend that they co-ordinate to fix the problem. — cas, Apr 12 '21 at 17:00
Is there a way to do a source based routing? Like if 10.150.114.190 wants to reach 10.232.130.171, use eth2? — user1263746, Apr 12 '21 at 17:17
You could use policy-based routing to arrange that packets from a process bound to 10.150.114.190 would be routed using the 10.150.114.191 route, and similarly for the other one. But that would require every process on the central host to specifically bind to one of those interfaces. Conntrack zones also sound like something that might be useful here. Or, you could have 10.165.11.143 and 10.150.114.191 just do NAT for the hosts behind them, so that no-one needs to really care that there's two copies of the 10.232.130.0/23 block. — ilkkachu, Apr 12 '21 at 18:03
Is there traffic involving systems in eth0's LAN or beyond? I thought there would. Yes it's possilble to do source routing, but any local application will require an option to bind its client connection. And in this case (no traffic involving eth0) no need for conntrack zones and conntrack's memory. — A.B, Apr 12 '21 at 18:03

score 2 · Answer 1 · answered Apr 12 '21 at 19:00

2

Since you mentioned incoming connections and in particular replies going the wrong way, it sounds like it might be enough to have packets sent with the source address 10.150.114.190 (eth2) use the routes that go via 10.150.114.191, and similarly for eth1. Policy-based (source) routing should be able to do that.

It should basically boil down to something like:

echo "201 net1" >> /etc/iproute2/rt_tables
echo "202 net2" >> /etc/iproute2/rt_tables
ip rule add from 10.165.11.139 lookup net1
ip route add 10.232.130.0/23 via 10.165.11.143 dev eth1 table net1
ip rule add from 10.150.114.190 lookup net2
ip route add 10.232.130.0/23 via 10.150.114.189 dev eth2 table net2

But I may have missed some gotcha here.

See also e.g.

Note that doing that means that any process on the central host that wants to initiate a connection to one of the "duplicated" networks has to specifically bind to the appropriate interface / IP address, and also that if you route traffic from the two 10.232.130.0/23 blocks to the external world, then this won't be enough.

The above is also pretty much untested on my part. Caveat emptor. I'm only posting this is an answer so that it can be downvoted.

Honestly, if I had the choice, I would consider bribing the admins of the two networks to renumber one or both of them.

answered Apr 12 '21 at 19:00

ilkkachu

138,973

The the far end of the networks is actually a telco! So they wont change one bit. Testing out what you suggested – user1263746 Apr 12 '21 at 19:09
@user1263746 I think cases like this are exactly the reason IPv6 Unique Local Addresses (similar to the IPv4 private-use networks like 10/8 here) require random allocation – ilkkachu Apr 12 '21 at 19:13
ip route show table net2
10.232.130.0/23 via 10.150.114.189 dev eth3

ping -I eth3 10.232.130.170

PING 10.232.130.170 (10.232.130.170) from 10.150.114.190 eth3: 56(84) bytes of data. From 10.150.114.190 icmp_seq=2 Destination Host Unreachable From 10.150.114.190 icmp_seq=3 Destination Host Unreachable
– user1263746 Apr 12 '21 at 20:20
@user1263746, hmm, now that I look, your post says 10.165.11.143 as the gw for the eth1 in the first listing, but then 10.165.11.137 just after that. Similarly two IPs for eth2. And I mixed them up when copypasting, so check if the the IPs are right? – ilkkachu Apr 12 '21 at 20:33
I made those corrections, cross checked multiple times, but ping wont go past the gateway... – user1263746 Apr 12 '21 at 20:37
1

As the routing tables net1 and net2 are evaluated before the main routing table, I think those tables should also include for proper resolution as a minimum respectively: table net1 10.165.11.143 dev eth1 and table net2 10.150.114.189 dev eth2 . Of course putting the LAN instead of the single /32 (ie: duplicating the entry from the main table) would work too. – A.B Apr 12 '21 at 21:01
@A.B I didnt get you, what's the actionable here? Sorry I am reaching beyond what I understand – user1263746 Apr 12 '21 at 21:10
Of course since OP gives different gateways in different parts of the question for lan B (once 10.150.114.191 and once 10.150.114.189), it depends on the accuracy of OP's information. – A.B Apr 12 '21 at 21:10
@user1263746 fixed and rewrote my previous comment: ip route not route. translation: in addition to the answer, and once you're sure about the correct gateway for lanB (.189 or .191?), you also type: ip route add table net1 10.165.11.143 dev eth1 ; ip route add table net2 10.150.114.189 dev eth2. – A.B Apr 12 '21 at 21:20
@A.B Thanks but the the after the corrections and the command you suggested, the behavior stays the same. No nodes reachable beyond the gateway – user1263746 Apr 12 '21 at 21:27
@user1263746 so is it .191 instead? You never told – A.B Apr 12 '21 at 21:29

Routing between two networks that have duplicate IP addresses

1 Answers1

ping -I eth3 10.232.130.170