1

The situation I'm in is where we have three separate interfaces on a particular machine. Each connected network only allows servers (by network-based routing/firewalling) to only offer particular services. Obviously, clients could be connecting from a wide variety of subnets and I'm relying on network firewalls for access controls on that.

The management interface allows ssh traffic, but in order for clients to be able to connect the default route must be set to the gateway on the management network. This resolves SSH but it breaks HTTP services which are only allowed over the Production interface. The third interface is a backup interface but all traffic for that one is on the same subnet.

The routing table looks something like this:

[root@xxxcpr2 ~]# ip route
10.19.1.0/24 dev eth0 proto kernel scope link src 10.19.1.10
10.18.29.0/24 dev eth1 proto kernel scope link src 10.18.29.25
192.168.5.0/24 dev eth2 proto kernel scope link src 192.168.5.35
default via 10.18.29.1 dev eth1

In the above, eth0 is the management interface, eth1 is the production interface, and eth2 is the backup interface.

Essentially, the condition I'm trying to get to is where the default gateway for any and all port 22 traffic is routed out the management interface but all other traffic outside a directly connected subnet should be routed out the production interface. Essentially, I'm 90% where I'm wanting to be, I just want to add an exception for SSH traffic.

I found this but I wasn't able to get it to work. Maybe it's my lack of understanding or some difference between RHEL and Ubuntu. Since the main table's default gateway was already set to go out the production interface, what I did was:

# echo 1 > /proc/sys/net/ipv4/ip_forward
# ip route add table 22 default via 10.19.1.1
# iptables -t mangle -A PREROUTING -p tcp --dport 22 -j MARK --set-mark 22
# ip rule add fwmark 22 table 22
# iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 10.18.29.25
# iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source 10.19.1.10

To my way of thinking (maybe someone can point out how this is wrong) marking the packet and creating a rule to use table 22 should work (not sure why the original post mentioned SNAT but I tried to copy it as best I could for completeness) but connections still time out unless I change the main table's default route.

Basically, at this point I need someone to point out what I'm not understanding or that I'm overlooking.

Community
  • 1
Bratchley
  • 16,824
  • 14
  • 67
  • 103
  • It is not clear to me whether you are wanting to accept incoming ssh connections to your machine or whether you are trying to handle outgoing ssh connections to other machines. The rules for both are different. – Richm Nov 24 '14 at 18:43
  • Also are you using any connection tracking in your rules or is it just what you have posted above? – Richm Nov 24 '14 at 18:44
  • Just what I have above. Nice management is the only network that allows for ssh I was wanting both incoming and outgoing to use that interface. – Bratchley Nov 24 '14 at 19:05

2 Answers2

0

For incoming ssh connections (without connection tracking)

iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 22

For outgoing ssh connection (without connection tracking)

iptables -t mangle -A OUTPUT -p tcp --sport 22 -j MARK --set-mark 22

You should be able to combine them both using --port instead of --dport and --sport

It would likely be better practice to use connection tracking but the above rules should get you going.

Richm
  • 3,872
  • Are the nat rules required? – Bratchley Nov 24 '14 at 19:31
  • I added these rules but the packet counts don't ever go up after trying to connect over SSH. I added them to the OUTPUT chain as a test and the packet count for those rules went up like it was matching, but the traffic still failed. Is there a way to get it to log what routing table it's trying to use for the traffic? – Bratchley Nov 24 '14 at 19:49
  • I would leave the nat rules for now. They should be correct. – Richm Nov 24 '14 at 21:29
  • I tested the above (simple) rules and observed expected port 22 traffic going out of an alternative route. It worked both with and without the SNAT rules. – Richm Nov 26 '14 at 19:04
0

A more complete set of firewall rules that is based on a similar setup that I have for incoming ssh traffic.

Do make sure that any existing iptables rules are flushed before testing.

iptables -A INPUT   -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT  -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp -m tcp -m multiport --ports 22  -m state --state NEW  -j ACCEPT
iptables -A OUTPUT -p tcp -m tcp -m multiport --ports 22  -m state --state NEW  -j ACCEPT

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
iptables -N MarkThem -t mangle
iptables -t mangle -A OUTPUT -p tcp -m tcp -m multiport --ports 22  -j MarkThem
iptables -t mangle -A PREROUTING -p tcp -m tcp -m multiport --ports 22  -j MarkThem
# May need to filter out local LAN connections here.
# But try without filtering first.
# iptables -t mangle -A MarkThem  -s 10.18.29.0/24   -j RETURN
# iptables -t mangle -A MarkThem  -s 10.19.1.0/24   -j RETURN
# iptables -t mangle -A MarkThem  -s 192.168.5.0/24   -j RETURN
iptables -t mangle -A MarkThem  -j MARK --set-mark 22
iptables -t mangle -A MarkThem  -j CONNMARK --save-mark

iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 10.18.29.25
iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source 10.19.1.10

Note I have explicitly set the rules to match both source or destination on port 22. You will likely want to tighten up the firewall rules once you have something working.

When testing outgoing ssh connections it might be worthwhile specifying a bind address with the -b option to see if it works for traffic that is sourced from your management address.

ssh -b 10.19.1.10 auser@a.server.example.com

Richm
  • 3,872
  • The packet count is going up on the MarkThem chain but the SSH session is still timing out. – Bratchley Nov 25 '14 at 16:43
  • Use tcpdump to look at what is really going out on which interfaces. That should give you a clue as to what is going wrong. tcpdump -n -i eth0 port 22 – Richm Nov 26 '14 at 10:17
  • Here's the output of that command, it shows the client packets coming in eth0 but with no response going out. Earlier I did something similar with iptables -j LOG and it showed the response packets going out eth1 I tried changing the interface numbers in your command to show the same but it didn't show anything in tcpdump for some reason. – Bratchley Nov 26 '14 at 14:50
  • Hang on. In the commands I posted above I used --set-mark 1. Your routing rules are expecting --set-mark 22. Try changing that. I'll edit the answer. – Richm Nov 26 '14 at 16:52
  • Don't worry, I corrected for that when I first read it. – Bratchley Nov 26 '14 at 17:16
  • I also missed out -m multiport to get --ports 22 working. Without that you need to specify two rules for --dport 22 and --sport 22. Once we have got things working we should be able to use use --dport 22 only. – Richm Nov 26 '14 at 17:59
  • I also tried it with --dport and --sport with the same result. Is there a kernel module that needs to be loaded for CONNMARK? – Bratchley Nov 26 '14 at 18:29
  • Yes xt_connmark. – Richm Nov 26 '14 at 18:35
  • I just tested don't have a suitable setup handy to do a full test but I did just test on a machine with two interfaces where one has a fake router on it. I saw the kernel making the right routing decisions for the port 22 traffic. To get that working I only need the routing tables/rules and the mangle table iptables rules. The --save-mark and --restore-mark rules were necessary to make things work. – Richm Nov 26 '14 at 19:02
  • I think it's probably the kernel module but it may be a bit before I can reset the machine and test again. – Bratchley Nov 26 '14 at 19:03
  • I did a modprobe on the xt_conntrack module as well as enable ip_forward via sysctl and the packet count is still going up (not on the OUTPUT rules, though) but the ssh session is still timing out. I've tested through VPN and from another server on the same subnet. – Bratchley Dec 01 '14 at 18:58