How to monitor the performance of "iptables" kernel module?

Question

I would like to know how much CPU / memory my current iptables rules consume.

I have tried looking in ps and htop, but even with kernel threads displayed and did not see anything related to iptables.

I am using the conntrack module with these module-specific settings: xt_recent.ip_pkt_list_tot=1 xt_recent.ip_list_tot=4096. I think 4096 is quite high. And then, in my iptables configuration, I am using two kinds of block lists: BLACKLIST and PORTSCAN.

-A INPUT  -i eth0 -p icmp             -j ACCEPT
-A INPUT  -i eth0 -s  1.2.3.4/32      -j ACCEPT
-A INPUT  -i eth0 -m  recent --rsource --name BLACKLIST --seconds 14400 --update -j DROP
-A INPUT  -i eth0 -p  tcp  -m tcp --dport 25 -j ACCEPT

-A INPUT  -i eth0 -m  recent   --rsource --name PORTSCAN --seconds 3600 --update -j DROP
-A INPUT -i eth0  -p  udp  -m udp --dport 5060 -j ACCEPT
-A INPUT -i eth0  -p  tcp  -m tcp --dport 5061 -j ACCEPT
-A INPUT -i eth0  -p  udp  -m udp --dport 5062:5100 -j ACCEPT

-A INPUT  -i eth0 -m  conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT  -i eth0 -m  recent  --rsource --name PORTSCAN --set -j DROP

-A INPUT  -i eth0 -j DROP
-A INPUT  -j DROP

I am experiencing network problems on the server, where I suspect my iptables rules could play a role. For instance:

• My ssh sessions are being dropped quite often.
• Ping reports 0.2% packet loss

• when I am connecting on allowed ports, ie 5060 it takes noticeably longer when PORTSCAN has many items, as compared when it is empty

• What would be the best way to troubleshoot this issue?

• is there some optimization I could do to my iptables rules?
• How can I see how much of my CPU is being consumed by iptables ?

Answer 1

Linux Kernel's Process:

Many Kernel's functions like Iptables are processed in the Kernel level as kworker tasks, they are visible on task managers like top. As mentioned on the comments, you can compute the CPU and memory usage by comparing the total ressource usage usage with and without loading the iptables rules. Note that ipset already consumes memory even if you do not use it in a rule.

Kworker is a placeholder process for kernel worker threads, which perform most of the actual processing for the kernel, especially in cases where there are interrupts, timers, I/O, etc. These typically correspond to the vast majority of any allocated "system" time to running processes. It is not something that can be safely removed from the system in any way, and is completely unrelated to the desktop applications (except if these programs make system calls, which may require the kernel to do something). Also kworker means a Linux kernel process doing "work" (processing system calls). You can have several of them in your process list: kworker/0:1 is the one on your first CPU core, kworker/1:1 the one on your second etc.. All Kernel's processes are started as children of kthreadd process on the Kernel space.

Parent process: The process ID of kthreadd is 2 and this kernel workers can be listed with:

pstree 2 -l -p 
# or
ps --ppid 2 -u 
# or
ps --ppid 2 -o pid,user,%mem,command,time,etime,cpu,pcpu,nice,pcpu,vsz 

That last one can be used with a bash + cron script to watch changes... alternatively for a direct timed analysis, perf can be used (apt-get install linux-tools-common linux-tools-3.11.0-15-generic)

# Record 10 seconds of backtraces on all your CPUs:
sudo perf record -g -a sleep 10
Analyse your recording:
sudo perf report

Navigate the call graph with ←, →, ↑, ↓ and Enter.

---

Links: 1, 2, 3, 4, 5, 6.