how can I parse tcpdump stream live?

Question

I have a debian installation, where the OEM has a bunch of processes I dont recognize running, and I want to figure out if any of these things are dialing home.

I ran sudo tcpdump | grep ^e <ssh_ip>

but the pipe keeps breaking. I am connected via ssh, and the grep ^e is to omit the ssh ip of my client. Is there a way to continuously monitor the network connection for outbound requests? I don't know that this thing dials home, or where specifically that is, I just suspect that it does, from other behaviour not related to this question.

broken pipe usualy means the connection breaks, but my ssh connection is fine, it seems to be something else todo with tcpdump

If the point is to filter out the ssh traffic Why not tcpdump 'not (host <ssh_ip> and port ssh)'? — Stéphane Chazelas, Dec 01 '22 at 16:21
To filter out an IP address with grep, the syntax would rather be grep -vFw 10.11.12.13. grep ^e assuming a shell where ^ is not treated specially greps for lines that start with the e character. If grepping tcpdump's output, you'll likely want to add the -l option to tcpdump for line buffering. — Stéphane Chazelas, Dec 01 '22 at 16:22
wireshark/tshark. SSL traffic cannot be decrypted anyways unless you wanna MITM it by installing/using/forcing your own certificate. — Artem S. Tashkinov, Dec 01 '22 at 16:59
You can also strace the processes to see what they're doing including what they're sending. See also How to trace networking activity of a command? — Stéphane Chazelas, Dec 01 '22 at 17:29
@StéphaneChazelas there are so many processes, strace seems like the nightmare option — j0h, Dec 01 '22 at 17:41
Im trying to recall, isnt there a parser in tcpdump? bfp or something... — j0h, Dec 01 '22 at 17:49
strace can get you one strace file per thread/process with -ffo prefix. See also Capture the X11 protocol's traffic for how to decode strace traces into pcap — Stéphane Chazelas, Dec 02 '22 at 07:56

score 2 · Accepted Answer · answered Dec 02 '22 at 08:42

In:

sudo tcpdump | grep ^e <ssh_ip>

First, that ^e has different meanings depending on the shell.

in the Bourne shell, ^ is an alias for | (^, originally rendered more like ↑ being the pipe operator in its predecessor, the Thompson shell), so you'd pipe grep to e command there.
in fish versions up to 3.2, ^ was to redirect stderr (short for 2>).
in rc and derivatives, ^ is a sort of concatenation operator so grep ^e would become grepe.
in zsh -o extendedglob, ^ is a negation glob operator, so ^e would expand to all the non-hidden files in the current directory other than the one called e.
in csh/tcsh/bash/zsh, ^ is only special at the start of the line and only when history expansion is not disabled. So not here.
in most other shells, ^e would be passed literally to grep.

So assuming one of those latter shells, that command would start sudo tcpdump and grep ^e <ssh_ip> concurrently with the output of one connected to the input of the other via a pipe.

The syntax of grep arguments is grep [<options>] <regular-expression> [<files>] ([...] indicating optional parts). If there's no file, grep greps its stdin¹. Options start with the - character.

So here ^e is the regular-expression, and <ssh_ip> is the file to look for lines matching that regexp into.

As a regexp, ^e matches the start of the line followed by e, so that reports lines starting with e.

Here, most likely the <ssh_ip> file doesn't exist so grep will return immediately with an error.

At that point, tcpdump's stdout, inherited from sudo's will become a broken pipe, so the first time tcpdump tries to output something, it will receive a SIGPIPE signal and die or if it ignores that signal, its write() will fail with a EPIPE error. Here, tcpdump managing to write an error message suggests it either ignores that signal and it reports its failing write(), or it has a handler on it and reports its reception before terminating.

If you wanted to filter out the lines containing an IP address from the output of tcpdump, you'd rather need grep -Fvw <ssh_ip> where -F, -v and -w here bundled together in a single argument and are respectively for

matching on Fixed strings rather than a regexp (for . to match a literal . rather than any single character as it does in regexps).
reverse the match: return the lines that don't match.
matching on words only, (to not find <ssh_ip> == 10.10.10.10 in 210.10.10.109 for instance).

By default, when tcpdump detects that its output doesn't go to a terminal it reverts to block buffering, so if you want to see the output as soon as it comes, you'd likely want to add the -l option to tcpdump to reenable line buffering.

But in any case, tcpdump can filter packets by itself. Here, to exclude the packets from/to a given IP address, you just need:

sudo tcpdump not host <ssh_ip>

Or to only filter out the ssh traffic from that IP address:

sudo tcpdump 'not (host <ssh_ip> and tcp port ssh)'

(assuming the ssh connection is on the default port 22).

Those filters (Berkeley Packet Filters, BPF) are compiled into a binary filter expression passed to the kernel (setsockopt(fd, SOL_SOCKET, SO_ATTACH_FILTER, filter) on Linux at least). The filtering is happening there, so it's actually less work for tcpdump and the system in general than if tcpdump asked for all traffic, decoded all of it and passed it to grep.

wireshark, and its command line interface tshark is a much more advanced traffic analysing application which you may want to consider here as well. They support two types of filter: the BPF one as above called capture filter (passed with -f) and a display filter (passed with -Y) further applied on the packets returned by the kernel using wireshark's own filter syntax and more advanced capabilities.

^{¹ or recursively into the current working directory if the -r/-R options are given in recent versions of the GNU implementation of grep.}

how can I parse tcpdump stream live?

1 Answers1