-2

On Ubuntu, I often run some server/daemon and some clients both locally. The server/daemon and clients can be arbitrary programs (emacs daemon and clients, Screen daemon and clients, server and clients someone wrote), and suppose you don't how they are named.

  • Is there some way to find the PID of the server/daemon, given only the PID of a client process?

  • Is there some way to find the PIDs of all the clients, given only the PID of the server/daemon?

If what I am asking is impossible, what extra minimum information do you need to achieve the goals for as general purpose as possible?

Thanks.

Tim
  • 101,790
  • 1
    If the applications are using sockets to communicate, you could see which sockets each PIDs are connected to and which PIDs are listening on the other end. netstat and lsof are your friends here. – multithr3at3d Dec 01 '18 at 01:54
  • Thanks. Can you give some example commands to achieve both goals? What are other ways to implement server clients besides using sockets to communicate? – Tim Dec 01 '18 at 01:55
  • @Tim this could be of interest -- but beware that both lsof and ss are pretty difficult, to use a mild word. –  Dec 01 '18 at 03:42
  • @mosvy Thanks. "difficult" in what sense? – Tim Dec 04 '18 at 00:22
  • In the sense that I find their user interface pretty hugged up. If you're able to use either of them without having to look up each time the options in the the manual and/or write a script to make sense of the output -- great for you ;-) –  Dec 04 '18 at 01:56

2 Answers2

3

Most forms of IPC (inter-process communication) can be traced with a few utilities. Sockets (both network and UNIX sockets) are very commonly used and can be traced using some common tools. Let's look at an example using netstat -ap:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      810/python3         
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      858/nginx: master process 
<snip>
tcp        0      0 127.0.0.1:46858         127.0.0.1:5000          ESTABLISHED 860/nginx: worker process
<snip> 
tcp        0      0 127.0.0.1:5000          127.0.0.1:46858         ESTABLISHED 810/python3

The two processes with PIDs 860 and 810 are communicating; 810 being the server in this case. We can see this by visually parsing the netstat output or grep for it.

Also, say we wanted to see what clients are talking with PID 810, we could do lsof -p 810:

COMMAND PID USER   FD      TYPE             DEVICE  SIZE/OFF    NODE NAME
<snip>
python3 810 user    8u     IPv4              35702       0t0     TCP 127.0.0.1:5000 (LISTEN)
python3 810 user   10u     IPv4            4682120       0t0     TCP 127.0.0.1:5000->127.0.0.1:46858 (ESTABLISHED)

Here we can identify the endpoint that is communicating with our process, but not the PID. To identify the other PID, we could do lsof -i :46858:

COMMAND PID  USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
python3 810  user   10u  IPv4 4682120      0t0  TCP localhost:5000->localhost:46858 (ESTABLISHED)
nginx   860 nginx   18u  IPv4 4681280      0t0  TCP localhost:46858->localhost:5000 (ESTABLISHED)

Further down in the netstat output is UNIX sockets:

Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name     Path
<snip>
unix  2      [ ACC ]     STREAM     LISTENING     21936    1/systemd            /run/dbus/system_bus_socket
<snip>
unix  3      [ ]         STREAM     CONNECTED     28918    648/dbus-daemon      /run/dbus/system_bus_socket

We can see that both of these processes are using the UNIX socket at /run/dbus/system_bus_socket. So if you knew one of the processes, looking at this, you should be able to determine the other end. lsof can be used again in this case, and can also be pointed at the socket file like lsof /run/dbus/system_bus_socket.

I realize this is a little convoluted and a little complicated, but I hope it helps. Note that other types of IPC that use some kind of file/handle (such as pipes) can be traced using lsof as well.

multithr3at3d
  • 522
  • Thanks. (1) "The two processes with PIDs 860 and 810 are communicating; 810 being the server in this case". How do you know which is the server which is client? – Tim Dec 03 '18 at 22:50
  • @Tim because you can see a separate entry for the server's socket in a listen state. – multithr3at3d Dec 04 '18 at 01:24
  • netstat will only print the name for a unix socket in the case where the socket is bound to that path/name, not when it's connected to it. For instance, if you start xclock it will connect to the X11 socket deducted from the DISPLAY envvar, but you won't be able to tell from the output of netstat -xp | grep xclock which socket it's connected to. –  Dec 04 '18 at 02:07
  • @mosvy Does a socket binding to a pathname means the relation between a socket and its socket file? "when it's connected to it", do you mean a socket connects to another socket? – Tim Dec 04 '18 at 03:07
  • @Tim Any socket may have two addresses it's in "relation" with -- the address to which it's bound (with bind(2), returned by getsockname(2)), and the address to which it's connected (with connect(2), returned by getpeername(2)). In the case of unix sockets a) the addresses are paths in the filesystem (or simple strings on linux which also supports "abstract" sockets) b) a connected socket may be anonymous, with no address it's bound to (the kernel doesn't bind it to an automatic address, as it does with tcp sockets). Most "client" unix sockets are like that. –  Dec 04 '18 at 03:28
  • @mosvy Thanks. Given the problem that netstat only prints the name for a unix socket in the case where the socket is bound to that path/name, not when it's connected to it, do you have a better way? – Tim Dec 04 '18 at 03:40
  • @Tim I've already pointed you to the linked question. I plan to add my own answer to that, when I get time. –  Dec 04 '18 at 03:42
  • I am wondering why netstat doesn't show Screen client but only Screen server process? – Tim Dec 05 '18 at 03:56
1

I wrote px for this purpose (among others).

px will (among other useful things) tell you which other processes yours is talking to.

Example output, scroll to the bottom for IPC tracing:

~ $ sudo px 49903
/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
  --enable-audio-service-sandbox
  --origin-trial-disabled-features=MeasureMemory

kernel(0)                                     root
  launchd(1)                                  root
--> Google Chrome(49903)                      johan
      Google Chrome Helper(49922)             johan
      Google Chrome Helper(49958)             johan
      Google Chrome Helper (GPU)(49920)       johan
      Google Chrome Helper (Renderer)(49935)  johan
      Google Chrome Helper (Renderer)(49936)  johan
      Google Chrome Helper (Renderer)(49943)  johan
      Google Chrome Helper (Renderer)(49950)  johan
      Google Chrome Helper (Renderer)(49951)  johan
      Google Chrome Helper (Renderer)(49957)  johan
      Google Chrome Helper (Renderer)(64466)  johan
      Google Chrome Helper (Renderer)(75275)  johan
      Google Chrome Helper (Renderer)(76225)  johan
      Google Chrome Helper (Renderer)(76650)  johan
      Google Chrome Helper (Renderer)(76668)  johan
      Google Chrome Helper (Renderer)(76712)  johan


7d21h ago Google Chrome was started, at 2020-09-04T19:15:03+02:00.
0.3% has been its average CPU usage since then, or 32m25s/7d21h


Other processes started close to Google Chrome(49903):
  Google Chrome/chrome_crashpad_handler(49912) was started just after Google Chrome(49903)
  AlertNotificationService(49924) was started 1.0s after Google Chrome(49903)
  Google Chrome Helper(49922) was started 1.0s after Google Chrome(49903)
  Google Chrome Helper (GPU)(49920) was started 1.0s after Google Chrome(49903)
  Google Chrome Helper (Renderer)(49935) was started 1.0s after Google Chrome(49903)
  Google Chrome Helper (Renderer)(49936) was started 1.0s after Google Chrome(49903)
  VTDecoderXPCService(49934) was started 1.0s after Google Chrome(49903)


Users logged in when Google Chrome(49903) started:
  johan


2020-09-12T16:28:30.587930: Now invoking lsof, this can take over a minute on a big system...
2020-09-12T16:28:30.901834: lsof done, proceeding.


Others sharing this process' working directory (/)
  Working directory too common, never mind.


File descriptors:
  stdin : [CHR] /dev/null
  stdout: [CHR] /dev/null
  stderr: [CHR] /dev/null


Network connections:
  [IPv4] : (LISTEN)


Inter Process Communication:
  mDNSResponder(291): [unix] ->0x2b8028c5de1ab5b
  mDNSResponder(291): [unix] ->0x2b8028c5de1c5eb


For a list of all open files, do "sudo lsof -p 49903", or "sudo watch lsof -p 49903" for a live view.
~ $
Johan Walles
  • 181