3
  1. What does this mean?
  2. How does lsof find such FDs? I.e. compared to normal FDs, which can be found easily like ls -l /proc/$PID/fd/$FD.
$ lsof -p $(pgrep pulseaudio) | head -n1
COMMAND     PID       USER   FD      TYPE             DEVICE  SIZE/OFF    NODE NAME

$ lsof -p $(pgrep pulseaudio) | grep DEL
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2334404 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2340448 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2335426 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2340018 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2340021 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2334322 /memfd:pulseaudio
pulseaudi 25911 alan-sysop  DEL       REG                0,5           2336421 /memfd:pulseaudio

man lsof only documents a DEL value for the TYPE column, not the FD column.

sourcejedi
  • 50,249

1 Answers1

3

lsof usually reports entries from the Linux /proc/<PID>/maps file with mem in the TYPE FD column. However, when lsof can't stat(2) a path in the process maps file and the maps file entry contains (deleted), indicating the file was deleted after it had been opened, lsof reports the file as DEL.

https://stackoverflow.com/a/37160579/1601027, as pointed out by @don_crissti.

lsof cannot show the size of these files, even when run as root. (My lsof is version 4.89).

However, if you have both a new enough kernel and root access, you can both see the maps in ls -l /proc/$PID/map_files/, and you can run stat --dereference on individual files to show their size. This could be used to inspect the resources used by "deleted" mapped files. In particular memfds, which never appear in the filesystem and are always considered as (deleted) files.

$ ls -l /proc/$(pgrep pulseaudio)/map_files | head
total 0
lr--------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 562004ac5000-562004ada000 -> /usr/bin/pulseaudio
lr--------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 562004cda000-562004cdb000 -> /usr/bin/pulseaudio
lr--------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 562004cdb000-562004cdc000 -> /usr/bin/pulseaudio
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7fab98000000-7fab9c000000 -> /memfd:pulseaudio (deleted)
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7fab9c000000-7faba0000000 -> /memfd:pulseaudio (deleted)
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7faba0000000-7faba4000000 -> /memfd:pulseaudio (deleted)
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7faba4000000-7faba8000000 -> /memfd:pulseaudio (deleted)
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7faba8000000-7fabac000000 -> /memfd:pulseaudio (deleted)
lrw-------. 1 alan-sysop alan-sysop 64 Mar 18 23:50 7fabac000000-7fabb0000000 -> /memfd:pulseaudio (deleted)

$ sudo stat --dereference /proc/$(pgrep pulseaudio)/map_files/7fab98000000-7fab9c000000
  File: /proc/25911/map_files/7fab98000000-7fab9c000000
  Size: 67108864    Blocks: 0          IO Block: 4096   regular file
Device: 5h/5d   Inode: 2399078     Links: 0
Access: (0777/-rwxrwxrwx)  Uid: ( 1000/alan-sysop)   Gid: ( 1000/alan-sysop)
Context: unconfined_u:object_r:user_tmp_t:s0
Access: 2018-03-18 23:47:48.714061694 +0000
Modify: 2018-03-18 23:47:48.713061683 +0000
Change: 2018-03-18 23:47:48.713061683 +0000
 Birth: -

E.g. at least it was possible to see that no individual memfd, at least held directly by an FD or by a memory mapping, was consuming gigabytes on its own. It would still be nice to have some better tooling or scripts around this though.

$ sudo du -aLh /proc/*/map_files/ /proc/*/fd/ | sort -h | tail
du: cannot access '/proc/self/fd/3': No such file or directory
du: cannot access '/proc/thread-self/fd/3': No such file or directory
108M    /proc/10397/map_files/7f1e141b4000-7f1e1ad84000
111M    /proc/14862/map_files/
112M    /proc/10397/map_files/
113M    /proc/18324/map_files/7efdda2fb000-7efddaafb000
121M    /proc/18324/map_files/7efdea2fb000-7efdeaafb000
129M    /proc/18324/map_files/7efdc82fb000-7efdc8afb000
129M    /proc/18324/map_files/7efdd42fb000-7efdd4afb000
129M    /proc/18324/map_files/7efde52fb000-7efde5afb000
221M    /proc/26350/map_files/
3.9G    /proc/18324/map_files/

$ ps -x -q 18324
  PID TTY      STAT   TIME COMMAND
18324 pts/1    S+     0:00 journalctl -b -f

$ ps -x -q 26350
  PID TTY      STAT   TIME COMMAND
26350 ?        Sl     4:35 /usr/lib64/firefox/firefox

$ sudo ls -l /proc/18324/map_files/7efde52fb000-7efde5afb000
lr--------. 1 root root 64 Mar 19 00:32 /proc/18324/map_files/7efde52fb000-7efde5afb000
-> /var/log/journal/f211872a957d411a9315fd911006ef03/user-1001@c3f024d4b01f4531b9b69e0876e42af8-00000000002e2acf-00055bbea4d9059d.journal
sourcejedi
  • 50,249