4

If file descriptors are specific to each process (i.e. two processes may use the same file descriptor id to refer to different open files) then how is it possible to share transfer file descriptors (e.g. for shared mmaps) over sockets etc?

Does it rely on the kernel being mapped to the same numerical address range under each process?

benjimin
  • 153

1 Answers1

8

When you share a file descriptor over a socket, the kernel mediates. You need to prepare data using the cmsg(3) macros, send it using sendmsg(2) and receive it using recvmsg(2). The kernel is involved in the latter two operations, and it handles the conversion from a file descriptor to whatever data it needs to transmit the file descriptor, and making the file descriptor available in the receiving process.

How can same fd in different processes point to the same file? provides useful background. The sending process sends a file descriptor which means something in relation to its (private) file descriptor table; the kernel knows what that maps to in the system-wide open file table, and creates a new entry as necessary in the receiving process’ file descriptor table.

Stephen Kitt
  • 434,908
  • Surprising as the sendmsg(2) manual makes no mention of this behaviour (although recvmsg does). – benjimin Mar 09 '18 at 01:22
  • Similar explanation here: https://keithp.com/blogs/fd-passing/ The infrastructure for different processes possessing shared file descriptors was necessary for POSIX forking; the mystic bit is Linux's intercept-and-translate mechanism for communicating them on other occasions. – benjimin Mar 09 '18 at 01:36
  • Is there any way to usefully transfer a file descriptor (or a physical memory address) between processes without kernel mediation? Also, is there any way to perform this mediation process without a socket connection? (I'm interested in thread-safe ways to conveniently communicate an anonymous shared mmap to a process pool in python.) – benjimin Mar 09 '18 at 01:41
  • Physical memory addresses aren’t accessible from userspace, so no. Same goes for file descriptors: the full meaning of a file descriptor is only available using information in the kernel, so you can’t transfer one without going through the kernel. To transfer a file descriptor without a socket connection, you can fork... – Stephen Kitt Mar 09 '18 at 15:34
  • I'm guessing the linux kernel does actually happen to be in the same reserved address space for every process (regardless that physical addresses are inaccessible), but as you say this is still not the mechanism for sharing file pointers, as the descriptors refer to resources indirectly (via kernel-managed per-process tables). – benjimin Mar 16 '20 at 01:41