2

I tried web search but have not found answer by searching for Linux redirect output on the fly. As far as I understand, output to say stdout works by using file descriptor 1:

$ ls -l /proc/pid/fd/1
lrwx------ 1 user1 user1 64 Sep 27 13:37 1 -> /dev/pts/1

There are Unix commands that work with the proc filesystem, e.g. ls here, tee can write to files, cat can read. Why for e.g. Linux it was not implemented that users can change those links manually via ln -s in bash? (when I tried to I got error "No such file or directory"). Would it be difficult to implement? How was it decided what utility should work with proc - by programmers of that utility?

RalfFriedl
  • 8,981
Marisha
  • 343
  • 2
    You can redirect a fd inside a process by attaching to it with ptrace(2) (you can easily try that with gdb). You can also "steal" its open fds via unix domain sockets + SCM_RIGHTS. You can also create your own virtual fileystem in userland with fuse(4). Now you have all the needed pieces to demo something better than linux's proc(5) and explore ways to make it less broken. Good luck! –  Sep 27 '19 at 19:57
  • You might want to use reredirect or dupx as discussed here https://stackoverflow.com/questions/593724/redirect-stderr-stdout-of-a-process-after-its-been-started-using-command-lin – eckes Sep 28 '19 at 09:10
  • Why would you want to use ln to redirect them, when you can do it trivially with fdup2() syscalls? In this case, exec 1>/tmp/somefile will make 1 be a symlink to /tmp/somefile. There's already a well-supported mechanism that works on all POSIX-compliant systems; why would we want to add another one that's going to be incompatible with every other OS? – Charles Duffy Sep 28 '19 at 15:45
  • 1
    @CharlesDuffy you cannot use dup2(newfd, 1) or exec >foo to redirect the output of another process. –  Sep 28 '19 at 20:07
  • @eckes Oh if someone just told them that dup2(open("/dev/null", 0), 1) will a) make stdout unwritable, any subsequent write(2) to it will fail with EBADF, not succeed as when it's redirected via >/dev/null in the shell and b) leak a file descriptor. –  Sep 28 '19 at 21:14
  • @mosvy I think it works with fopen filehandles (At least that would explain why there are project which claim to work ,) – eckes Sep 28 '19 at 23:33
  • @eckes No it doesn't, it's just that nobody cares if printf() actually prints anything ;-) –  Sep 28 '19 at 23:39
  • Hm, I am not sure why use dup at all, can’t you just close the fd1 and open a new target file then. Isn’t that what most daemonize and redirect functions do, anyway? – eckes Sep 28 '19 at 23:51
  • @eckes no you can't, because if you close fd 1 or 0, the next successful open() will return 0 or 1 resulting in complete mayhem. And it's not what daemon(3) & co; they never close the standard fds, but redirect them to/from /dev/null. –  Sep 28 '19 at 23:55
  • I do not follow, isn’t this exactly what you want, replacing fd1/2 with a new target? – eckes Sep 28 '19 at 23:57
  • You want to do that in a controlled way, not by exploiting side-effects, and doing it properly with dup2() does just that (that's how shells implement redirections, etc). –  Sep 28 '19 at 23:59

6 Answers6

10

Why for e.g. Linux it was not implemented that users can change those links manually via ln -s in bash?

Because the proc filesystem just shows internal kernel state. In some cases you can actually influence the kernel state by writing stuff to specific files, but I can't think of a single example where you can move things around.

Would it be difficult to implement?

The kernel is open source, have a look and decide for yourself ("difficult" would be relative to your programming experience).

Also think about what should happen if the process you are modifying is in the middle of writing something to stdout. Also think about security implications.

Chances of getting such a modification into the upstream kernel would, in my estimation, be probably pretty low, however.

How was it decided what utility should work with proc - by programmers of that utility?

/proc is a virtual filesystem and as such implements the kernel functions required for a virtual filesystem. When various kernel programmers use /proc, they usually try to do it in the way that is the least effort to program.

dirkt
  • 32,309
6

On Linux, standard input/output file descriptors are created via kernel system calls, not through the /proc filesystem.

The symbolic links you are seeing in /proc are not the actual means being used to pipe data, but are rather a reflection of what the kernel is doing to give processes access to data. That's why you can't use ln -s to change them.

Are for your question about redirecting output on the fly, I don't know of a command that does that. But it's certainly possible. Here's an example of how such a program could be developed:

  1. First, start by implementing the functionality of cat. Hence, take standard input and dump it to standard output. This would be a non-interactive application.
  2. Next, change the program so that it's interactive. A simple example would be to transfer a small amount of the input and wait for the user to press ENTER. Then transfer a bit more input, and repeat. At this stage, the program is able to pause and make decisions, which is important for being able to redirect the output on the fly.
  3. Finally, modify the program so that's able to take commands which specify what the inputs and outputs should be. It could be a daemon process with an API, an NCURSES UI, etc.

The above is certainly easier said than done. I glossed over plenty of important details, but I think it's enough to show that redirecting output on the fly is at least feasible.

Emmanuel Rosa
  • 7,128
6

Instead of altering where stdin, etc. point, you could create a pseudo-terminal (PTY) and redirect input and output through that, so that the program doesn't have to know that anything has changed: in fact, there are several programs that do just that (often called terminal multiplexers.)

As you are likely aware, you can disconnect from a session on one terminal and reconnect to it somewhere else using a muxer like tmux, but you can also have it pipe either standard input or output as well at any time. For example, if I have a shell running inside a tmux pane and send the command pipe-pane 'cat >>~/test.txt' (taken from the man page) to tmux, standard output will now get appended to ~/test.txt without changing anything inside the session pane.

ErikF
  • 4,042
4

How was it decided what utility should work with proc - by programmers of that utility?

No, by the developers of procfs, the virtual filesystem that exports kernel info in the form of a filesystem, allowing normal POSIX system calls to operate on it.

ls and ln just make the same system calls they always do: In Unix everything is a file; that's the whole point, and why you can use ls -l on /proc/PID instead of needing a special tool like lsof or ps.

You can run anything you want on a /proc/PID/fd/* file, e.g. if a program has its config file open, you can read that file with less, edit it with emacs or vim, or create a compressed copy of it with gzip < /proc/.../3 > foo.gz

You might think this works just by following the symlink. But that's not actually true. The lstat on that file reports it as a symlink, and readlink shows you a link target. But that link target is just text made up by the kernel: it can be something like /foo/bar (deleted) for an FD that's still open on a file after it's been unlinked. But open() on that /proc path will still open the actual file. (Like most system calls, open follows symlinks internally. For regular symlinks on regular filesystems, the link target does get treated like a normal pathname.)

There's no way to link that inode back into the filesystem (mostly for security reasons; it's possible using linkat(2) on an FD opened with O_TMPFILE which never had a pathname in the first place. https://stackoverflow.com/questions/4171713/relinking-an-anonymous-unlinked-but-open-file)

For your case, the right question to ask is "how it was decided which system-calls should do what on things under /proc/PID?"

AFAIK, no system calls on /proc files can modify the process environment. i.e. the kernel information is exported read-only.

You'll have to use other hacks, like described in comments, to make system calls like dup2 inside the context of the process. e.g. using the ptrace API. e.g. by using GDB or other debuggers which use ptrace to do things inside the target process.

Peter Cordes
  • 6,466
1

The core misunderstanding here seems to be that changing a symbolic link while it is been written through would change anything in the manner expected.

It would not. Once the actual file (or device or pipe) is opened, all activities will pertain to the open file, ignoring anything that happens to eg a symbolic link pointing to it. Actually, this is not limited to symbolic links, but actually pertains equally to the "hard" links that make files have names at all: In unixoid systems, you can actually move or even delete(!) a file (NOT: Overwrite it!) that some process has opened, and the process will see nothing changed until it closes the file.

rackandboneman
  • 489
  • 2
  • 5
1

The symlink is just a helper to locate the actual file, which is actually a helper to locate the the actual inode. After that what you open is determined by the inode itself. In the case of /dev/pts/1 what you are actually opening is the character device with major device number 136 and minor 1.

From that point on the filenames, the presence of the files or the symlinks don't matter, in the same way that you can still access a file that was deleted as long as you had it opened before it was deleted.

V13
  • 4,749