I have some arbitrary data as an output of a command (e.g. a fragment of a binary data put out using tail). I want to feed it to another command/program which allows only a file as an argument (e.g. rawtopng [filein] [fileout]), does not support dash (-) argument, and I cannot make a temporary file in between. How can I do it in bash?
Will something like rawtopng <(tail myfile) fileout work?
The process substitution rawtopng <(tail myfile) fileout creates a pipe (just like tail myfile | rawtopng - fileout), and passes a name for the pipe to the program. On Linux, rawtopng will see a name like /dev/fd/42, and if it queries the file type, it'll be told that it's a pipe. No write to the filesystem is required (this is an anonymous pipe, not a named pipe). You can get a glimpse of what's going on by running a command like ls -ld <(echo foo)
Will this work? It depends what the rawtopng command requires (which I don't know, I'm not familiar with that command). If all it does is read from the file, that's fine. If it needs to seek in the file (i.e. if it doesn't read its input linearly from start to end), or if it needs to know the size of the input before it starts working, that won't work. In such cases, you will need a temporary file.
If the command needs seekable input or input with a size that's known in advance, then you don't have a choice, you need to create a temporary file. If you want to avoid writing to disk, you can arrange to put the file on a tmpfs filesystem, if your Unix variants offers that (Linux does). You can use the mktemp command to create a temporary file:
tmp=$(TMPDIR=/tmp mktemp XXXXXXXXXXXX.raw)
rawtopng "$tmp" fileout
rm "$tmp"
In zsh, the process substitution variant rawtopng =(tail myfile) fileout creates a temporary file and remove it once the command has finished.
Another case where process substitution won't cut it is if the program insists on a file with a particular extension. In this case, creating a named pipe might help:
mkfifo /tmp/foo.raw
tail myfile >/tmp/foo.raw &
rawtopng /tmp/foo.raw fileout
rm /tmp/foo.raw
- as an argument. Will tail myfile | rawtopng /dev/stdin fileout work in such case too?
– niutech
Feb 01 '16 at 10:54
I think you're on the right track! <(tail myfile) will create an Anonymous Named Pipe, and is a type of Bash Process Substitution.
Normally, this will pass /dev/fd/XX as the "filename", which is a file descriptor interface to the running process.
From the examples in the linked documentation:
bash$ wc <(cat /usr/share/dict/linux.words)
483523 483523 4992010 /dev/fd/63
To read from a process, use <(command), and to write to a process, use >(command).
You can read more about Process Substitution on Wikipedia as well.
You can create a named pipe.
Do something like mkfifo input to create the pipe.
Start the program that you want to read the output of, and redirect the output to the pipe, aka myprog > input
Then run your other command with input as the file.
Note that pipes are one directional. The program will be able to read, but not to write.
