How to open process substituted file from php?

Question

Here's what I tried to do myself:

$ type 1.sh
#!/bin/bash -eu
php -r 'var_dump(file_get_contents($_SERVER["argv"][1]));' -- <(echo 1)
$ ./1.sh
PHP Warning:  file_get_contents(/dev/fd/63): failed to open stream: No such file or directory in Command line code on line 1

Warning: file_get_contents(/dev/fd/63): failed to open stream: No such file or directory in Command line code on line 1
bool(false)

I tested it on Debian 6 (php-5.4.14, bash-4.1.5) and Arch Linux (php-5.4.12, bash-4.2.42).

UPD

$ strace -f -e trace=file php -r 'var_dump(file_get_contents($_SERVER["argv"][1]));' -- <(echo 1)
...
open("/usr/lib/php5/20100525/mongo.so", O_RDONLY) = 3
lstat("/dev/fd/63", {st_mode=S_IFLNK|0500, st_size=64, ...}) = 0
readlink("/dev/fd/63", "pipe:[405116]"..., 4096) = 13
lstat("/dev/fd/pipe:[405116]", 0x7fff5ea44850) = -1 ENOENT (No such file or directory)
lstat("/dev/fd", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
readlink("/dev/fd", "/proc/self/fd"..., 4096) = 13
lstat("/proc/self/fd", {st_mode=S_IFDIR|0500, st_size=0, ...}) = 0
lstat("/proc/self", {st_mode=S_IFLNK|0777, st_size=64, ...}) = 0
readlink("/proc/self", "31536"..., 4096) = 5
lstat("/proc/31536", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
lstat("/proc", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/31536/fd/pipe:[405116]", O_RDONLY) = -1 ENOENT (No such file or directory)
PHP Warning:  file_get_contents(/dev/fd/63): failed to open stream: No such file or directory in Command line code on line 1

Warning: file_get_contents(/dev/fd/63): failed to open stream: No such file or directory in Command line code on line 1
bool(false)

$ strace -f -e trace=file php <(echo 12)
...
open("/usr/lib/php5/20100525/mongo.so", O_RDONLY) = 3
open("/dev/fd/63", O_RDONLY)            = 3
lstat("/dev/fd/63", {st_mode=S_IFLNK|0500, st_size=64, ...}) = 0
readlink("/dev/fd/63", "pipe:[413359]", 4096) = 13
lstat("/dev/fd/pipe:[413359]", 0x7fffa69c3c00) = -1 ENOENT (No such file or directory)
lstat("/dev/fd/63", {st_mode=S_IFLNK|0500, st_size=64, ...}) = 0
readlink("/dev/fd/63", "pipe:[413359]", 4096) = 13
lstat("/dev/fd/pipe:[413359]", 0x7fffa69c19b0) = -1 ENOENT (No such file or directory)
lstat("/dev/fd", {st_mode=S_IFLNK|0777, st_size=13, ...}) = 0
readlink("/dev/fd", "/proc/self/fd"..., 4096) = 13
lstat("/proc/self/fd", {st_mode=S_IFDIR|0500, st_size=0, ...}) = 0
lstat("/proc/self", {st_mode=S_IFLNK|0777, st_size=64, ...}) = 0
readlink("/proc/self", "32214"..., 4096) = 5
lstat("/proc/32214", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
lstat("/proc", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
2

I added output of strace to the question, please take a look. — x-yuri, May 03 '13 at 22:25
Process Substitution -- <(cmd) and >(cmd), See http://mywiki.wooledge.org/BashSheet#Expansions http://mywiki.wooledge.org/ProcessSubstitution http://mywiki.wooledge.org/BashFAQ/024 — Rany Albeg Wein, May 03 '13 at 22:31
@Rany Albeg Wein What exactly do you mean with your links? Are there any information explaining the issue? — x-yuri, May 04 '13 at 09:43

cuonglm · Answer 1 · 2014-04-15T04:24:35.320

4

The problem is you want php to read input from a file descriptor, but you force it to read like regular file.

First, try this:

$ echo <(ls)
/dev/fd/63

then you can process ls output by reading /dev/fd/63. The process substitution will return the file descriptor, which is use by other command to reading its output.

In your case, you use $_SERVER["argv"][1], meaning that php will interpreter like this:

file_get_contents(/dev/fd/63)

From php manual, you can see protorype of file_get_contents function:

string file_get_contents ( string $filename [, bool $use_include_path =
false [, resource $context [, int $offset = -1 [, int $maxlen ]]]] )

Ops, php will consider /dev/fd/63 as normal file here, but it's really a file descriptor.

To access file descriptor, you must use php://fd, php://fd/63 will access content of file descriptor 63:

$ php -r 'var_dump(file_get_contents("php://".substr($_SERVER["argv"][1],-5)));' -- <(echo test.txt)
string(9) "test.txt
"

You can see, now php can process content in /dev/fd/63. But our purpose is reading content of file, which is provide via process substitution (In my example, it's test.txt). I don't know much about php, so I add another file_get_contents:

$ php -r 'var_dump(file_get_contents(file_get_contents("php://".substr($_SERVER["argv"][1],-5))));' -- <(echo -n test.txt)
string(13) "Hello world!
"

I use echo -n to remove newline from echo ouput, otherwise, php will see the ouput "test.txt\n".

Note

For more information about accessing file descriptor in php, you can see here.

edited Apr 15 '14 at 04:24

answered Apr 15 '14 at 03:55

cuonglm

153,898

This clears things up a bit. But the important thing here is why can't php do as everybody does and just open the file? Like: cat <(echo test). Also, you're wrong about your wording. I don't want php to read input from a file descriptor, process substitution doesn't return file descriptors and so on. /dev/fd/* files are files of procfs (virtual filesystem), which correspond to file descriptors of the process. You can't open file descriptor. They are means by which we specify to the kernel, what open file we're talking about. – x-yuri Apr 15 '14 at 20:34
What do you mean "everybody"?cat <(echo test) meaning cat /dev/fd/63 and will print test because cat have done its job, like you do cat /path/to/regular_file. You can see more here - http://en.wikipedia.org/wiki/Process_substitution - to know why process substitution return a file descriptor. – cuonglm Apr 16 '14 at 01:38
Let's see: ruby -e 'p File.read ARGV[0]' <(echo test), perl -e 'open F, @ARGV[0]; print <F>' <(echo test), cat <(echo test). They all don't try to dereference the link and just output what they're asked. As opposed to php. Because that's what open system call does. Do you have any counterexamples? Also, your wikipedia article has no word descriptor in it. Process substitution returns a symbolic link to non-existent file, which corresponds to a file descriptor. – x-yuri Apr 16 '14 at 06:11
@x-yuri: You should reading the article carefully, esspecially section Limitations. And you should know ruby File.read, perl diamond operator <> is very different with php file_get_contents. – cuonglm Apr 16 '14 at 06:35
I'd read that article before you mentioned it. And I understand that the pipes are not seekable, and therefore not all the programs will be able to work with process substitution. The question is why exactly php can't work with process substitution, what prevents it from doing that? perl, ruby, bash and whatnot deal with it just fine. Do you know any other languages with similar problems? Also, I do understand that File.read and <> are different from file_get_contents, but not in the context of the discussion. Considering the topic, they just read a file. – x-yuri Apr 16 '14 at 11:02
No, file_get_contents can only read regular file, see the manual page for more details. – cuonglm Apr 16 '14 at 11:07
Also, it works in nodejs: node -e 'var fs = require("fs"); fs.readFile(process.argv[1], "utf8", function(err, data) { console.log(data); })' <(echo test) and c. And the manual says: A URL can be used as a filename with this function – x-yuri Apr 16 '14 at 11:17
URL is different with path. See: http://www.php.net/manual/en/wrappers.php.php for more details. – cuonglm Apr 16 '14 at 11:24
That doesn't explain anything. For one reason or another, php doesn't deal with paths like /dev/fd/*. Although it has full right to do so, the question is what prevents it from handling such files. As you may see, Hauke Laging shares my opinion. Anyway, let's see what the developers has to say :) – x-yuri Apr 16 '14 at 14:19
On a side note, it's better (safer) to use basename instead of substr in your code. – x-yuri Apr 16 '14 at 14:27
@x-yuri: Yeap, I only use for this example. Looking for responing from developer :) – cuonglm Apr 16 '14 at 14:30

score 2 · Answer 2 · answered May 03 '13 at 23:17

2

This is the problem:

readlink("/dev/fd/63", "pipe:[405116]"..., 4096) = 13
lstat("/dev/fd/pipe:[405116]", 0x7fff5ea44850) = -1 ENOENT

Without any good reason (IMHO) php tries to get the real name of the link target. Unfortunately the link target is not part of the file system thus the try to access that name fails and causes this error. The symlink can only be opened as such. I consider this a bug in php. You could use a FIFO instead:

mkfifo /my/fifo; output_cmd >/my/fifo & php -r ... /my/fifo

answered May 03 '13 at 23:17

Hauke Laging

90,279

But php, opening the file on its own, makes the same system calls and the file is opened successfully. See updated question. – x-yuri May 04 '13 at 09:38
@x-yuri There is neither a "file" nor "on its own". In both cases there is a lstat("/dev/fd/pipe:[405116]", 0x7fff5ea44850) = -1 ENOENT call. Unfortunately the second block ends one line too early. What does the open() call to the file look like? Maybe php can be tricked into opening the link path instead of the "physical" path. But that cannot be forced from the outside. – Hauke Laging May 04 '13 at 12:56
There is no more system calls for php <(echo 12) case. I only forgot to add output of the script itself. – x-yuri May 05 '13 at 13:51
@x-yuri The file is obviously not opened if there is no open() or openat(). The missing error message does not mean that the file was opened successfully. – Hauke Laging May 05 '13 at 14:02
I meant, there is only one call to open(), on the second line of the output (open("/dev/fd/63"...) . Not at the end. So, the file is opened. And the script is run, so the file is opened successfully. Just in case... do note who tries to open the file: php -r '...file_get_contents(...' -- <(echo 12) - the file is opened by the script. php <(echo 12) - the file is opened by php itself and contains the script. – x-yuri May 05 '13 at 15:23
@x-yuri: I think the file descriptor 63 is opened by bash, not php. – cuonglm Apr 15 '14 at 03:25
@Gnouc Indeed, you're right. – x-yuri Apr 15 '14 at 20:04

score 1 · Answer 3 · answered Dec 19 '18 at 01:45

This is an old question but I just figured out the answer so I thought I'd share. Maybe it will help someone.

You can use the php:// stream wrapper to open the file descriptor instead:

 $fd = $argv[1];
 $handle = fopen(str_replace('/dev/','php://',$fd));

So instead of opening the /dev/fd/[nn] file descriptor provided by the OS. You'll be opening php://fd/[nn] which will work. I'm not sure why opening the file descriptor fails on some systems but not others.

This was an old bug and was supposed to have been fixed.

score 0 · Answer 4 · answered May 26 '16 at 07:58

Here's what I ended up using...

php -r "var_dump(file_get_contents('php://stdin'));" < <(echo this is a win)

If you're trying to do other stuff with stdin this obviously won't work for you, and this is no different from

php -r "var_dump(stream_get_contents(STDIN));" < <(echo this is a win)

which leads me to wonder what it is you were actually trying to do, and why are you stuck on file_get_contents? :)

Reference - http://php.net/manual/en/wrappers.php.php

I was trying to embed php script into bash script, see my first question. — x-yuri, May 27 '16 at 19:03

How to open process substituted file from php?

4 Answers4

Linked