1

I know that one can define additional file descriptors for ad-hoc use. However, I see real use for a "stdmeta" file descriptor that would be supported by common CLI tools. This file descriptor would output lines that are not part of the data yet are not errors. How might one go about proposing such an addition to Unix or Linux?

One example usage would be to output the headers of commands such as ps:

$ ps
  PID TTY          TIME CMD
 6394 pts/15   00:00:00 bash
10294 pts/15   00:00:00 ps
10295 pts/15   00:00:00 bash

If one were to wish to sort, grep or otherwise filter that output, one would have to either lose the header or have it get mixed up in the data. This can be worked around by redirecting the first line to stderr, or a myriad of different creative solutions, but it is obvious that those are all clunky ad-hoc and difficult to remember workarounds for the fact that there is no dedicated stdmeta file descriptor.

Another example is the output of curl. Curl pipes metadata to stderr so that it can inform humans of the progress without adding complexity to applications to which might be piped its output. Again, this is the use of stderr as an substitute for the missing stdmeta.

Should I just write to the LKML and state my case? Since this would break compatibility with some Unix legacy apps, should I direct my case to a more Unixy body?

Community
  • 1
dotancohen
  • 15,864
  • 1
    Another good use for a 'meta' file descriptor: Progress Reports – dotancohen Dec 21 '16 at 11:05
  • This strikes me as reasonably answered—and not facts and expertise, not just opinion—by an answer explaining how stdin/stdout/stderr work, and how/by whom conventions/standards are made on Unix. – derobert Jan 13 '17 at 22:30
  • 2
    "How do I propose an additional standard file descriptor?" seems like a valid question; there is an awful lot of fluff in the question in that case. "Why was there not a stdmeta?" might be a valid historical question, but it's not really asking that either. There's too much going on in it to reopen (though POB was probably the wrong reason), but I don't agree with the deletion vote either. – Michael Homer Jan 15 '17 at 21:02

1 Answers1

1

Your proposal for capturing metadata on stdmeta appears to be simply so that you can discard it in favour of "real" data appearing on stdout in a consistent manner. (Not necessarily a bad requirement in a green field situation.)

On a practical level, programs are coded to handle stdout and stderr. If you were to create a new stdmeta that took part of the data usually sent to either of the existing descriptors you could easily break many things badly. Consider your proposal with ps -ef and legacy code that did something like ps -ef | sed 1d | .... If you no longer send the header via stdout but instead push it to stdmeta this code will behave incorrectly.

It seems to me that you would be better with a -q flag (for quiet output) available on all utilities. But even that is fraught with difficulties when you consider that just about every script that understood its utilities could use -q would have to handle older versions of those same utilities that did not.

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
  • Regarding the first paragraph, I've addressed with two examples that the idea is not to discard the data, but rather to let other applications that the output may be piped to to handled the output properly. ps prints metadata on stdout, curl prints metadata on stderr. – dotancohen Apr 24 '15 at 10:08
  • Regarding the rest of the answer, the legacy app situation could be handled with a flag for those commands that support stdmeta to only use it with the --stdmeta flag for the foreseeable future. – dotancohen Apr 24 '15 at 10:08