Can someone explain the command: file -- *

Question

I ran the command file -- * and got this output:

-file04: data
-file05: data
-file06: data
-file07: ASCII text

So it basically prints filename and content type. But what do the parameters mean?

Answer 1

file -- *

Here would be code in a Unix shell language. Unix shell language specialising in looking up and starting commands in new processes, passing them arguments and environment variables and plumbing their standard input/output/error streams.

The language is rudimentary but has a few fancy features.

That line in an example of a simple command in the shell language grammar.

White space is used to delimit words. Here file, -- and *.

The first word, here file, is taken as the name of the command to run. The shell will look up a path to that command using the $PATH special variable. I could be something like /usr/bin/file.

The * word contains a shell wildcard character (*), so it triggers a special feature of shells that is called filename generation or globbing (POSIX calls it pathname expansion). The shell will replace * with the list of filenames that match that pattern. * is a pattern that matches any number of character. So the shell will expand it all the names of all the files in the current directory (with the exception of hidden ones), alphabetically sorted.

Then the shell will fork a child process, and in that child process will execute /usr/bin/file with file, --, and that list of filenames (so in your case -file04, -file05...) as arguments.

The file command is a separate command from sh. While sh's job is to parse command line, file's job is to try and identify the nature of file's contents.

Upon starting, like most commands, file will parse its arguments. The first one file, just tells it how it's called, how it should refer to itself like in error messages. The next arguments tell it what to do.

Like in most commands, arguments that start with - are taken as options. Here, the first argument is --. Technically, it's not an option, it's the end-of-option marker. It tells file that it should stop processing options from now on, and that the next arguments should not be taken as option even if they start with -.

Which is a good thing here, as the next argument, -file01, was actually not meant to be taken as an option, but a filename for file to tell the nature of its contents.

You need that -- when passing non-option arguments that start with - (+ can also be a problem with some commands) or may start with -/+ like when they're the result of some expansion, like when using cmd -- "$arg", cmd -- *, cmd -- "${args[@]}", cmd -- $(cmd2)...

Failure to do that can result in bugs and security vulnerabilities.

For arguments that are meant to be file paths, another approach is to prefix the ones that don't start with / with ./ (./-foo/bar and -foo/bar are paths to the same file, but only the latter could be taken as an option). Using that approach could be necessary for commands that don't support -- as an end-of-option marker, and can also work around other problems like - alone being interpreted as meaning stdin instead of the file called - even if passed after --, or awk's handling of arguments containing =.

Answer 2

Most GNU console utilities cannot work with files starting with a dash unless you pass -- to them, e.g.

$ cat -file1

produces

cat: invalid option -- 'f'
Try 'cat --help' for more information.

That's why in your case -- is required as it instructs the file utility to stop processing options and treat the rest of arguments as files.

I wonder if this peculiarity could be exploited somehow because it surely looks like it's a viable option. Send a person a number of files in e.g. tar archive and ask them to perform some action, e.g. command *. Files starting with a dash will be interpreted as options unless the user exercises due diligence and most don't. For instance I don't remember the last time I used -- because I don't have a single file starting with a dash.