11

I was trying to look into some reference from O'Reilly about Unix and Bash about the meaning of */ but couldn't find any. We can echo */ and see all the directories.

It seems like it means all "directories" only, while * means "all files and directories", but for some reason, many users seem to not know it, and books don't mention it either.

Is there a definitive source that talks about what */ means and perhaps its variations?

nonopolarity
  • 3,069

4 Answers4

19

The POSIX specification is the definitive reference on how unix tools should behave. The section on pathname resolution explains the meaning of trailing slashes in file names.

A pathname that contains at least one non-<slash> character and that ends with one or more trailing <slash> characters shall not be resolved successfully unless the last pathname component before the trailing <slash> characters names an existing directory or a directory entry that is to be created for a directory immediately after the pathname is resolved.

In other words: foo/ requires foo to be an existing directory, or a directory that the program will create (so mkdir foo/ is permitted). If foo is some other type of file (regular file, name piped, etc.), then accessing it as foo/ must not work.

The sentence above is incomplete: foo/ is actually a valid way to reference a symbolic link to a directory. This is specified below:

If a symbolic link is encountered during pathname resolution, the behavior shall depend on whether the pathname component is at the end of the pathname and on the function being performed. If all of the following are true, then pathname resolution is complete:

1. This is the last pathname component of the pathname. 2. The pathname has no trailing . 3. The function is required to act on the symbolic link itself, or certain arguments direct that the function act on the symbolic link itself.

In all other cases, the system shall prefix the remaining pathname, if any, with the contents of the symbolic link (…).

In other words: if foo is a symbolic link, and the program is supposed to follow symbolic links, then foo/ is equivalent to the target of the link, which may be a directory. So if foo is a symbolic link to a directory, foo/ is a valid way to refer to this directory. But if foo is a symbolic to a regular file or other non-directory, then foo/ is not a valid way to refer to that file.

Functions such as open return the error ENOTDIR if given a pathname with a trailing slash that is an existing non-directory.

[ENOTDIR]
(…) O_CREAT and O_EXCL are not specified, the path argument contains at least one non-<slash> character and ends with one or more trailing <slash> characters, and the last pathname component names an existing file that is neither a directory nor a symbolic link to a directory (…)

The impact of a slash in */ is described implicitly in the section on pattern matching in sh. There is no special rule for the / in a pattern (other than the rule that it cannot appear in a bracket expression, which is not relevant here). Therefore a / in a pattern must match a / in a pathname. For example, the pattern */ matches foo/ but not foo. Therefore */ matches directories and symbolic link to directories, but not regular files, symbolic link to regular files, broken symbolic links, named pipes, etc.

Kusalananda
  • 333,661
  • but interestingly, if a directory is named foo, its name is foo but not foo/... so */ doesn't really match it "by name". Unless officially, the name is really foo/ but we can do mv foo bar just as a shorthand. – nonopolarity Mar 29 '20 at 04:59
  • 4
    I'd say foo is the official name, but foo/ is a valid path which refers to foo. Compare this to other patterns: */bar, for instance, matches a bar file located in any directory - it's not looking for a file named literally "/bar", it's looking for a path <anything> / bar which locates a valid file. The glob matching isn't matching on a path, not a name. – daboross Mar 29 '20 at 08:08
  • 6
    @nonopolarity This view is gibberish. In POSIX, file names, including directory names, must not contain a slash. – rexkogitans Mar 29 '20 at 08:41
  • 1
    About zsh specifically, see the discussion at https://www.zsh.org/mla/users/2020/msg00032.html though. You'd expect */ to expand to the same as *(-/M) (the files of type directory (/) after symlink resolution (-), Marked with /), but it doesn't in current versions of zsh as it doesn't include directories that are not searchable on systems other than Cygwin (expands to the same dirs as */.) – Stéphane Chazelas Mar 29 '20 at 09:36
  • actually, the * and ** matching by path makes a lot of sense... so that's why it can be recursive and why */ means "anything that can be a path" (or top of the path) – nonopolarity Mar 29 '20 at 13:41
  • FWIW, the glob(3) function of GNU standard C library (glibc) is broken wrt */. It will also return matching non-directories when the filesystem doesn't support the dirent->d_type optimization or when the GLOB_ALTDIRFUNC feature is used, and d_type is not set to the correct type. Fortunately, most mainstream Linux fs do support d_type, and the only critical program which makes use of GLOB_ALTDIRFUNC is GNU make, which I have fixed to pass the right d_type through. But if you're using some older version of GNU make, it will still blow in your face. –  Mar 29 '20 at 18:57
  • @nonopolarity "Path" and "directory" are not synonyms. "Path" is a name refering to an object in the file system, which consists of a base name preceded by zero or more containing directories. If your current working directory is /home/bob, then foo.txt and /home/bob/foo.txt are both paths to the same file (the former being a relative path, the latter an absolute path.) */ only matches directories in the current working directory. – chepner Mar 29 '20 at 22:34
  • If we look at it this way, doesn't it make sense? echo */*.js shows all possible ways such as foo/bar.js, so by the same principle, then echo */ shows all possible ways foo/ and Downloads/ – nonopolarity Mar 30 '20 at 06:40
  • I think the first part of this answer misses the point. When globbing, the shell doesn't yet know how the result will be used. echo */ and ls */ have the same globbing rules. And echo */ obviously doesn't care whether the globbed result names a directory or a file; it just prints a string. The relevant part seems to come from one of your earlier older answers. There's an implied . after a trailing slash; it's matched as if it were */. – MSalters Mar 30 '20 at 09:21
  • @MSalters As noted in that answer, the wording in SUSv4 is different from earlier POSIX versions and a trailing slash is no longer defined as equivalent to */. (even if the end result is often the same). – Gilles 'SO- stop being evil' Mar 30 '20 at 09:46
3

There are several hints that a */ should match only directories is bash manual (emphasis mine):

  1. *
    Matches any string, including the null string. When the globstar shell option is enabled, and * is used in a pathname expansion context, two adjacent *s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a /, two adjacent *s will match only directories and subdirectories.

    It is explicit in that a **/ will match only directories and subdirectories, which may seem to imply that a */ should too.

  2. globstar
    If set, the pattern ** used in a pathname expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a /, only directories and subdirectories match.

    A repetition of the hint above.

And in the zsh manual:

  1. -g globstring
    The globstring is expanded using filename globbing; it should be quoted to protect it from immediate expansion. The resulting filenames are taken as the possible completions. Use`*(/)' instead of`*/' for directories.

    Which explicitly states that a */ should mean directories in some other place.

But the only real confirmation, comes from ksh manual:

*
Matches any string, including the null string. When used for filename expansion, if the globstar option is on, two adjacent *'s by itself will match all files and zero or more directories and subdirectories. If followed by a / then only directories and subdirectories will match.

Which is explicitly clear about what a single * should do.

Given that POSIX was mostly based on the way ksh works, and that bash and zsh try to emulate the way ksh works, it is reasonable to expect this same feature will be repeated on all three manuals.

  • 1
    About zsh specifically, see the discussion at https://www.zsh.org/mla/users/2020/msg00032.html though. You'd expect */ to expand to the same as *(-/M) (the files of type directory (/) after symlink resolution (-), Marked with /), but it doesn't in current versions of zsh as it doesn't include directories that are not searchable on systems other than Cygwin (expands to the same dirs as */.) – Stéphane Chazelas Mar 29 '20 at 09:40
1

You can find some explanation in the bash manual, Pattern matching

3.5.8.1 Pattern Matching

Any character that appears in a pattern, other than the special pattern characters described below, matches itself. The NUL character may not occur in a pattern. A backslash escapes the following character; the escaping backslash is discarded when matching. The special pattern characters must be quoted if they are to be matched literally.

The special pattern characters have the following meanings:

*

Matches any string, including the null string. When the globstar shell option is enabled, and ‘*’ is used in a filename expansion context, two adjacent ‘*’s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a ‘/’, two adjacent ‘*’s will match only directories and subdirectories.

And in The linux documentation project you find an explanation on echo *:

Bash performs filename expansion on unquoted command-line arguments. The echo command demonstrates this.

bash$ echo * a.1 b.1 c.1 t2.sh test1.txt`

bash$ echo t* t2.sh test1.txt

bash$ echo t?.sh t2.sh```
0

The expansion of */ means nothing special to the shell. It is exactly the same as expanding *.txt will list all existing pathnames that end in .txt.

For any shell, a */ will expand to all accessible pathnames that end in a /.

The only issue then is to understand when a pathname could end in a /.

That is only possible when the last filename exists and is a directory.

Explicit confirmation (from man ksh):

*
... If followed by a / then only directories and subdirectories will match.