1

Is there a way to reverse a file list via glob?

So I would get the same result as with:

ls -r *

I'm using this in a shell-script and shellcheck keeps complaining:

^--------^ SC2045: Iterating over ls output is fragile. Use globs.

nath
  • 5,694

4 Answers4

7

zsh

With the zsh shell, the sorting order of globs can be customised with the oC (order), OC (same as ^oC for reverse) and n (for numeric) glob qualifiers,

Quoting info zsh qualifier:

oC

specifies how the names of the files should be sorted. If C is n they are sorted by name; if it is L they are sorted depending on the size (length) of the files; if l they are sorted by the number of links; if a, m, or c they are sorted by the time of the last access, modification, or inode change respectively; if d, files in subdirectories appear before those in the current directory at each level of the search -- this is best combined with other criteria, for example 'odon' to sort on names for files within the same directory; if N, no sorting is performed. Note that a, m, and c compare the age against the current time, hence the first name in the list is the youngest file. Also note that the modifiers ^ and - are used, so '*(^-oL)' gives a list of all files sorted by file size in descending order, following any symbolic links. Unless oN is used, multiple order specifiers may occur to resolve ties.

The default sorting is n (by name) unless the Y glob qualifier is used, in which case it is N (unsorted).

oe and o+ are special cases; they are each followed by shell code, delimited as for the e glob qualifier and the + glob qualifier respectively (see above). The code is executed for each matched file with the parameter REPLY set to the name of the file on entry and globsort appended to zsh_eval_context. The code should modify the parameter REPLY in some fashion. On return, the value of the parameter is used instead of the file name as the string on which to sort. Unlike other sort operators, oe and o+ may be repeated, but note that the maximum number of sort operators of any kind that may appear in any glob expression is 12.

So here, to get the file list in reverse alphabetical order:

list=(*(NOn))

or

list=(*(N^on))

(here also using the N (for nullglob) qualifier so that if there's no matching file, the list becomes empty).

For the list sorted by modification time in reverse (like ls -rt):

list=(*(NOm))

With zsh, you can also sort the elements of an array with the o and O parameter expansion flags.

list=(*(N))
list_reversed=(${(Oa)list})

That is, $list sorted in reverse (because of the capital O) on array members.

bash

With recent versions of the bash shell and with GNU sort, you can get a list of file names sorted in reverse with:

readarray -td '' list < <(
   shopt -s nullglob
   set -- *
   (($# == 0)) || printf '%s\0' "$@" | sort -rz)

readarray -td '' array reads a list of NUL-delimited records into an array.

With the GNU implementation of ls, another approach is to do:

eval "list=($(ls --quoting-style=shell-always -r))"

where ls --quoting-style=shell-always uses single quotes to quote file names (and \' outside of single quotes to quote single quotes themselves).

That approach also works with yash (assuming all file names are valid text in the locale), ksh93, zsh and mksh, though for ksh93, make sure you declare the variable as an array beforehand (with typeset -a list) otherwise if ls gives no output, that would create $list as a compound variable instead of an array variable.

Beware ast-open's ls implementation also supports a --quoting-style=shell-always option, but it uses the $'...' form of quoting which is not safe to use in all locales.

POSIXly

POSIXly, in sh, to reverse the "$@" array, you can do:

eval "set -- $(awk 'BEGIN {for (i = ARGV[1]; i; i--) printf " \"${"i"}\""}' "$#")"

The idea being to have awk output something like set -- "${3}" "${2}" "${1}" when "$@" has 3 elements for instance

So to get that list of files in reverse order:

set -- *
eval "set -- $(awk 'BEGIN {for (i = ARGV[1]; i; i--) printf " \"${"i"}\""}' "$#")"
echo file list:
printf ' - %s\n' "$@"

Beware POSIX sh doesn't have any equivalent of the nullglob option (a zsh invention), zsh's (N) glob qualifier or ksh93's ~(N) glob operator. If there's no non-hidden file in the current directory, you'll end up with a list with one * element. A common way to work around it is to do:

set -- [*] *
case "$1$2" in
  ('[*]*') shift 2;;
  (*)      shift;;
esac

Where the [*] combined with * is a way to discriminate between a * that comes from no match (where [*] would also expand to [*]) and one that comes from a file called * literally (where [*] would expand to *).


In any case, if you're going to pass a list of file names to ls for it to sort it like in your ls -r * approach where the shell passes the expansion of * to it and you'll want to use the -d option and mark the end of options with --:

ls -rd -- *

But, that output will still not be post-processable reliably since the file names are delimited by newline characters and the newline character is as valid as any in a file's name.

3

Assuming that this question is about reversing the list resulting from a filename globbing pattern, and not about reimplementing the exact output of ls -r.

Get the result of the glob match into e.g. the positional parameters:

set -- *

Then reverse that list:

names=()
for name do
    shift
    names[$#]=$name
done

This creates an array called names that contains the matches of the * globbing pattern reversed.

The reversing is done by iterating over the positional parameters (the result of the match) in order, and for each entry, inserting it in the position it would have in the reverse list. $# is the number of positional parameters (matches of our globbing pattern) and shift removes one from this list, decrementing $# by one in each iteration, so we're inserting elements from the start of the list of positional parameters into the end of the names array.

Once you have the names array, print it:

printf '%s\n' "${names[@]}"

... or do whatever else you need to do with it.

This will not be the same as ls -r * as the ls command would list the contents of any directory matched by the * glob.

When the pattern does not match anything, bash will leave the pattern unexpanded. Use shopt -s nullglob to set the shell's nullglob option, which, when activated, will remove the pattern completely when it does not match any name. To match hidden names, additionally set the shell's dotglob option.

Kusalananda
  • 333,661
  • You wouldn't want to do with the bash shell in a directory with more that a few dozen files as array manipulation in bash is excruciatingly slow. Here, in a directory with about 17k files, and with bash5, it's still not finished after 3 minutes. – Stéphane Chazelas Jan 05 '20 at 08:42
  • @StéphaneChazelas That is true. The list reversing loop recreates the list in each iteration, probably leading to all sorts of interesting memory fragmentation issues (or whatever it is that slows it down). I'll make an alternative loop instead, using indexing. – Kusalananda Jan 05 '20 at 08:46
  • @StéphaneChazelas The new loop should be faster. I dropped the alternative using a named array instead of the list of positional parameters as the syntax would have been to wordy. Also, you can't "shift" a named array in bash without recreating it. – Kusalananda Jan 05 '20 at 08:56
  • For the record, in that 17k files directory, that previous approach took 5 minutes in bash, 33 seconds in zsh, 1:08 minute in ksh93, 1:17 in yash, still not finished after 8 minutes in mksh. – Stéphane Chazelas Jan 05 '20 at 08:59
  • @StéphaneChazelas Yeah, there's some sort of suboptimal memory management happening there. In any case, the new code in the answer should only take a fraction of a second (or possibly a single digit number of seconds) to run. – Kusalananda Jan 05 '20 at 09:14
  • That's much better now (2 seconds in bash) though still not as fast as my GNU or POSIX approaches (;-b) – Stéphane Chazelas Jan 05 '20 at 09:14
0

You don't say which shell you're using, I've never yet run across a shell that globs in anything but ascending alphanumeric order. So this is something you'll need to do with ls, sort, or something of the sort. See the shellcheck wiki for why iterating over the output of ls is fragile, and what to do about it. Basically, there are two main problems to watch out for: 1) when a directory is empty, echo * prints * by default. 2) If you have files with spaces in their names, that can confuse whatever's reading the input. For (1), if you're using bash, you'll want to set the nullglob option. For (2), you'll probably want to write something along the lines of

ls -1r | while read filename; do
    ...
done

(again, assuming that you're using sh, bash, or zsh.)

arensb
  • 399
0

Portable to most shells, avoid error with one file (using only seq and shell given printf, set and eval):

set -- *
[ $# -gt 1 ] && eval set -- "$(printf "\"\${%s}\" " $(seq "$#" -1 1))"
printf '%s '   "$@"

It is reasonable to set nullglob and failglob where available.