How to select a sub-range of the files in the current directory in a script/on the command line, based on e.g. alphabetical order?

Question

In a file manager you can usually select a file, then hold Shift and select another file. Every file in between will be selected.

I want to do the bash / zsh equivalent of this.

I.e: I want to give 2 filenames and have every filename in between (in alphabetical order - the way ls outputs them).

I am aware of ? {} * and other wildcard options, however I want to be able to use this on files with highly scrambled names.

E.g.: given the files

$ ls
aoeitoae.txt
oaeistn.txt
oaie.txt
paeoai.txt
sotaoe.txt

I would like to give a command like rm aoeitoae.txt-oaie.txt and then end up with:

$ ls
paeoai.txt
sotaoe.txt

How can I achieve this?

Answer 1

This answer focuses on zsh. Most of it can't be done easily in bash.

Many common cases can be done with wildcards. In particular:

• Everything with a fixed prefix: foo-*
• Everything with a fixed prefix followed by one more letter within a range: foo[b-m]* (includes foobar, food, foomz1, but not fooar or foon)
• Numbers in a range: IMG-<7-42>.* (includes IMG-8.PNG and IMG-0042.JPG but not IMG-77.JPG)

With glob qualifiers, there's an easy way to identify a range, but it requires counting: foo*([1,3]) matches the first 3 files listed by foo*, whatever they are (or all files if there are fewer than 3). This takes place after whatever sorting is done, for example foo*(om[1,3]) matches the three most recently modified files whose name starts with foo.

You can make zsh figure out the numbers for you. Do it in two steps: first put all the matches in an array, then find the endpoints using the subscript flags i and I (also e if you want to prevent any wildcard matching): $a[$a[(i)foo],$a[(I)bar]] is the part of the array $a from the element foo to the element bar inclusive, and empty if either foo or bar is not present.

a=(*.txt(oL))
# List the files that appear between small.txt and large.txt in a listing by size.
# Note that files that have the same size as one of the bounds may or may not be included.
echo $a[$a[(I)small.txt],$a[(I)large.txt]]

So here's a function that implements exactly the requirements of the question (except the exact syntax, which can't be done):

# Usage: select_range FROM TO WILDCARD_PATTERN
# Sets the array $s to the files matching PATTERN from FROM to TO inclusive.
function select_range {
  if (($# < 2)); then
    echo >&2 "select_range: missing range arguments"
    return 120
  fi
  local from=$1 to=$2
  shift 2
  from=$@[(ie)$from]
  if ((from == 0)); then
    echo >&2 "select_range: not matched: $from"
  fi
  to=$@[(Ie)$to]
  if ((to == 0)); then
    echo >&2 "select_range: not matched: $from"
  fi
  s=($@[$from,$to])
}

Usage: select_range aoeitoae.txt oaie.txt * && rm $s

The e glob qualifier lets you write arbitrary code to filter results, but it already starts getting a little unwieldy. Quoting can be tricky in complex cases; to keep things simple, use ' as the delimiter (which needs to be quoted with a backslash) and put the filter code in single quotes, meaning the pattern looks like this: foo-*(e\''code goes here'\'). (If the quoting gets too complicated, write a function and use the + qualifier.) To filter files that come after aoeitoae.txt and before oaie.txt in lexicographic order: *(e\''! [[ $REPLY < aoeitoae.txt || $REPLY > oaie.txt ]]'\').

Note that the comparisons done in the filter don't necessarily use the same order as the wildcard expansion. For example, foo-*(n) lists foo-9 before foo-10 thanks to the n qualifier, but [[ foo-9 > foo-10 ]] in a string comparison, and there's no conditional operator similar to > that compares integer substrings numerically. If you want to do a string comparison with integer parts sorted numerically, you can use the n parameter expansion flag for array sorting and check that it keeps the matched name in the middle: *(ne\''a=(b11r $REPLY f10o); [[ $a[2] == "${${(@n)a}[2]}" ]]'\')) includes b101r, b11s, d1, f02o, …, but not b9r, f011, …

If you're matching files by date, you can use the -nt conditional (note that a file is not newer than itself): *(ome\''! [[ $REPLY -ot from || $REPLY -nt to ]]'\') only includes files modified between the modification time of from and the modification time of to, inclusive.

Answer 2

Using a bash function:

sfiles ()
(
    # run this in a subshell, so we don't have to care if nullglob/dotglob were enabled or not
    [ $# -eq 0 ] && exit
local nullsep=0
if [ "$1" = "-0" ]; then
    nullsep=1; shift
fi
local first=$1
shift $(($# -1))
local last=$1
local files=( )

shopt -s nullglob dotglob
for i in *; do
    # first argument found or array not empty?
    if [ "$i" = "$first" ] || [ "${#files[@]}" -ne 0 ]; then
        files+=( "$i" )
    fi
    # last argument found? break loop
    [ "$i" = "$last" ] && break
done

if [ "${#files[@]}" -gt 0 ]; then
    [ "$nullsep" -eq 1 ] && 
        printf '%s\0' "${files[@]}" ||
        printf '%s\n' "${files[@]@Q}"
fi

)

It outputs all files between the first and the last argument (inclusive).

Examples:

$ ls -A
 btjhyyxrlv.txt    otewagahzp.txt       .xxx
 crlcsbzizl.txt    ssffszhdmp.txt      'zdjtgahx q.txt'
 hgiagchkgt.txt   'tt'$'\t''aa.txt'    'zmwik zhur.txt'
 jusupbivit.txt    umikyfucgu.txt      'z otmleqlq.txt'
' kcyigyurc.txt'  ' upvpntdfv.txt'      .zzz
 kfthnpgrxm.txt   'uu'$'\t\t''aa.txt'
 lgzsmquxwj.txt    wlwexgzohs.txt

$ sfiles c* k*
'crlcsbzizl.txt'
'hgiagchkgt.txt'
'jusupbivit.txt'
' kcyigyurc.txt'
'kfthnpgrxm.txt'

$ sfiles .xxx .zzz
'.xxx'
'zdjtgahx q.txt'
'zmwik zhur.txt'
'z otmleqlq.txt'
'.zzz'

$ LC_ALL=C sfiles .xxx .zzz
'.xxx'
'.zzz'

Wrong order, this one returns nothing:

$ sfiles .zzz .xxx

Remove selected files with xargs:

$ sfiles .xxx .zzz | xargs rm

For filenames with tabs or newlines, add option -0 as first argument for null-separated output without bash quoting.

$ sfiles -0 tt* uu* | xargs -0 ls
'tt'$'\t''aa.txt'  ' upvpntdfv.txt'
 umikyfucgu.txt    'uu'$'\t\t''aa.txt'

Answer 3

You could write your own shell function that does that for you: get a list of all files, and remove all before the first file name, and all after the second. Roughly, in zsh, because I think this will be more annoying in bash:

#!/usr/bin/zsh
filerange() {
  # variable "reply" is an array
  typeset -ag reply
  reply=("$1")
  # don't get upset if nothing matches
  setopt nullglob
get a complete list of candidate files
#allfiles=*(^/^on)
^--------- *   glob
^----^--- (…) glob modifiers within parentheses
^------- ^   invert match
^------ /   match directories (inverted: match all but dirs)
^----- ^   invert sort order (uninvert the above inversion)
^---- o   order by:
^--- n    character values of name
for fname in *(on^/); do
    [[ ( "$2" > "${fname}" && "${fname}" > "$1" ) ]] 

    && reply+=("${fname}")
  done
  reply+=("$2")
}

That gives you a nice and "safe" array in $reply which you can well use e.g. in for:

# prepare a test dir
mkdir /tmp/testdir
cd /tmp/testdir
sudo mknod blockdev b 1 1
sudo mknod chardev c 1 1
mkdir dir
mkfifo fifo
touch file
touch "file with spaces"
touch "file with\nnewlines"
touch "last file"
touch "ok one more file"
ln -s blockdev symlink
This will look funny, because there's a file name with a new line in there
echo *
Let's try this:
filerange chardev "last file"
We're expecting to get all files from (incl) chardev to (incl) last file,
but not dir, as that's a directory
for f in ${reply}; do;
  echo "entry: ${f}"
done

Of course, if you just want to print these files (e.g., for parallel or xargs), writing a function that takes an optional -0 as Freddy's excellent answer illustrates is a great choice! That function would be pretty barebone: it could just call filerange and deal with the $reply from that in a printf'ing for loop.

Answer 4

The obvious way would be

ls | sed -n -e "/$first/,/$last/p" | xargs rm

with $first and $last being regular expressions to match the first and last files to process:

first='^aoeitoae\.txt$'
last='^oaie\.txt$'

Of course, this is risky for a couple of reasons - firstly, ls implementations have differing behaviour for names containing non-printing characters, and xargs will do the Wrong Thing with names containing newline.

If we're using GNU tools, we can solve this by using \0 as the separator, with sed -z and xargs -0:

printf '%s\0' * | sed -zn -e "/$first/,/$last/p" | xargs -0r rm

Answer 5

This is not a complete answer, but I wanted to mention that awk has a nice range operator that could be used:

$ ls | awk '/^aoeitoae\.txt$/,/^oaie\.txt$/ { print }'
aoeitoae.txt
oaeistn.txt
oaie.txt

The awk code prints input lines, starting with the first one that matches the first regular expression and ending with the first one after that which matches the second regex.

Beware that the matches use regular expressions and that their is no quoting whatsoever in this example, so this might lead to unexpected results with whitespace or other strange characters in filenames.

And maybe you want to try using a text-based file manager like Midnight Commander.

Answer 6

This version can give you files with starting characters in a range:

filerange() {
    (
        local OPTARG OPTIND
        local fmt='%s\n'
        while getopts :0 opt; do
            [[ $opt == '0' ]] && fmt='%s\0'
        done
        shift $((OPTIND - 1))
    local pattern="[$1-$2]*"
    shopt -s nocasematch
    shopt -s nullglob
    for f in *; do 
        if [[ $f == $pattern ]]; then
            printf "$fmt" "$f"
        fi
    done
)

}

And you'd use it like

filerange e m
filerange -0 e m | xargs -0 rm -i

Remove shopt -s nocasematch if you don't want case insensitivity.

I suspect this isn't what you're after since it's not providing "from this file to that file" precision.

Answer 7

Based on your question, it seems that given your comparison to the graphical File Manager you might be interested in having a bit of interactivity - and also in turn, having a bit more general solution that applies to a different set of files.

The extra benefit is also that the code is fairly simple. All you need is the vipe pipe editor utility - that comes in moreutils package in most distributions.

DISCLAIMER: Before you move on to this, please be aware that this solution is not a perfect one - if your filenames contain characters such as line breaks, they will result in separate lines in your editor, which can result in errors and/or removing the files you don't want. Make sure you have no such files first.

What you will need to do, is to run the following command, and edit the list in the editor that pops up, so that it contains only the files that you want to remove.

$ ls | vipe | while read; do rm "$REPLY"; done

If you don't like the editor that pops up with vipe, you can change the EDITOR/VISUAL variable beforehand (VISUAL takes precedence):

$ EDITOR=nano

Answer 8

This solution maybe considered somewhat ugly, but it works:

(T=$(mktemp -u) && mkfifo "$T" && (ls -1 >"$T" & sleep 1) && echo '/^first_pattern$/,/^last_pattern$/p' | ed -s "$T" && rm "$T")

The sleep 1 isn't an elegant solution to deal with the race condition that ed might start before the FIFO had been created, but anyway:

• Write the directory contents to a temporary FIFO, one filename per line. As the FIFO may block the writer, do it asynchronously (in background).
• Use ed to read from the FIFO, outputting the lines from first_pattern matching first until (and including) last_pattern matching first.

For safety's sake I did not place rm $(...) around the command. So if you really want it, ...

This pattern is known to fail if the lines (read: filenames) contain spaces or other unusual characters.

Answer 9

Since you mentioned the Shift-click/select workflow: If you are indeed interested in interactively choosing a set of files to pipe to a next stage, you can use fzf.

For example, let's say you have 4 files - and you want to select some to delete. To make it more interesting, let's have spaces inside filenames:

$ mkdir a
$ cd a
$ touch a b c 'd e'  # Create 4 files, one containing spaces in its name
$ ls -l
... (see screenshot below)
$ ls | fzf -m --print0 | xargs -0 rm 

This screen will pop up:

And you can then select with the TAB key, and de-select with Shift-TAB, the files you want to pass to fzf's stdout:

Every time you hit TAB, the file will get a ">" marker on its left. De-select with Shift-TAB. You can navigate the list with fuzzy finding (what the acronym 'fzf' stands for) so feel free to type letters from the file(s) you want to jump to.

When you are satisfied, hit ENTER.

As you can see, the list of files was emitted to stdout, using null terminators to separate them (option --print0). The next part of the pipeline was xargs - which, due to the -0 option splitted the input back into separate filenames, and passed each one of them to rm. Our two selected files were thus removed.

The use of null separators guarantees that the pipeline will work no matter what crazy characters may exist in your filenames (spaces, commas, newlines, whatever).

Hope this helps.