Why does using double brackets "[[" to check if wildcard matched files exists fail, while single brackets "[" work?

Question

Let's say I generate a two directories with text files in each like so

mkdir "Directory1"
mkdir "Directory2"
touch "Directory1/fileclass1_"{1..5}".txt"
touch "Directory1/fileclass2_"{1..5}".txt"
touch "Directory2/fileclass1_"{1..5}".txt"
touch "Directory2/fileclass2_"{1..5}".txt"

Suppose I verify all the files are inside by doing

A=( "Directory1" "Directory2" )
B=( "fileclass1" "fileclass2" )
for a in "${A[@]}"; do
    for b in "${B[@]}"; do
        for i in {1..5}; do
            name="${a}/${b}*${i}.txt"
            [[ ! -e $name ]] && echo "$name Does Not Exist"
        done
    done
done

This returns

Directory1/fileclass1*1.txt Does Not Exist
Directory1/fileclass1*2.txt Does Not Exist
...

However, if I instead replace the double brackets with singles, I get

A=( "Directory1" "Directory2" )
B=( "fileclass1" "fileclass2" )
for a in "${A[@]}"; do
    for b in "${B[@]}"; do
        for i in {1..5}; do
            name="${a}/${b}*${i}.txt"
            [ ! -e $name ] && echo "$name Does Not Exist"
        done
    done
done

which returns nothing, indicating all files are indeed there.

Why is it in this case, the double brackets fail, while single brackets work? I was under the assumption I should always employ double brackets, is the wildcard in string matching creating something I shouldn't have?

From man bash: Word splitting and pathname expansion are not performed on the words between the [[ and ]] — steeldriver, Jul 10 '20 at 22:05
FWIW neither syntax provides a robust way to do what you want - for better alternatives, see Test if there are files matching a pattern in order to execute a script — steeldriver, Jul 10 '20 at 22:53

score 6 · Accepted Answer · answered Jul 10 '20 at 23:47

The -e conditional operator tests whether a file with the given name exists. It doesn't do any pattern matching. [[ -e fileclass1*1.txt ]] tests whether there is a file called fileclass1*1.txt with a literal asterisk in the name, not whether there is at least one file matching the wildcard pattern fileclass1*1.txt.

With a single bracket [ ! -e $name ], since $name is unquoted, the value of name undergoes wildcard substitution and word splitting. If the value of name is Directory1/fileclass1*1.txt, the effect depends on how many files match the wildcard pattern.

If the pattern doesn't match any file, it's left unexpanded. Then the -e operator sees the file name Directory1/fileclass1*1.txt, which doesn't exist, so [ ! -e $name ] is true.
If the pattern matches a single file, it's replaced by that file's name. Then the -e operator sees that file name, which exists¹, so [ ! -e $name ] is false.
If the pattern matches two or more files, it's replaced by the list of matching file names, which results in a syntax error in the conditional. In this case, [ ! -e $name ] displays an error message and returns a failure status.

The effect is different with [[ -e $name ]] because [[ … ]] is special syntax (whereas [ is an ordinary command whose arguments are expanded normally). No word splitting or wildcard expansion is done inside [[ … ]].

To solve your actual problem, see Test if there are files matching a pattern in order to execute a script

¹ _{Except in the unlikely case that due to a race condition, the file is removed between the time the shell lists the directory content to expand the wildcard pattern, and the time the -e operator checks for the file's existence.}

Why does using double brackets "[[" to check if wildcard matched files exists fail, while single brackets "[" work?

1 Answers1