bash count files in path with wildcard and assign to a variable

Question

What am I missing?

I want to find out how many files match a given pattern in a directory and assign it to a variable.

If I type it all straight on the command line it works fine

ls /backups/system\ db\ files/temp/daily/backup_filename_202203*.tar.gz | wc -l

(n.b. the * is the date the backup was created, e.g. 01, 02, 03 etc.)

But as soon as I add it to my bash script it fails.

So far I have:

base_dir="/backups/system\ db\ files/temp" 
sub_dir="${base_dir}/daily"
filename_base="backup_filename_"

and I then try and run:

counter=$(ls ${sub_dir}/${filename_base_}202203*.tar.gz | wc -l)

or with quotes:

counter=$(ls "${sub_dir}/${filename_base_}202203*.tar.gz" | wc -l)

The first one fails as it tries to split it based on the whitespaces. The second one doesn't expand the * to look for the wildcards.

I have tried just quoting some of it, e.g.

counter=$(ls "${sub_dir}/${filename_base_}"202203*.tar.gz | wc -l)

But again it ignores the wildcard.

I've been searching but for the life of me can't find a way to get the total number of matching files.

Answer 1

Your issue is that your directory is not called /backups/system\ db\ files/temp, it is called backups/system db files/temp. When you quote the string, you make the backslashes part of the string. Since you quote the string, there is no need to escape the spaces. You also have the wrong spelling of the variable filename_base in a few places (with a trailing underscore).

You also don't need to use ls and wc to count the number of names matching a shell pattern. Remember that filenames can have newlines in their names and that wc -l counts newlines, not filenames.

All you need to do to see how many matches there are to a shell pattern is to expand the pattern into an array or into the list of positional parameters and then see how long that array or list became.

Using an array with an example pattern:

list=( "some/path to the/files"/*.txt )
count=${#list[@]}

Using the list of positional parameters:

set -- "some/path to the/files"/*.txt
count=$#

In the bash shell, you will have to set the nullglob shell option using shopt -s nullglob to correctly handle empty directories (the pattern is usually left unexpanded if it does not match anything).

Without setting nullglob, you may instead test whether there were no matches using

list=( "some/path to the/files"/*.txt )
count=${#list[@]}
if [ ! -e "${list[0]}" ] && [ ! -h "${list[0]}" ]; then
    count=0
fi

or, if you used the positional parameters,

set -- "some/path to the/files"/*.txt
count=$#
if [ ! -e "$1" ] && [ ! -h "$1" ]; then
    count=0
fi

If the negated -e and -h tests are true for the first element in the resulting list, then the first element is not an existing file, and it is not a broken symbolic link. This means that there were no matches for the pattern.

Answer 2

You don't escape the spaces when you already quoted them, otherwise the backslashes will be taken literally:

base_dir="/backups/system\ db\ files/temp" 

should be

base_dir="/backups/system db files/temp"

Also why do you have _ in ${filename_base_} ?

Your last command should have probably worked if everything else was correct:

counter=$(ls "${sub_dir}/${filename_base}"202203*.tar.gz | wc -l)

---

Anyways, you should not parse ls.

Rather use find:

find "${sub_dir}" -name "${filename_base}202203*.tar.gz" -printf '.' | wc -c

or an array:

shopt -s nullglob
files=("${sub_dir}/${filename_base}"202203*.tar.gz)
echo "${#files[@]}"

You need the nullglob option, otherwise the result will be 1 if no files match your pattern as the string is then taken literally.