Using an if statement to see if files are in a folder using a wildcard

Question

I have to process multiple files that I dont have the entire name for. I am running an if statement using a wildcard. for example...

if [ "ESGSN_*.SMS.EDR" ]; then
    echo "There are files to process in this run"
    for F in ESGSN_*.SMS.EDR; do
        gzip < "$F" > "$F.gz"
    done
fi

If there are multiple files or a single file this will work and zip them. But if there are no files it will still run the zip rather than just end. I was wondering if there is any way to get it to complete and move on.

I have tried using the operators "-f" and "-a" within the brackets just before the filename, but those will error out with: "binary operator expected"

for some reason the asterisk in the filename is not showing down below in the message. So the file name is "ESGSN_"*".SMS.EDR" where the double quotes is the asterisk.

Answer 1

So I have simplified it a bit for you.

#!/bin/bash

shopt -s nullglob
files=(yourdir/*)

if [[ ${#files[@]} == 0 ]]  # If array is equal to 0
  then echo "No files there" >&2
  exit 1
else
  gzip "${files[@]}" # gzip all elements within the array.
fi
shopt -u nullglob 

Explanation what the script does:

Setting nullglob by shopt -s nullglob

nullglob is used to count the files that match the pattern. Without nullglob the glob * would expand to a literal * meaning that the count will always be > 1 in an empty directory, resulting in an erroneous count.

Unsetting nullglob by shopt -u nullglob

This is importand if we have more code after this line that could be affected by nullglob.

Answer 2

if ! gzip ESGSN_*.SMS.EDR 2>/dev/null; then
    printf 'No files to process (gzip exit code: %d)\n' "$?"
fi

Your test will always be true as the string ESGSN_*.SMS.EDR is non-empty. The * glob will not be processed when it's in double quotes.

Also, most shells will by default retain the pattern unexpanded if it doesn't match anything, which in your case means that your code would end up creating an empty file called ESGSN_*.SMS.EDR.gz if the pattern doesn't match any filenames.

The code above will simply try to compress all files that match the pattern, and if it fails it prints a message.

Answer 3

There are two ways to answer this: fix your current script, or propose an alternative which will be faster and more precise. I'll attempt both :)

First of all, a problem with your script the way it is, is that [ "ESGN_*.SMS.EDR" ] will always evaluate as true. The [ "<string>" ] construct simply tests whether the string is non-empty. Even when there are no files, the string you've given is non-empty, therefore the test is not doing anything.

What you would need to do to fix it is to loop over all files and test whether they match the pattern. The following works for me:

for F in *; do
    if [[ "$F" =~ ESGSN_.*\.SMS\.EDR ]]; then
        printf "Processing '%s'\n" "$F"
        gzip < "$F" > "$F.gz"
    fi
done

However, that's not how I would really do it, if I wanted to solve this problem for my own use. Instead it is faster and more natural to use GNU find, which easily does pattern-matching:

find ./ -type f -regex ".*/ESGSN_.*\.SMS\.EDR" -exec bash -c "gzip < {} > {}.gz" \;

The find command is very powerful and if you're going to be using Linux a lot you will wish to make yourself familiar with it. I recommend having a look through man find.

Answer 4

Thanks everyone. I was able to get what I needed from:

if ls -lrt ESGSN_*.SMS.EDR ; then