'For' loop won’t iterate over list when path given as argument

Question

I'm practicing shell scripts and am trying to make a simple script that takes a directory as an argument, loops through each file within and prints out its name and size.

#!/bin/bash
# A practice shell script to try and display a list of file names
# and their sizes using the output of ls -l and cut.
declare -i index
export index=0
export name=""
export size=0
for file in $1 ; do
   index+=1
   name=basename $file
   size=ls -l $file | cut -d " " -f 5
   echo "$index: $name, size: $size bytes"
done

When I give ./* as the argument, it does it for one file and that's it. However, if I edit the code above and just put ./* in place of $1, it works and loops through all files in the current directory.

Why won’t it do the same, when $1 is supposed to equal ./*?

Answer 1

The reason is that the shell from which you call the script expands the globbing pattern ./* before it is passed to the script. That means, that if your globbing pattern matches e.g. file1.txt to file4.txt, calling the script as

./my_script.sh ./*

will actually be interpreted as

./my_script.sh file1.txt file2.txt file3.txt file4.txt

and these will be the arguments that the shell script sees.

For further reading, have a look at the section on shell expansion order in the Bash Reference manual.

There are two possibilities to overcome the problem:

1. If you are sure that you always want to iterate over all files in a given directory, pass the directory as argument, and iterate over

   for f in "$1"/*
   do
       # operations on "$f"
   done
   

2. Alternative, if you are sure that you will only pass file names to operate on, iterate over the entire argument list, as in

   for f in "$@"
   do
       # operations on "$f"
   done
   

---

If you want to do it by passing a glob pattern into the script - which certainly is an interesting exercise - this is also possible (see comment by @ilkkachu). As mentioned by @fra-san in a comment, the approach has advantages - it can add more flexibility to the script usage, and it circumvents the limitation on shell command-line parameters (cf. "argument list too long"; though RAM will still limit the length of the resulting filename list) - but requires you to be extra careful.

• You can prevent the shell from expanding the glob by enclosing the argument in quotes (single or double), or escaping the glob character with a backslash:

  ./my_script.sh "./*"
  ./my_script.sh './*'
  ./my_script.sh ./\*
  

• Inside the script, you would refer to the positional parameter $1 unquoted so that it actually is interpreted by the shell (something that we often want to avoid).
• Since that "interpretation" not only involves expansion (see above), but also word splitting you need to set the input field separator IFS to the empty string to ensure that no word-splitting occurs.
• The for loop would then look like

  IFS=
  for f in $1
  do
      # Operations on "$f"
  done
  

---

A few general notes on your script:

• Always quote shell variables, in particular when they contain filenames, as otherwise your script will stumble on filenames with spaces or other, even more exotic characters in them - remember, even the newline is an allowed character for filenames (yuck)!
• Parsing the output of ls is highly discouraged for similar reasons. If you want to identify attributes of a particular file, the stat tool is a better choice. For determining the size of a file, e.g., you could use

  size=$(stat --printf="%s" "$f")
  

• It is recommended to use the "new" $( ... ) style for command substitutions rather then the old "backtick-style" ` ... `.
• It is a good habit to check your shell scripts with shellcheck, also available as standalone tool in many Linux distributions, to guard against this (and other) possible error sources.

Answer 2

You need to be aware of how bash interprets wildcards and arguments. When there is a wildcard, bash interprets it on time and replaces it with all files matching. And that happends, when you replace $1 with ./* - it takes all files from current directory and cycles through them.

When you have

for file in $1 ; do

it will take just the first argument. That's not what you want. If you want to cycle through all files, you need to use:

for file in $* ; do

(which will take ALL arguments)

Alternatively, you can cycle through them using shift - which removes first argument as long as there are other:

while [ $# -gt 0 ]
do
    file=$1
    shift
    ...
done