Bash: Why is eval and shift used in a script that parses command line arguments?

Question

As I was looking this answer https://stackoverflow.com/a/11065196/4706711 in order to figure out on how to use parameters like --something or -s some questions rised regarding the answer's script :

#!/bin/bash
TEMP=`getopt -o ab:c:: --long a-long,b-long:,c-long:: \
     -n 'example.bash' -- "$@"`

if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi

# Note the quotes around `$TEMP': they are essential!
eval set -- "$TEMP"

while true ; do
    case "$1" in
        -a|--a-long) echo "Option a" ; shift ;;
        -b|--b-long) echo "Option b, argument \`$2'" ; shift 2 ;;
        -c|--c-long) 
            # c has an optional argument. As we are in quoted mode,
            # an empty parameter will be generated if its optional
            # argument is not found.
            case "$2" in
                "") echo "Option c, no argument"; shift 2 ;;
                *)  echo "Option c, argument \`$2'" ; shift 2 ;;
            esac ;;
        --) shift ; break ;;
        *) echo "Internal error!" ; exit 1 ;;
    esac
done
echo "Remaining arguments:"
for arg do echo '--> '"\`$arg'" ; done

First of all what does the shift program in the following line:

        -a|--a-long) echo "Option a" ; shift ;;

Afterwards what is the purpose to use the eval command in the following line:

eval set -- "$TEMP"

I tried to comment the line in script mentioned above and I got the following response:

$ ./getOptExample2.sh  -a 10 -b 20 --a-long 40 -charem --c-long=echi
Param: -a
Option a
Param: 10
Internal error!

But if I uncomment it it runs like a charm:

Option a
Option b, argument `20'
Option a
Option c, argument `harem'
Option c, argument `echi'
Remaining arguments:
--> `10'
--> `40'

Answer 1

One of the many things that getopt does while parsing options is to rearrange the arguments, so that non-option arguments come last, and combined short options are split up. From man getopt:

Output is generated for each element described in the previous section.
Output is done in the same order as the elements are specified  in  the
input,  except  for  non-option  parameters.   Output  can  be  done in
compatible (unquoted) mode, or in such way that  whitespace  and  other
special  characters  within  arguments  and  non-option  parameters are
preserved (see QUOTING).  When the output is  processed  in  the  shell
script,  it  will  seem to be composed of distinct elements that can be
processed one by  one  (by  using  the  shift  command  in  most  shell
languages).
[...]
Normally, no  non-option  parameters  output  is  generated  until  all
options  and  their  arguments  have  been  generated.   Then  '--'  is
generated as a single parameter, and after it the non-option parameters
in  the  order  they were found, each as a separate parameter.

This effect is reflected in your code, where the option-handling loop assumes that all option arguments (including arguments to options) come first, and come separately, and are finally followed by non-option arguments.

So, TEMP contains the rearranged, quoted, split-up options, and using eval set makes them script arguments.

Why eval? You need a way to safely convert the output of getopt to arguments. That means safely handling special characters like spaces, ', " (quotes), *, etc. To do that, getopt escapes them in the output for interpretation by the shell. Without eval, the only option is set $TEMP, but you're limited to what's possible by field splitting and globbing instead of the full parsing ability of the shell.

Say you have two arguments. There is no way to get those two as separate words using just field splitting without additionally restricting the characters usable in arguments (e.g., say you set IFS to :, then you cannot have : in the arguments). So, you need to able to escape such characters and have the shell interpret that escaping, which is why eval is needed. Barring a major bug in getopt, it should be safe.

---

As for shift, it does what it always does: remove the first argument, and shift all arguments (so that what was $2 will now be $1). This eliminates the arguments that have been processed, so that, after this loop, only non-option arguments are left and you can conveniently use $@ without worrying about options.

Answer 2

Afterwards what is the purpose to use the eval command in the following line:

eval set -- "$TEMP"

The util-linux version of getopt produces output that's usable as input to the shell. It surrounds strings containing whitespace with quotes, and handles escaping of literal quotes and other special characters.

E.g.

$ getopt -o a:b -- -a 'foo bar' -b "single'quotes'here"
 -a 'foo bar' -b -- 'single'\''quotes'\''here'

The quotes won't be processed from the result of a plain expansion, but a full round of parsing is required. And that's what eval does.

If the output is assigned to $tmp, then after eval set -- "$tmp", the positional parameters $1, $2, ... contain -a, foo bar -b -- single'quotes'here, and that's relatively easy to process in a loop.

Merely using set -- $tmp would set the positional parameters to -a, 'foo, bar', etc..., which is not what you want. (Also you'd get globbing on top, if one of the args was e.g. *.)

The problems are similar to those in How can we run a command stored in a variable?, both cases involve lists of arbitrary strings.

---

Note that the behaviour where getopt produces shell-quoted output is specific to the util-linux version of it. Other systems often have a getopt that is used without eval, and which happily breaks arguments that contain whitespace or look like globs. As in this one, there's no way to tell from the output that foo bar is supposed to be a single argument.

$ getopt a:b -a 'foo bar' -b "single'quotes'here"
 -a foo bar -b -- single'quotes'here

When using getopt, make sure to use the -T/--test option first to see if you have the safe version that can deal with arbitrary strings.

Answer 3

The script works correctly when it gives you an error for -a 10. The -a option needs no parameter in this script. You should only use -a.

The shift described in the man page as the following:

shift [n]
              The positional parameters from n+1 ... are renamed to $1 ....  Parameters represented by the numbers $# down to $#-n+1 are unset.  n must be a non-negative number less than or equal to $#.  If  n  is
              0,  no  parameters are changed.  If n is not given, it is assumed to be 1.  If n is greater than $#, the positional parameters are not changed.  The return status is greater than zero if n is greater
              than $# or less than zero; otherwise 0.

So basically it drops -a and shift the remaining arguments so the second parameter will be $1 in the next cycle.

-- is also described in the man page:

 --        A -- signals the end of options and disables further option processing.  Any arguments after the -- are treated as filenames and arguments.  An argument of - is equivalent to --.