Passing paths with spaces as arguments

Question

I am having difficulty in passing some string variables having spaces in them as arguments to a program.
For debugging and showing the arguments being passed, I created a demo Python script -:

##### show_args.py #####

import sys

def main():
    # Display the arguments passed to the script
    print "Number of arguments =", len(sys.argv)
    for arg in sys.argv:
        print arg

if __name__ == '__main__':
    main()

Now, the script demonstrating the problem -:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc $path_with_spaces/xyz"

python show_args.py $arg_list

Output -:

Number of arguments = 5
show_args.py
/home/anmol/blah
blah/abc
/home/anmol/blah
blah/xyz

What I actually want is this -:

path_with_spaces="$HOME/blah blah"

python show_args.py "$path_with_spaces/abc" "$path_with_spaces/xyz"

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah blah/abc
/home/anmol/blah blah/xyz

To confirm that the problem was occurring only for paths with spaces in them, I created the following script -:

path_without_spaces="$HOME/blah"
arg_list="$path_without_spaces/abc $path_without_spaces/xyz"

python show_args.py $arg_list

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah/abc
/home/anmol/blah/xyz

While searching for the solutions to this problem, I encountered this answer, according to which the correct way is to put the arguments in an array variable rather than a string variable.
The script showing this new approach -:

path_with_spaces="$HOME/blah blah"
arg_list=("$path_with_spaces/abc" "$path_with_spaces/xyz")

python show_args.py "${arg_list[@]}"

Output -:

Number of arguments = 3
show_args.py
/home/anmol/blah blah/abc
/home/anmol/blah blah/xyz

Although, this solution is working correctly, I still want to know if there is a way with which we can accomplish the same thing using a string variable rather an array variable.

My system configuration -:

Ubuntu 14.04 LTS
Bash 4.3.11
Gnome Terminal 3.6.2

there could be several strategies. one ugly back i used to do in bash was replacing spaces for _ or "x". use your imagination — Rui F Ribeiro, Dec 22 '15 at 18:45

score 4 · Accepted Answer · answered Dec 22 '15 at 19:04

In:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc $path_with_spaces/xyz"

python show_args.py $arg_list

You're using a scalar/string variable and using the split+glob operator (leaving that variable unquoted) to split the content of that variable and generate the arguments to pass to python.

The split part of the split+glob operator splits on the characters stored in the $IFS special parameter.

Here, you could disable the glob part and split on a character not found in your paths. For instance, if newline is not found in the paths:

path_with_spaces="$HOME/blah blah"
arg_list="$path_with_spaces/abc
$path_with_spaces/xyz"
IFS="
"
set -f # disable glob.
python show_args.py $arg_list

You could also use shell quoting and use eval to have the shell interpret those quotes (here using bash-specific features):

printf -v arg_list '%q ' "$path_with_spaces/abc" \
                         "$path_with_spaces/xyz"

eval "python show_args.py $arg_list"

mikeserv · Answer 2 · 2015-12-22T20:06:11.907

Yes.

You can use the default path separator to separate paths. There are two kinds of characters which are illegal in a path-name:

the \0NUL character
the / path separator

As it happens, you can separate path-names on the separator with the shell's Internal Field Separator $IFS special parameter. When two or more separators occur in sequence, the resulting component will be a \0NUL valued argument - which cannot be a path-name.

To work you must root all paths directly (and so rule out a root //), and you must ensure you have canonical paths - or at the least that all //* have been squeezed to a single occurrence per (dots won't hurt - or help). Once you have done so:

paths='/path/one//path/t w o//p a t h/t h r e e/'
set -f --; IFS=/
for p in ${paths#/}$IFS
do  printf ${p:+/}%s\\n "$*"
    set -- ${p:+"$@"/$p}
done

/path/one
/path/t w o
/p a t h/t h r e e

You can get a canonical path in $PWD w/ cd.

cd -- /some//../screwy/../path/to///destination
printf %s\\n "$PWD"

/path/to/destination

If you use the -P switch cd will ensure an absolute, physical path to the current working directory is set to $PWD. Else with -L or by default cd will maintain any indirections implied by previous changes to symlinked directories as your shell session progresses.

And so you can most reliably collect the names in a tree by walking it.

Passing paths with spaces as arguments

2 Answers2