3

I was reading the man page of find and I found myself confused with the following commands. What is the difference between one and its corresponding one.

  1. What is the difference between the following two commands:

    find -execdir command "{}" \;
    find -execdir "command {}" \;
    

Cause of confusion: I thought quotation should instruct the shell to take quoted parts as a single chunk. Hence, when I saw the second one, I thought it would fail because there wouldn't be a command command <file-name>.

  1. What is the difference between the following two:

    find -execdir bash -c "command" "{}" \;
    find -execdir bash -c "command {}" \;
    

Cause of confusion: enclosing the command along with the curly braces in the second version, to my understanding, should be passed as a whole to the bash command, and find should not interpret the braces with their corresponding files name.

  1. What is the difference between the following two:

    find -execdir bash -c "something \"$@\"" {} \;
    find -execdir bash -c 'something "$@"' bash {} \;
    

To my understanding, both are identical. How does passing the braces to the shell, the second version, will be any safer than the first.

UPDATE

Just noticed that the first version of the commands in question#3 is not working! Tried the following (none is working):

find -execdir bash -c 'something \"$@\"' {} \; # changed external quotes to single ones.
find -execdir bash -c "something $@" {} \; # removed the inner quotes.
find -execdir bash -c 'something $@' {} \; # same as above but with outer single quotes instead.

What am I missing here? I believe I should be able to leave the curly braces out of the quotes of enclosing the command?

joker
  • 594

1 Answers1

3

This is (as you've noticed) rather complicated; I'll try to explain it. It's helpful to think in terms of the parsing/processing sequence the command(s) go through, and watch what happens at each step. In general, the process looks something like this:

  1. The shell parses the command line, breaking it into tokens ("words"), substituting variable references etc, and removing quotes and escapes (after their effect has been applied). It then (usually) runs the first "word" as the command name ("find" in these cases), and passes the rest of the words to it as arguments.
  2. find searches for files, and runs the stuff between its "-execdir" and ";" as commands. Note that it replaces "{}" with the matched filename, but does no other parsing -- it just runs the first arg after "-execdir" as the command name, and passes the following arguments to that as its arguments.
  3. In the case where that command happens to be bash and it gets passed the -c option, it parses the argument right after -c as a command string (sort of like a miniature shell script), and the rest of its arguments as arguments to that mini-script.

Ok, a couple of other notes before I dive into this: I'm using BSD find, which requires that the directory to search be explicitly specified, so I'll be using find . -execdir ... instead of just find -execdir .... I'm in a directory that contains the files "foo.txt" and "z@$%^;*;echo wheee.jpg" (to illustrate the risks of using bash -c wrong). Finally, I have a short script called pargs in my binaries directory that prints its arguments (or complains if it didn't get any).

Question one:

Now let's try out at the two commands in your first question:

$ find . -execdir pargs "{}" \;
pargs got 1 argument(s): '.'
pargs got 1 argument(s): 'foo.txt'
pargs got 1 argument(s): 'z@$%^;*;echo wheee.jpg'
$ find . -execdir "pargs {}" \;
find: pargs .: No such file or directory
find: pargs foo.txt: No such file or directory
find: pargs z@$%^;*;echo wheee.jpg: No such file or directory

This matches your expectation: the first worked (and BTW you could've left off the double-quotes around {}), and the second failed because the space and filename was treated as part of the command name, rather than an argument to it.

BTW, it's also possible to use -exec[dir] ... + instead of -exec[dir] \; -- this tells find to run the command as few times as possible, and pass a bunch of filenames at once:

$ find . -execdir pargs {} +
pargs got 3 argument(s): '.' 'foo.txt' 'z@$%^;*;echo wheee.jpg'

Question two:

This time I'll take the options one at a time:

$ find . -execdir bash -c "pargs" "{}" \;
pargs didn't get any arguments
pargs didn't get any arguments
pargs didn't get any arguments

"Huh", you say? What's going on here is that bash is getting run with an argument list like "-c", "pargs", "foo.txt". The -c option tells bash to run its next argument ("pargs") like a miniature shell script, something like this:

#!/bin/bash
pargs

...and sort-of passes that "mini-script" the argument "foo.txt" (more on this later). But that mini-script doesn't do anything with its argument(s) -- specifically, it doesn't pass them on to the pargs command, so pargs never sees anything. (I'll get to the proper way to do this in the third question.) Now, let's try the second alternate of the second question:

$ find . -execdir bash -c "pargs {}" \;
pargs got 1 argument(s): '.'
pargs got 1 argument(s): 'foo.txt'
pargs got 1 argument(s): 'z@$%^'
bash: foo.txt: command not found
wheee.jpg

Now things are sort of working, but only sort of. bash gets run with the arguments "-c" and "pargs " + the filename, which works as expected for "." and "foo.txt", but when you pass bash the arguments "-c" and "pargs z@$%^;*;echo wheee.jpg", it's now running the equivalent of this as its mini-script:

#!/bin/bash
pargs z@$%^;*;echo wheee.jpg

So bash will split that into three commands separated by semicolons:

  1. "pargs z@$%^" (which you see the effect of)
  2. "*", which expands to the words "foo.txt" and "z@$%^;*;echo wheee.jpg", and hence tries to run foo.txt as a command and pass it the other filename as an argument. There's no command by that name, so it gives an appropriate error.
  3. "echo echo wheee.jpg", which is a perfectly reasonable command, and as you can see it prints "wheee.jpg" to the terminal.

So it worked for a file with a plain name, but when it ran into a filename that contained shell syntax, it started trying to execute parts of the filename. That's why this way of doing things is not considered safe.

Question three:

Again, I'll look at the options one at a time:

$ find . -execdir bash -c "pargs \"$@\"" {} \;
pargs got 1 argument(s): ''
pargs got 1 argument(s): ''
pargs got 1 argument(s): ''
$ 

Again, I hear you say "Huh????" The big problem here is that $@ is not escaped or in single-quotes, so it gets expanded by the current shell context before it's passed to find. I'll use pargs to show what find is actually getting as arguments here:

$ pargs . -execdir bash -c "pargs \"$@\"" {} \;
pargs got 7 argument(s): '.' '-execdir' 'bash' '-c' 'pargs ""' '{}' ';'

Note that the $@ just vanished, because I was running this in an interactive shell that hadn't received any arguments (or set them with the set command). Thus, we're running this mini-script:

#!/bin/bash
pargs ""

...which explains why pargs was getting a single empty argument.

If this were in a script that had received arguments, things would be even more confusing. Escaping (or single-quoting) the $ solves this, but still doesn't quite work:

$ find . -execdir bash -c 'pargs "$@"' {} \;
pargs didn't get any arguments
pargs didn't get any arguments
pargs didn't get any arguments

The problem here is that bash is treating the next argument after the mini-script as the name of the mini-script (which is available to the mini-script as $0, but is not included in $@), not as a regular argument (i.e. $1). Here's a regular script to demo this:

$ cat argdemo.sh 
#!/bin/bash
echo "My name is $0; I received these arguments: $@"
$ ./argdemo.sh foo bar baz
My name is ./argdemo.sh; I received these arguments: foo bar baz

Now try this with a similar bash -c mini-script:

$ bash -c 'echo "My name is $0; I received these arguments: $@"' foo bar baz
My name is foo; I received these arguments: bar baz

The standard way to solve this is to add a dummy script-name argument (like "bash"), so that the actual arguments show up in the usual way:

$ bash -c 'echo "My name is $0; I received these arguments: $@"' mini-script foo bar baz
My name is mini-script; I received these arguments: foo bar baz

This is exactly what your second option does, passing "bash" as the script name and the found filename as $1:

$ find . -execdir bash -c 'pargs "$@"' bash {} \;
pargs got 1 argument(s): '.'
pargs got 1 argument(s): 'foo.txt'
pargs got 1 argument(s): 'z@$%^;*;echo wheee.jpg'

Which finally works -- for real, even on weird filenames. That's why this (or the first option in your first question) is considered a good way to use find -exec[dir]. You can also use this with the -exec[dir] ... + method:

$ find . -execdir bash -c 'pargs "$@"' bash {} +
pargs got 3 argument(s): '.' 'foo.txt' 'z@$%^;*;echo wheee.jpg'
  • I wanna thank you immensely for your detailed answer. I have some points that I have would love you shed some light on.
    1. Do you mind explaining a bit more this part Thus, it runs "pargs" as a script, passing the script the argument "foo.txt", but that script doesn't do anything with its argument -- specifically, it doesn't pass it on to the pargs command, so pargs never sees it. What do you mean by running as script? Seems like you have something specific in mind when you wrote it. Passing the script an argument is the same as passing the pargs the argument. Right?
    – joker Jan 11 '19 at 14:20
  • I understand that $@ expands to all arguments to a script. $0 is special as it holds the script name. How is it that it doesn't work to just type in {} without any preceding dummy script name? {} should evaluate as $0 based on your explanation.
  • – joker Jan 11 '19 at 14:23
  • BTW, I tried lol, foo, 1, k as dummy words (not actually script names) in the place of the dummy script and it worked. So, seems like pretty much anything there words (doesn't have to be an actual script name). Just sharing. – joker Jan 11 '19 at 14:26
  • 1
    @joker I've edited in expanded explanations of how bash -c runs its next argument like a miniature script, and how it handles $0 vs $1 and $@. See if that clarifies things enough. – Gordon Davisson Jan 15 '19 at 23:30
  • I appreciate your clear explanation. Thanks a million. I have one last question for me to conclude the whole discussion. Why in the command find . -execdir bash -c "pargs {}" \; the {} get expanded to its corresponding file name? I mean, I thought enclosing {} with pargs in a single argument using the quotes would prevent the shell from interpreting parts of an argument. This, to my understanding, argument should be treated as a mere string. All special characters inside a double quotes lose their meaning except $, \``, and\`. – joker Jan 16 '19 at 10:41
  • 1
    @joker It prevents the first shell from interpreting it. Refer back to the list of processing steps at the beginning of my answer: the double-quotes prevent the shell from doing anything special with the {} in step 1 (though there's nothing it would to anyway -- the only effect the double-quotes have there is making it treat "pargs {}" as a single "word" instead of two). The quotes are then removed before it's passed to find as an argument. In step 2, find replaces {} with the matched filename (the quotes are gone by this point, and wouldn't matter anyway). – Gordon Davisson Jan 16 '19 at 22:33
  • 1
    Then in step 3, the new instance of bash parses what I've been calling the mini-script, which contains the filename and no quotes at all. Note that bash parses the argument right after -c as containing shell syntax (because it's a mini-script), but not the rest of its arguments (they're just arguments to the mini-script -- raw data, not shell code). That's why shell syntax in a filename causes trouble when it's embedded in the mini-script argument, but not when it's passed separately. – Gordon Davisson Jan 16 '19 at 22:36
  • I understand. The cause of the confusion I had is that I thought find would only replace {} if it's a word by itself. Enclosing {} with double quotes with a command made both of them one word as you mentioned. So I thought find wouldn't interpret the embedded {}. Interesting! How then can someone print {} as is without getting expanded to some file name? – joker Jan 17 '19 at 06:56
  • 1
    @joker All versions of find I'm familiar with will replace {} even if it's just part of an argument, but this isn't fully standard. According to the POSIX standard for find -exec "If a utility_name or argument string contains the two characters '{}', but not just the two characters '{}', it is implementation-defined whether find replaces those two characters or uses the string without change." I don't think there's way to use {} as part of one of the command args without it being replaced. – Gordon Davisson Jan 17 '19 at 07:08
  • I don't know how to thank you for your patience and taking your time and putting your effort to respond to all my queries. – joker Jan 17 '19 at 11:09