2

In short, I want to use directories listed by a command in a find command:

find $(produces_dir_names --options...) -find-options...

The problem comes with white space in the directory names. I thought quoting them in the output of the producing command (which I can change) would be enough:

"a" "a b" "a b c"

but bash complains:

find: ‘"a"’: No such file or directory
find: ‘"a’: No such file or directory
find: ‘b"’: No such file or directory
find: ‘"a’: No such file or directory
find: ‘b’: No such file or directory
find: ‘c"’: No such file or directory

As you can, see bash will split the output of the command on spaces, even with the quotation marks. I tried to fiddle with IFS and set it to \n, but my understanding of it seems too limited to get it working.

The only workaround I found was in this Stack Overflow question: bash command substitution remove quotation, namely putting an eval in front of it, but this seems kind of ugly.

My Questions:

Is there an easy way and how would it look like to write this substitution, without the eval?

Are the quotations even necessary?

Example (producing the same output):

find $(echo '"a" "a b" "a b c"')
ness
  • 23
  • 1
    This is the famous don't parse ls problem, also discussed extensively here. You might be able to find a way to make it work, but it's almost always going to be fraught. Also, setting IFS to \n isn't safe either, because filenames can contain \n – Eric Renouf Oct 24 '16 at 17:56
  • Oh, sorry. I used the output of another try, which was ls $(produces_dir_names --options...) to see if find was not understanding the output of the substituion correctly. I changed the example output in my question. – ness Oct 24 '16 at 18:14
  • It pretty much doesn't matter what the "outer" command is, when the inner command is supposed to produce a list of filenames to operate on it's going to be subject to word splitting and processing in ways that are not meant to apply to filenames, so you're going to have a really hard time making it work right. You're usually better of just finding another approach than trying to make it work with all filenames – Eric Renouf Oct 24 '16 at 18:26

3 Answers3

2

Maybe in two lines

IFS=$'\n' DIRS=( $(produces_dir_names --options...) ) 
find "${DIRS[@]}" -find-options...

Example:

$ mkdir -p "/tmp/test/a b/foo" "/tmp/test/x y/bar"

$ IFS=$'\n' DIRS=( $(printf "/tmp/test/a b\n/tmp/test/x y\n") )
$ find "${DIRS[@]}" -mindepth 1
/tmp/test/a b/foo
/tmp/test/x y/bar

But in general this is no good style. For example you will be in trouble if your DIRS contain newlines. Better fix your "produces_dir_names" to print null byte terminated strings. Regarding my example this would be something like:

$ printf "/tmp/test/a b\0/tmp/test/x y\0" | xargs -0 -I '{}' find '{}' -mindepth 1
/tmp/test/a b/foo
/tmp/test/x y/bar

If you can't fix "produces_dir_names", regarding my last comment, the most general solution would look like this:

produces_dir_names --options... | tr '\n' '\0' | xargs -0  -I '{}' find '{}' -find-options...

Still problems with "newlines" unless you fix "produces_dir_names" to avoid tr.

rudimeier
  • 10,315
  • I suspect this will have trouble with any filenames that contain \n, which is a legal character in paths/filenames. – Eric Renouf Oct 24 '16 at 18:16
  • @EricRenouf But this is what the OP wants. I'll add a comment. – rudimeier Oct 24 '16 at 18:26
  • IFS is being set globally, not just locally to the assignment to DIRS. I would use readarray -t DIRS < <(produces_dir_names --options ...) (or the equivalent while loop prior to bash 4) instead. – chepner Oct 25 '16 at 13:44
  • In bash IFS is only changed for that commandline. Compare VAR=a; VAR="b" true; echo "$VAR" and .VAR=a; VAR="b"; echo "$VAR". readarray looks good! – rudimeier Oct 25 '16 at 13:56
  • This works for my current setup. And yes I could manipulate the producing command so it would delimit the path with \0. But since I also, somewhat control the input (sorry for not mentioning that) I did not bother. But you're right that I should take \0 delimiting into consideration. And since every xargs command I checked(gnu, freebsd and netbsd) would understand -0 i'm satisfied with this. – ness Oct 27 '16 at 19:22
1

rudimeier’s answer is good — specifically, the part about modifying produces_dir_names to print null-terminated strings — but it might not be obvious from his answer that it executes find once for each directory.  If this is good enough, than fine.  But, of course, it is possible to invoke find with multiple starting points; e.g.,

find  dir1 dir2 dir3  -find-options...

and it appears from the question that that’s what you want.  This can be done as follows:

printf "a\0a b\0a b c" | xargs -0 sh -c 'find "$@" -find-options...' sh

This causes xargs to invoke sh -c once, with all the directory names appended to the command.  The shell will then expand "$@" to a list of those directory names.

P.S. If produces_dir_names lists too many directory names to put on one command line, then xargs will be forced to spawn a few commands.  Use xargs --verbose to see what commands xargs is spawning.

1

Just to clear up the mystery of the error messages you are getting:

find: ‘"a"’: No such file or directory
find: ‘"a’: No such file or directory
find: ‘b"’: No such file or directory
find: ‘"a’: No such file or directory
find: ‘b’: No such file or directory
find: ‘c"’: No such file or directory

The answer is that Bash quote removal doesn't remove quotes that resulted from command substitution.

From LESS='+/^ *Quote Removal' man bash

Quote Removal
    After the preceding expansions, all unquoted occurrences of the charac-
    ters  \,  ', and " that did not result from one of the above expansions
    are removed.

The "preceding expansions" referenced include:

EXPANSION
   Brace Expansion
   Tilde Expansion
   Parameter Expansion
   Command Substitution
   Arithmetic Expansion
   Process Substitution
   Word Splitting
   Pathname Expansion
   Quote Removal
Wildcard
  • 36,499
  • 1
    Arguably, this is obvious.  If I say a="Don't say I didn't warn you."; echo "$a", the output is Don't say I didn't warn you.; the single quotes (a.k.a. apostrophes) are not removed or even recognized. – G-Man Says 'Reinstate Monica' Oct 25 '16 at 01:47
  • @G-Man, good thing I'm not arguing, then. Just clarifying. :) Direct quote from the Original Poster: "I thought quoting them in the output of the producing command...would be enough." – Wildcard Oct 25 '16 at 01:50
  • 1
    I don't really care about the qoutes more about the word spliting since this is the only thing that matters to the invokation of 'find'. – ness Oct 27 '16 at 19:08