Maintain filenames as separate arguments in successive commands

Question

A disclaimer: This isn't related to one specific command like ls or grep, although I use them in the examples, this is more a bash thing.

Let's say I want to see the file sizes, timestamps, and permissions/owner/group on a set of files (ls -l) that all contain "a certain word". Typically I'd do this:

ls -l $( grep -l 'a certain word' * )

And that works like magic until any of the file names contain a space, like a file named "file name". Then ls can't find what it thinks are two file names, "file" and "name".

Bash scripts have a familiar way of solving a variant of this problem, if you pass file names in on the command line to a script, they are referenced as ${1}, ${2}, etc. And if you wrap them in quotes, i.e. "${1}", "${2}", then if they have spaces you don't have a problem and you can even process all the file names at once by using "${@}" which will correctly list each of the parameters, and keep them intact even if they have spaces - I just haven't found such a mechanism for the grep/ls style problem I list above. If you wrap the expansion command, $( ) with quotes it treats all the files as one big parameter for the ls.

I've usually worked around it with something like grep -l .... | while read file; do ls -l "${file}"; done, but I really shouldn't have to do that. And doing that still doesn't allow things like sorting the files by date/size/etc (i.e. ls -ltr $( grep -l ....))

I keep thinking bash has a solution for this and I just haven't came across it yet and this problem only surfaces once in a while and I usually work around it, but it's now bugged me enough I think there must be a way.

And yes, I've seen the following, they are either questions solved by the specific tool being used (like tar), or the answers are "sorry, you can't get to there from here" because their question is command-specific. I'm asking for handling any space-embedded file name(s) given as results to be passed as parameters to another command, not more while or find magic to solve this issue. I would rather this question sit unanswered (a testament that there isn't a solution) rather than have any of the not-the-same-thing questions offered:

• Using files that have spaces in their name in pipes
• Handling files with whitespace in their names
• Create directories from a list of files with spaces in name

Answer 1

Hmm. @don_crissti already gave the answer for grep in a comment. But since you said it wasn't really about grep or ls, I'm going to rewrite the command in question to not use those commands.

What I think you want is:

do-something-with $(produce-list-of-files)

where the output of one command should be dropped in as command line parameters to another command. There just happens to be a utility just for that, it is called xargs (man page).

If the file names are "nice", we could do just

produce-list-of-files | xargs do-something-with

If the file names can contain spaces but are separated by newlines, we have to tell xargs to not split on any whitespace, but only newlines:

produce-list-of-files | xargs -d '\n' do-something-with

If the file names can contain newlines too, the list has to be separated by NULs ('\0', byte with value zero), and we need an xargs that supports it. At least some versions of various utilities support listing files separated by NULs instead of newlines, there's at least find -print0, sort -z and grep -Z in the GNU versions of those tools. In xargs the flag is --null or -0. So:

produce-list-of-files -0 | xargs -0 do-something-with

An example run with cat and, well, ls -l:

$ touch "abba acdc" "foo bar" $'new\nline'
$ echo -en "abba acdc\0foo bar\0new\nline\0" > list
$ cat list | xargs -0 ls -l
-rw-r--r-- 1 itvirta itvirta 0 Aug  2 01:23 abba acdc
-rw-r--r-- 1 itvirta itvirta 0 Aug  2 01:23 foo bar
-rw-r--r-- 1 itvirta itvirta 0 Aug  2 01:23 new?line

Answer 2

If you have a command that produces a newline-separated list of items such as filenames, and the items can themselves contain newlines, then you can't get there from here.

Some commands on some systems can produce or manage lists delimited by null bytes instead of newlines, e.g. GNU grep (grep -Z), GNU sed (sed -z), GNU awk (gawk RS='\0' ORS='\0'), GNU and BSD find (find -print0), … You can then use xargs -0 to execute a command on the items of the null-delimited list.

find … -print0 | sed -z … | xargs -0 frob …

If you're willing to assume that the file names don't contain newlines, then you can use the split+glob operator, i.e. unquoted expansion, to split the newline-delimited list. You need to turn off gobbing with set -f and to set the delimiter characters for splitting (IFS variable) to a newline.

IFS='
'; set -f
ls -l -- $(grep -l 'a certain word' *)
set +f; unset IFS

The last line restores the default settings. Accurately saving and restoring the settings for set -f and IFS can be done, but it's a pain.

If you need to save the output for later processing, you can either split it at the point of definition and store the data in an array, or split it at the point of use and store the data in a string. With an array (which requires ksh, bash or zsh):

IFS='
'; set -f
files=($(grep -l 'a certain word' *))
set +f; unset IFS
…
ls -l -- "${files[@]}"

With a string:

files=$(grep -l 'a certain word' *)
IFS='
'; set -f
ls -l -- $files
set +f; unset IFS

See also Why does my shell script choke on whitespace or other special characters?