How to retain newlines on xargs -I {}

Question

The following extracts a few dozens of lines into a variable, but they are somehow all placed together into a single line. That is, they loose the original newline character:

ALL_FOUND_LINES=$(find "$TEMP" -type f -name "debug.log*" | xargs -I {} grep -F "STARTING HOST " {})

Is there a way to retain the newline character in a statement like the one above?

Answer 1

If you are testing this with

echo $ALL_FOUND_LINES

then I'm not surprised that all newlines are gone since the shell would split the value in $ALL_FOUND_LINES on spaces, tabs and newlines (by default) into words (and then further expand each word by means of filename generation (globbing)). It does this since the expansion is unquoted. The echo utility then gets a list of words that it prints on a single line.

A better test would be

printf '%s\n' "$ALL_FOUND_LINES"

Note the quoting of the variable expansion. For the choice of printf over echo, see Why is printf better than echo?.

---

Your command can be improved into

find "$TEMP" -type f -name 'debug.log*' -exec grep -h -F 'STARTING HOST ' {} +

Here, instead of passing the filenames off to xargs, we let find execute grep directly on as many debug.log* files as possibly at a time. Note that getting rid of xargs does not solve your newlines problem, as xargs had nothing to do with this. This would speed things up a bit since it involves fewer invocations of grep.

See also Understanding the -exec option of `find`.

If you need to do something with each of the found lines, then you may loop over them like this:

find "$TEMP" -type f -name 'debug.log*' -exec grep -h -F 'STARTING HOST ' {} + |
while IFS= read -r line; do
    # use "$line" here (with quotes)
done

(or replace the while loop with whatever other processing step that you need to carry out). So there is never a need to store all of the data in a variable as a newline-delimited string.

See also Understanding "IFS= read -r line"

Answer 2

man xargs

   -L max-lines
          Use at most max-lines nonblank input  lines  per  command  line.
          Trailing blanks cause an input line to be logically continued on
          the next input line.  Implies -x.
-l[max-lines], --max-lines[=max-lines]
          Synonym for the -L option.  Unlike -L, the max-lines argument is
          optional.   If  max-lines  is not specified, it defaults to one.
          The -l option is deprecated since the POSIX  standard  specifies
          -L instead.