Why xargs does not process the last argument?

Question

Observe:

mark@L-R910LPKW:~$ echo a b | xargs -d' ' -I{} bash -c 'echo {} 1'
a 1
b
bash: line 2: 1: command not found
mark@L-R910LPKW:~$

What is going on?

Answer 1

What went wrong

b appears in the output, so it was processed, just not the way you expected.

As a first step, ask bash to tell you what it sees: pass the -x option to enable its traces.

$ echo a b | xargs -d' ' -I{} bash -x -c 'echo {} 1'
+ echo a 1
a 1
+ echo b
b
+ 1
bash: line 2: 1: command not found

So bash was first invoked code line echo a 1 as expected. But the next line is echo b and not echo b 1 as you presumably expected. And there's an extra line with 1. Why?

Well, you told xargs to split at spaces. And you passed the input a b where is a newline. So xargs saw that the input contains two fragments: a and b . As instructed, xargs called bash for each fragment: first to execute echo a 1, then to execute echo b 1.

How to do it right

Some versions of find or xargs let you embed {} in a shell fragment. This is almost always a bad idea, which will break with some file names or other data, and is often a security vulnerability. Pass the data as a separate argument.

Is it possible to use `find -exec sh -c` safely?

Answer 2

As Gilles mentioned in their answer, the -d ' ' option to GNU xargs makes it consider the space, and only the space, as a separator, leaving the newline as part of the data, here embedded in your shell code like the letters themselves. That might not be what you want. (Likely the most common use for -d is -d '\n' to tell it to just use lines as-is, without any of the further processing that e.g. -L does.)

If, instead, you want to get each whitespace-separated word as a separate item, one option is to utilize the default behaviour where it splits items on whitespace, so this works directly:

$ echo a b | xargs -n1 bash -c 'echo "$1" 1' sh
a 1
b 1

Just note that it also processes quotes and backslashes (in a way slightly different from the shell), so that's not the same as using whitespace only as separator. The input a "b c" would produce the items a and b c.

Or, you could use -d with tr to preprocess the input and fold all characters you want to use as separators to one single character:

$ printf 'a b\nc\n' | tr ' ' '\n' | xargs -d'\n' -n1 bash -c 'echo "$1" 1' sh
a 1
b 1
c 1

However, -d isn't standard, and I think only implemented in GNU xargs, so you might want to instead use -0 which has wider support:

$ printf 'a b\nc\n' | tr ' \n' '\0' | xargs -0 -n1 bash -c 'echo "$1" 1' sh
a 1
b 1
c 1

In any case, avoid embedding the value directly into the shell snippet, as that's unsafe and impossible to get to work correctly with arbitrary values.