28

I'd like to be able to use xargs to execute multiple parameters in different parts of a command.

For example, the following:

echo {1..8} | xargs -n2 | xargs -I v1 -I v2 echo the number v1 comes before v2

I would hope that it would return

the number 1 comes before 2
the number 3 comes before 4 

... etc

Is this achievable? I suspect that my multiple use of -I is incorrect.

3 Answers3

54

I believe that you can’t use -I that way.  But you can get the effect / behavior you want by saying:

echo {1..8} | xargs -n2 sh -c 'echo "the number $1 comes before $2"' sh

This, essentially, creates an ad hoc one-line shell script, which xargs executes via sh -c.  The two values that xargs parses out of the input are passed to this “script”.  The shell then assigns those values to $1 and $2, which you can then reference in the “script”.

  • 7
    Thank you - that's great. I've been reading the man for sh though and am struggling understanding what the second call to sh does and, by extension, why its omission yields 'half' the result. – Damien Sawyer Aug 19 '17 at 01:28
  • 10
    @DamienSawyer There is no second call to sh. The trailing sh at the end is what is put into $0. $0 is usually what holds the name of the interpreter or the script. – Kusalananda May 19 '18 at 09:53
  • I just replaced trailing sh with echo $sh which it works. So final sh functions as a placeholder – kenn Mar 22 '20 at 21:04
  • Well, that’s an oversimplification of what Kusalananda said. – Scott - Слава Україні Mar 23 '20 at 05:57
  • 5
    I didn't get the explanation about trailing sh but running xargs --verbose made it clearer: the trailing value passed is fed into sh -c execution as $0 env variable. So if your script doesn't use $0, you can pass whatever. It's just customary to pass sh. And it's needed so that xargs can output the other params (you need $0 first to output $1, $2... replacements). Example: sh -c 'echo $1/$2/$0' sh A B writes A/B/sh (more about $0: https://bash.cyberciti.biz/guide/$0) – jakub.g Nov 02 '20 at 20:38
  • It doesn't work in a script and if you want to use double quotes to have access to variables defined before in the script because $0..$N are interpreted as script parameters. For instance echo "bar" | xargs -n 1 sh -c "echo \"${FOO}: $0\"" – Danny Lo Apr 22 '21 at 08:58
  • @Scott why in your answer xargs does not need to end with \;? – Porcupine Jul 26 '21 at 04:02
  • @Porcupine: xargs commands never need to end with \;. – Scott - Слава Україні Jul 26 '21 at 04:54
  • @Scott When xargs is used with find and bash shell is invoked in xargs pipe. It ends with \;. Could you please tell why is it needed here but not in your answer? – Porcupine Jul 26 '21 at 17:51
  • @Porcupine: Your question contains its own answer. The arguments to the -exec action of find end with \; (or +) — and my answer doesn’t use find. Bash doesn’t really have anything to do with it; in principle, you can run xargs and/or find without using a shell. … … … … … … … … … … … … … … … … … … … … … … Those of us who come here and answer questions do so because we want to help people. But we prefer not to deal with questions that could easily be answered by looking at documentation. – Scott - Слава Україні Jul 26 '21 at 19:18
  • After all this discussion, I'm here to praise such a beautiful answer. I'm pretty sure that I will use this one killer command for everything. Thx, stackexchange community. I would like not to have my compliments erased by some moderator, but I was feeling the urge to compliment everyone here. <3 – R. W. Prado Sep 24 '21 at 07:19
8

In the specific case of printf, you could always do:

echo {1..8} | xargs printf 'the number %s comes before %s\n'

because printf has an intrinsic xargs-like ability to execute multiple times if it is given more arguments than it needs for a single invocation.  Though that has little advantage over

printf 'the number %s comes before %s\n' {1..8}

And for large lists, the simple xargs command could result in xargs running several instances of printf, some of which might have odd numbers of arguments. You could pass -n 1000 to xargs to guard against that, where 1000 is an even number that should be small enough so as not to reach the arg list too long limit and large enough to avoid running so many printfs.

Note that xargs would call, not your shell's builtin printf, but the external printf, with each invocation in a separate new process.

Also note that for an empty input, except on some BSDs, it would still run printf once with no argument. GNU xargs and compatible have a -r (or --no-run-if-empty) option to avoid that.

To be clear, this simple answer is specific to your printf example, and would not work in the general case where you must pass two parameters at a time to your command (as would be the case for diff, for example).  To solve the general problem with zsh, you can use:

for i j ({1..8}) echo "the number $i comes before $j"
  • (1) IMHO, the zsh part of the above is a real answer, and the xargs-with-no-options part is just an oddity.  In such a case, I would consider putting the real answer first.  (2) I assume you have a reason for not using quotes in your zsh command.  If so, you might want to state it, lest people read the above and start to think that quotes aren’t important.  (Or simply include them, since they don’t hurt.) – Scott - Слава Україні Jul 23 '18 at 18:50
  • This is really cool. I landed here looking for something similar where I wasn't sure ahead of time how many args I was going to receive that had to be put into the command line of the next arg. did something like awk -F'\t' '/match/{printf $1"\0"$2"\0"}' | xargs -0 printf -- '-args1 "%s" -args2 "%s"' | xargs mycommand – keithpjolley Apr 19 '20 at 18:53
  • @keithpjolley, your usage of -0 on the first xargs to make it more reliable is defeated by you not using it on the second. Also, the first argument to printf (both the standalone utility and awk's printf() function) is the format, you shouldn't use variables there. Here, you could do awk -F'\t' '/match/{printf "-args1\0%s\0-args2\0%s\0", $1, $2}' | xargs -n400 -r0 mycommand. Again, -n400 needed to ensure xargs passes a number of arguments that is a multiple of 4. (note that not all awk implementations support using \0 like that here). – Stéphane Chazelas Apr 20 '20 at 06:41
  • didn't use -0 to make it more reliable, used it to have it do what i wanted to do, which it does. – keithpjolley Apr 20 '20 at 15:11
4

try this:

echo {1..8} |xargs -n 2 bash -c 'echo "the number $0 comes before $1"'
md11235
  • 41
  • 2
    This would work in this case, but it's better to run sh -c script with sh (or whatever) in $0. Note that $0 is not included in $@ if this was to be used by the sh -c script, for example if the script was echo "the two numbers were $@" – Kusalananda May 19 '18 at 09:54