4

Why does

for i in {1..5}; do x="${i}" echo "$x"; done

not output

1
2
3
4
5

?

What is the right way to do this?

(Tested for i in {1..5}; do x=$(i) echo "$x"; done

-bash: i: command not found

and others)

ilkkachu
  • 138,973
  • 3
    $(i) (command substitution) is not the same thing as ${i} (variable expansion) – steeldriver Nov 20 '17 at 00:51
  • @steeldriver Thank you very much for this hint! Now one can understand the error message. –  Nov 20 '17 at 01:07
  • Thank you all for your great answers! The problem was that I forgot a semicolon. –  Nov 20 '17 at 17:25

3 Answers3

11

Answering your question as asked

Why does for i in {1..5}; do x="${i}" echo "$x"; done not output 1, 2, 3, 4, 5?

The reason is to do with the order of which operations are evaluated during execution. Look at this command

x="${i}" echo "$x"

What this does is

  1. Substitute variables with their values
  2. Assign a temporary value to variable x
  3. Execute the command

So you get

  1. x=1 echo "" (or x=2 echo "", etc.)
  2. For the duration of this command, x is set to the value 1
  3. The command is executed: echo ""

It's probable you meant this command to be two instructions: assign a value to x, output its value. But in shell syntax what you wrote was perfectly legal code so the shell executed it without objection.

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
  • +1 The better answer, because it explains what happens without the semicolon! – Philippos Nov 20 '17 at 08:10
  • 2
    @roaima: Your explanation is not entirely correct. The crucial point in step 2 is that the variable x is placed in the execution environment (i.e. child process) for the command to be executed and hence not visible to the parent process. This can be illustrated by changing the example to for i in {1..5}; do x="${i}" printenv x; done, which outputs the expected 1,2,...,5. – user1934428 Nov 20 '17 at 10:25
  • @user1934428 that's a side-effect of what I have described. Try this as a counter example - x=apple; for i in {1..5}; do x="${i}" echo "$x"; done – Chris Davies Nov 20 '17 at 11:18
  • @Philippos It's possible that I may have misunderstood the source of confusion in the OP. I assumed that they were confused about the for-loop syntax rather than the command itself. I assumed that if the question were really about the command itself then the OP wouldn't have included the for-loop, since it would have been superfluous. But I was inspired by your comment to update my solution. – igal Nov 20 '17 at 13:09
  • @user1934428 I think that roaima's explanation is correct. I updated my solution to included references to the manual section that explicitly states how variable assignments in commands are evaluated. – igal Nov 20 '17 at 13:11
  • 2
    @roaima : This is not a counter-example; in contrary. If we replace in your counter-example the echo $x by printenv x, we get 1,2,3.... again. The x=... in front of a command does set the execution environment for the subsequent command, but the "$x" is, of course, still evaluated in the current environment and hence does not "see" the change of x. – user1934428 Nov 20 '17 at 16:30
  • @user1934428 your printenv suggestion doesn't try to evaluate the value of a variable that's being set for the duration of the command. In that respect it's irrelevant, and the (valid) effect of copying that variable to the environment is a complete side-issue. In the context of the question, and the way that I answered it, to which particular part do you object? – Chris Davies Nov 20 '17 at 19:45
  • 2
    Hmm, could it be that we are meaning the same, but expressing it in different way? Let me make my point clear: In sh/bash/ksh/zsh, a statement of the form X=U V, where V is an external program, spawns a child process, where V is started. This child process inherits the environment of the parent (i.e. the current shell, with the difference that the variable X is set to the value U. This means X becomes an environment variable in the child (and only then). If I have instead X=U; V, the variable X is set in the current process, and doesn't necessarily show up in the environment of V. – user1934428 Nov 21 '17 at 06:07
  • 1
    @roaima: This means that if you have a statement X=1 foo $X, foo does not get passed the value 1 as argument (but sees the environment variable X to be set to 1). We can see this nicely in this example: export A=5; export B=6; C=A; C=B printenv $C where 5 is printed, because $C evaluates to A and not to B. From your explanation, I had the impression that you wanted to say that in such an example, C=B would temporarily set C in the current shell, and I didn't find this accurate; but maybe I misunderstood you in this point? – user1934428 Nov 21 '17 at 06:10
8

You asked for answers to two questions:

  1. You asked for an explanation of why your current code does not produce the expected output.

  2. You asked for the correct way to write your code so that it does produce the expected output.

Looking at your code, I can see two likely explanations for why you wrote your code the way you did:

  1. There might be some slight confusion about the syntax of a for-loop.

  2. There might be some slight confusion about the order of evaluation in what's called a simple command.

for-loop syntax

In the first case, I would say that you're missing a semicolon after your variable assignment. If you want to write your for-loop on a single line then you need to put a semi-colon after each command in the loop body. Try this instead:

for i in {1..5}; do x="${i}"; echo "$x"; done

Another alternative would be to write the for-loop using the multi-line syntax with newlines in place of the semicolons:

for i in {1..5}
do
x="${i}"
echo "${x}"
done

You can also mix-and-match semicolons and newlines, e.g.:

for i in {1..5}; do
x="${i}"; echo "${x}"
done

evaluation of simple commands

In the second case, I would say that you probably had assumed that the variable assignment in the prologue of the command (i.e. the x="$i" assignment) occurs before the variable expansion in the body of the command (i.e. the expansion of ${x} in echo "${x}"). But this is actually not the case. To verify this we can refer to the page on simple command expansion in the Bash Manual or to the subsection on simple commands in the Posix Specification. Both of these references include the following passage:

A "simple command" is a sequence of optional variable assignments and redirections, in any sequence, optionally followed by words and redirections, terminated by a control operator.

When a given simple command is required to be executed (that is, when any conditional construct such as an AND-OR list or a case statement has not bypassed the simple command), the following expansions, assignments, and redirections shall all be performed from the beginning of the command text to the end:

  1. The words that are recognized as variable assignments or redirections according to Shell Grammar Rules are saved for processing in steps 3 and 4.

  2. The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.

  3. Redirections shall be performed as described in Redirection.

  4. Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.

Notice that step 2 is where the variable expansion in the command occurs, but step 1 tells us that the variable assignments are saved until steps 3 and 4. It follows that the expression echo "${x}" is expanded to echo "" before the assignment x="${i}" takes place. This explains why you were getting empty output.

For further discussion on this topic see the following posts:

igal
  • 9,886
  • 1
    All that verbiage, when the answer is "you forget a semicolon"... – RonJohn Nov 20 '17 at 15:25
  • @RonJohn That's what I had originally thought too, but apparently more people thought that the real issue was related to the command parsing. See the comments after the solution posted by roaima. – igal Nov 20 '17 at 15:30
  • 1
    @RonJohn Given the number of people who think that precommand environment modifiers should apply to echo, "you forgot a semicolon" isn't necessarily the most helpful answer. – chepner Nov 20 '17 at 19:59
  • @igal semicolons are part of the language, and required to correctly parse when "everything on a single line". Thus, the forgotten semicolon made it parse incorrectly (if by "incorrectly" you mean "didn't print 1 2 3 4 5". – RonJohn Nov 20 '17 at 20:13
  • @igal also, I made the same mistake earlier this morning... :( – RonJohn Nov 20 '17 at 20:14
  • @RonJohn Did you intend those comments for chepner? – igal Nov 20 '17 at 20:16
  • @chepner I think the main argument for addressing the semicolons rather than the precommand environment-variable modifiers is the presence of the for loop. On the other hand the title of the question focuses it on the modifiers. It's often hard to tell exactly where the confusion is. And sometimes it isn't anywhere. I guess that's part of the nature of confusion. – igal Nov 20 '17 at 20:19
  • @igal no, for you. – RonJohn Nov 20 '17 at 20:21
  • @RonJohn Then I don't understand why you're telling me this. My solution did address the semicolon. – igal Nov 20 '17 at 20:25
  • @igal it was a friendly comment, not a criticism. – RonJohn Nov 20 '17 at 20:30
  • @RonJohn No worries. Not offended, just confused. Thanks for the feedback. – igal Nov 20 '17 at 20:47
  • "apparently more people thought that the real issue was related to the command parsing" I answered the question exactly as asked, and then as an afterthought suggested that a missing semicolon might solve the question (that had not been asked). You answered the question that the OP should have asked :-) – Chris Davies Nov 20 '17 at 23:52
  • 1
    @roaima I think I also answered the question exactly as asked. There were actually two questions. The OP asked why the given example code wasn't working and also for the correct way to modify his code to produce the given output. The first question is inherently ambiguous. There is no one reason why the code in the OP didn't produced the expected output. I also focused more on the second question and on the problem description in the body of the post - basically ignoring the title. Your solution is definitely a more direct answer of the question as it was expressed in the title. – igal Nov 21 '17 at 02:41
  • @roaima Just so there's no confusion, I thought you gave a very good solution and I upvoted it immediately. – igal Nov 21 '17 at 02:42
1

In zsh, using a third ${var::=value} form of the ${var=value}, ${var:=value} Bourne operators where the assignment is unconditional (as opposed to only if var is unset/empty).

for i in {1..5}; do echo ${x::=$i}; done

Or:

for i ({1..5}) echo ${x::=$i}

In bash:

set -o posix # so the value of x remains after eval returns
for i in {1..5}; do x=$i eval 'echo "$x"'; done

That is, you need $x to have been set at the time the code that contains its expansion is evaluated.

For decimal integer values like here, you can also do:

for i in {1..5}; do echo "$((x = i))"; done

Or you can always use the ${var:=value} Bourne operator after having set x to the empty string.

for i in {1..5}; do x=; echo "${x:=$i}"; done