Why doesn't bash expand this variable when I prefix a command with a "one time variable assignment"

Question

If I run this bash command and prefix the statement, so that the variable fruit should exist, but only for the duration of this command:

$ fruit=apple echo $fruit

$

The result is an empty line. why?

To quote a comment from wildcard on this question:

parameter expansion is done by the shell, and the "fruit" variable is not a shell variable; it's only an environment variable within the environment of the "echo" command

An environment variable is still a variable, so surely this should still be available to the echo command?

Answer 1

The issue is the current shell is expanding the variable too early; it is not set in its context so the echo command doesn't get any argument, i.e. the commands ends up being:

$ fruit=apple echo

Here is a workaround where the variable doesn't get expanded too early because of the single quotes:

$ fruit=apple sh -c 'echo $fruit'

Alternatively, you can also use a one line shell script which demonstrates the fruit variable is correctly passed to the executed command:

$ cat /tmp/echof
echo $fruit
$ /tmp/echof

$ fruit=apple /tmp/echof
apple
$ echo $fruit

$

---

_{A few comments as this question has sparkled some unexpected controversy and discussions:}

• _{The fact the variable fruit is already exported or not doesn't affect the behavior, what matters is what the variable value is at the precise moment the shell is expanding it.}

$ export fruit=banana
$ fruit=apple echo $fruit
banana

• _{The fact the echo command is a builtin one doesn't affect the OP issue. However, there are cases where using builtins or shell functions with this syntax have unexpected side effects, e.g.:}

$ export fruit=banana
$ fruit=apple eval 'echo $fruit'
apple
$ echo $fruit
apple

• _{While there is a similarity between the question asked here and that one, this is not exactly the same issue. With that other question, the temporary IFS variable value is not yet available when the shell word split another variable $var while here, the temporary fruit variable value is not yet available when the shell expands the same variable.}

• _{There is also that other question where the OP is asking about the significance of the syntax used and more precisely is asking "why does this work?". Here the OP is aware of the significance but reports an unexpected behavior and asks about its cause, i.e. "why doesn't this work?". Ok, after reading more closely the poor screenshot posted on the other question, the same situation is indeed described there (BAZ=jake echo $BAZ) so yes, after all this is a duplicate}

Answer 2

To understand this properly, let's first differentiate shell variables from environment variables.

Environment variables are a property that ALL processes have, whether they utilize them internally or not. Even sleep 10 when it's running has environment variables. Just as all processes have a PID (process identifier), a current working directory (cwd), a PPID (parent PID), an argument list (even if empty)—and so on. So too do all processes have what's called an "environment," which is inherited from the parent process when it forks.

From a utility author perspective (someone who writes code in C), processes have the ability to set, unset or change environment variables. However, from a script author perspective, most tools do not offer that facility to their users. Instead, you use your shell to alter the process environment which is then inherited when the command (external binary) you called is executed. (The environment of your shell itself can be modified and the modification inherited, or you can direct your shell to make the modification after forking but prior to executing the command you called. Either way, the environment is inherited. We'll look at both approaches.)

Shell variables are another thing. Although within the shell they behave in the same way, the difference is that mere "shell variables" do not alter or influence the behavior of commands that you call from your shell. In proper terminology the distinction would actually be stated a little bit differently; exported shell variables will become part of the environment of tools which you call, whereas shell variables which are not exported will not. However, I find it more helpful for communication to refer to shell variables which are not exported as "shell variables" and shell variables which are exported as "environment variables" because they are environment variables from the perspective of processes forked from the shell.

---

That's a lot of text. Let's look at some examples and describe what's happening:

$ somevar=myfile
$ ls -l "$somevar"
-rw-r--r--  1 Myname  staff  0 May 29 19:12 myfile
$ 

In this example, somevar is just a shell variable, nothing special about it. Shell parameter expansion (see LESS='+/Parameter Expansion' man bash) occurs before the ls executable is actually loaded ("exec"ed) and the ls command (process) never even sees the string "dollar sign s-o-m-e-v-a-r". It only sees the string "m-y-f-i-l-e", interprets that as the path to a file in the current working directory, and gets and prints information about it.

If we run export somevar before the ls command, then the fact that somevar=myfile will appear in the environment of the ls process, but that won't affect anything because the ls command won't do anything with this variable. To see an effect of an environment variable, we have to choose an environment variable that the process we're calling will actually check for and do something with.

---

bc: Basic Calculator

There may be a better example, but this is one I came up with that's not too complicated. First you should know that bc is basic calculator, and processes and computes mathematical expressions. (After processing the contents of any input files, it processes its standard input. I'm not going to use its standard input for my examples; I'll just press Ctrl-D, which will not show in the text snippets below. Also I'm using -q to suppress an introductory message on every invocation.)

The environment variable I will illustrate is described in man bc:

   BC_ENV_ARGS
      This is another mechanism to get arguments to bc.  The format is
      the  same  as  the  command line arguments.  These arguments are
      processed first, so any files listed in  the  environment  argu-
      ments  are  processed  before  any  command line argument files.
      This allows the user to set up "standard" options and  files  to
      be  processed at every invocation of bc.  The files in the envi-
      ronment variables would typically contain  function  definitions
      for functions the user wants defined every time bc is run.

Here goes:

$ cat file1
5*5
$ bc -q file1
25
$ cat file2
6*7
8+9+10
$ bc -q file2
42
27
$ bc -q file1 file2
25
42
27
$

This is just to show how bc works. In each of these instances I had to press Ctrl-D to signal "end of input" to bc.

Now, let's pass an environment variable directly to bc:

$ BC_ENV_ARGS=file1 bc -q file2
25
42
27
$ echo "$BC_ENV_ARGS"

$ bc -q file2
42
27
$

Notice that what we put in that variable is not visible later from an echo command. By putting the assignment as part of the same command (with no semicolon, either), we put that variable assignment as part of the environment of bc—it left the shell itself that we are running unaffected.

Now let's set BC_ENV_ARGS as a shell variable:

$ BC_ENV_ARGS=file1
$ echo "$BC_ENV_ARGS"
file1
$ bc -q file2
42
27
$

Here you can see that our echo command can see the contents, but it's not part of the environment of bc so bc can't do anything special with it.

Of course if we put the variable itself in bc's argument list we would see something:

$ bc -q "$BC_ENV_ARGS"
25
$ 

But here, it is the shell expanding the variable, and then file1 is what actually appears in bc's argument list. So this is still using it as a shell variable, not an environment variable.

Now let's "export" this variable so it's both a shell variable AND an environment variable:

$ export BC_ENV_ARGS
$ echo "$BC_ENV_ARGS"
file1
$ bc -q file2
25
42
27
$

And here you can see that file1 is processed before file2, even though not mentioned on the command line here. It's part of the shell's environment and becomes part of bc's environment when you run that process, so the value of this environment variable is inherited and affects how bc operates.

We can still override this on a per-command basis, even override it to an empty value:

$ BC_ENV_ARGS= bc -q file2
42
27
$ echo "$BC_ENV_ARGS"
file1
$ bc -q file2
25 
42
27
$ 

But as you can see, the variable remains set and exported in our shell, visible both to the shell itself and to any later bc commands that don't override the value. It will remain this way unless we "unexport" or "unset" it. I'll do the latter:

$ unset BC_ENV_ARGS
$ echo "$BC_ENV_ARGS"

$ bc -q file2
42
27
$ 

---

Another example, involving spawning another shell:

Type the following commands one after another into your shell and consider the results. See if you can predict the results before running them.

# fruit is not set
echo "$fruit"
sh -c 'echo "$fruit"'
# fruit is set as a shell variable in the current shell only
fruit=apple
echo "$fruit"
sh -c 'echo "$fruit"'
sh -c "echo $fruit" ### NOT advised for use in scripts, for illustration only
# fruit is exported, so it's accessible in current AND new processes
export fruit
echo "$fruit"
sh -c 'echo "$fruit"'
echo '$fruit' ### I threw this in to make sure you're not confused on quoting
# fruit is unset again
unset fruit
echo "$fruit"
sh -c 'echo "$fruit"'
# setting fruit directly in environment of single command but NOT in current shell
fruit=apple sh -c 'echo "$fruit"'
echo "$fruit"
fruit=apple echo "$fruit"
# showing current shell is unaffected by directly setting env of single command
fruit=cherry
echo "$fruit"
fruit=apricot sh -c 'echo "$fruit"'
echo "$fruit"
sh -c 'echo "$fruit"'

And one last one for extra trickiness: Can you predict the output of the following commands run in sequence? :)

fruit=banana
fruit=orange sh -c 'fruit=lemon echo "$fruit"; echo "$fruit"; export fruit=peach'
echo "$fruit"

---

Please mention in the comments any clarifications desired; I'm sure this could use a few. But it should hopefully be helpful even just as it is.

Answer 3

Because expansions on the command line take place before variable assignments. The standard says so:

When a given simple command is required to be executed, the following ... shall all be performed:

1. The words that are recognized as variable assignments [...] are saved for processing in steps 3 and 4.

2. The words that are not variable assignments or redirections shall be expanded. [...]

3. Redirections shall be performed as described in Redirection.

4. Each variable assignment shall be expanded [...] prior to assigning the value.

Note the order: in the first step, assignments are only saved, then the other words are expanded, and only in the end do variable assignments take place.

Of course, it's highly likely that standard says so only because it's always been like that, and they just codified the existing behaviour. It doesn't say anything about the history or the reasoning behind it. The shell has to identify words that look like assignments at some point (if only to not take them as part of the command), so I suppose it could work in the other order instead, assign variables first, then expand anything on the command line. (or just do it left-to-right...)