Bash: Assign output of pipe to a variable

Question

I am trying to get the output of a pipe into a variable. I tried the following things:

echo foo | myvar=$(</dev/stdin)
echo foo | myvar=$(cat)
echo foo | myvar=$(tee)

But $myvar is empty.

I don’t want to do:

myvar=$(echo foo)

Because I don’t want to spawn a subshell.

Any ideas?

Edit: I don’t want to spawn a subshell because the command before the pipe needs to edit global variables, which it can’t do in a subshell. Can it? The echo thing is just for simplification. It’s more like:

complex_function | myvar=$(</dev/stdin)

And I don’t get, why that doesn’t work. This works for example:

complex_function | echo $(</dev/stdin)

Answer 1

The correct solution is to use command substitution like this:

variable=$(complex_command)

as in

message=$(echo 'hello')

(or for that matter, message=hello in this case).

Your pipeline:

echo 'hello' | message=$(</dev/stdin)

or

echo 'hello' | read message

actually works. The only problem is that the shell that you're using will run the second part of the pipeline in a subshell. This subshell is destroyed when the pipeline exits, so the value of $message is not retained in the shell.

Here you can see that it works:

$ echo 'hello' | { read message; echo "$message"; }
hello

... but since the subshell's environment is separate (and gone):

$ echo "$message"

(no output)

One solution for you would be to switch to ksh93 which is smarter about this:

$ echo 'hello' | read message
$ echo "$message"
hello

Another solution for bash would be to set the lastpipe shell option. This would make the last part of the pipeline run in the current environment. This however does not work in interactive shells as lastpipe requires that job control is not active.

#!/bin/bash
shopt -s lastpipe
echo 'hello' | read message
echo "$message"

Answer 2

Use command substitution:

myvar=`echo foo`

or

myvar=$(echo foo)

Answer 3

Given a function that modifies a global variable and outputs something on stdout:

global_variable=old_value
myfunction() {
  global_variable=new_value
  echo some output
}

In ksh93 or mksh R45 or newer you can use:

var=${
  myfunction
}

Then:

$ print -r -- "$global_variable, $var"
new_value, some output

${ ...; } is a form of command substitution that doesn't spawn a subshell. For commands that are builtin commands, instead of having them writing their output to a pipe (for which you'd need different processes to read and write on the pipe to avoid dead-locks), ksh93 just makes them not output anything but gather what they would have been outputting in to make up the expansion. mksh uses a temporary file instead.

$ ksh -c 'a=${ b=123; echo foo;}; print -r -- "$a $b"'
foo 123

fish's command substitution also behaves like ksh93's ${ ...; }:

$ fish -c 'set a (set b 123; echo foo); echo $a $b'
foo 123

In most other shells, you'd use a temporary file:

myfunction > file
var=$(cat file) # can be optimised to $(<file) with some shells

On Linux, and with bash 4.4 or older or zsh (that use temp files for <<<), you can do:

{
  myfunction > /dev/fd/3 &&
  var=$(cat<&3)
} 3<<< ''

In zsh, you can also do:

() {
   myfunction > $1
   var=$(<$1)
} =(:)

In Korn-like shells such as ksh, zsh or bash, command substitution, whether the $(cmd...) standard form or the $(<file) or ${ cmd...; } variants strip all trailing newline characters (from file or the output of cmd). See shell: keep trailing newlines ('\n') in command substitution for how to work around that.

In fish, set var (cmd) assigns each line of the output of cmd to separate elements of the $var array. $var will contain the same thing whether cmd outputs foo or foo<newline>. Since version 3.4.0, fish also supports set var "$(cmd)" which behaves like in Korn-like shells (removes all trailing newline characters).

Answer 4

Here is a simple way to do it in bash -

complexFunction(){
cat <<_endOfHereFile_
The lazy dog
jumped over 
the moon
_endOfHereFile_
}

{ read -d '' message; }< <(complexFunction)
echo "${message}"

The result of echo "${message}" is

The lazy dog
jumped over 
the moon

The complex function could be any function emitting to stdout. It happens to use a here file in this example, but that is not important.

The call read -d '' reads stdin up to the delimiter, which since it is the empty character `` means reading until the end of input.

The syntax {lhs;}< <(rhs) redirects the stdout of rhs to the stdin of lhs, where lhs enjoys the shared variable namespace so that echo ${message} works as desired.

Answer 5

I would use a temporary file to store the complex function result, and then read the value from the temp file for processing.

# Create a temp file for storage
tmppath=$(mktemp)
# run the command and get the output to the temp file
complex_command > "${tmppath}"
# read the file into variable
message=$(cat "${tmppath}")
# use the variable
echo "$message"
rm -f "${tmppath}"

The usage of mktemp can refer to How create a temporary file in shell script?

Answer 6

I am not really an expert, but have you tried the following?

echo "hello world" \
| { echo_out=$(< /dev/stdin); echo "echo output is: $echo_out"; } \
| cut -d":" -f 2

Answer 7

You could just use the read command and Command Grouping with curley braces: echo foo | { read myvar; echo $myvar; } except you want to use "global variables", which I think you mean "shell variables" (declare -p to list, or set | grep myvar or set -o posix; set). https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html

Using Command Grouping executes them in the "current shell context", however that is still a subshell as soon as a pipe is used, even if the braces surround the entire line { echo foo | read myvar; echo $myvar; }.

So you have to use shell options. (it seems, because the command line is interactive by default):

shopt -s lastpipe      # sets shell option for lastpipe on
set +m                 # turns job control off, part of interactive shell
echo foo | read myvar; echo $myvar
# foo

With those shell options, chaining works read works echo foo | read myvar; echo $myvar" bar" | read myvar2; echo $myvar2, to produce foo bar.

PS, you can put those shell options in your .bashrc, but I'm not sure if there are downsides to that beside possible incompatibility with scripts that use interactive shell flag -i.

Answer 8

Notes to editors: Please don't change the code. I'll report it.

I hope you remember your username and password because lot of people found that answer useful.. But not me, no Sr. and that's because in a Makefile rules are slightly different.

In my case I'm encoding some JSON data and providing that to curl so I'm going to share just the basic idea because it is already encoded if you use the right type.

# Remember, this is a Makefile!
# Assuming your environment knows the variables in brackets
more_tests:
    @LABORADO=$$(echo -n "{"user":"$(ALWAYS)","know":"$(NOTHING)"}" | base64) && 

    echo $(SHELL) && 

    echo "$$LABORADO"

And this is the output:

> make more_tests 
/bin/sh
"eyJ1c2VyIjoic2hvdWxkIiwia25vdyI6ImhpbXNlbGYifQ=="

The shell variable was just a random environment variable that I picked as an example because I know whatever your environment is it will know that value (I hope).

Remember.. I know $ALWAYS, $NOTHING, but I know my environment :D)