Replace environment variables in text if they exist

Question

I know that envsubst replaces declared environment variables in the input.

$ echo 'Hello $USER' | envsubst
Hello myusername

What I want is a way to replace the environment variable if it exists otherwise envsusbst (or any other command), leaves the variable string as it is. What I get is:

$ echo 'Hello $USER $UNDEFINED_VARIABLE' | envsubst
Hello myusername

What I want is:

$ echo 'Hello $USER $UNDEFINED_VARIABLE' | somecommand
Hello myusername $UNDEFINED_VARIABLE

Answer 1

You can use env to see all currently defined environment variables and then use that list to only replace those. (The man page isn't very clear on that, but see this answer for elucidation.)

echo 'Hello $USER $UNKNOWN' | envsubst "$(env | cut -d= -f1 | sed -e 's/^/$/')"

(The output of env lists the values of the variables as well, but envsubst also wants to see a leading $, so we can't just use cut -d= -f1 on its own, unfortunately. You could use a single sed to do cut's job as well, see previous revision, but I prefer the clarity of cut over a tiny performance gain.)

Answer 2

It you pass an argument like $USER$PATH to envsubst, then it expands only those variables that are referenced in that argument.

So one way could be to pass it all the currently defined environment variables in that format. With zsh:

echo 'Hello $USER ${USER} $UNDEFINED_VARIABLE' |
  envsubst \$${(kj:$:)parameters[(R)*export*]}

• $parameters is a special associative array that maps variable names to their type
• $parameters[(R)*export*] expands to all the elements of the associative array whose value contains export.
• with the k parameter expansion flag, the key instead of the value is returned
• j:$: joins those elements with $ in between, and we add one at the start.

With other shells, you can always revert to perl to get that list:

echo 'Hello $USER ${USER} $UNDEFINED_VARIABLE' |
  envsubst "$(perl -e 'print "\$$_" for grep /^[_a-zA-Z]\w*$/, keys %ENV')"

Beware both disclose your environment variable names in the output of ps.

Instead, you could also do the whole thing in perl:

perl -pe 's{(?|\$\{([_a-zA-Z]\w*)\}|\$([_a-zA-Z]\w*))}{$ENV{$1}//$&}ge'

Beware it has the same limitations as envsubst in that it won't expand things like ${VAR:-x} and would expand $HOME in things like \$HOME or $$HOME which a shell wouldn't.