Replacing only specific variables with envsubst

Question

I'm trying to perform environment variable replacement through envsubst, but I want to only replace specific variables.

From the docs I should be able to tell envsubst to only replace certain variables but I'm failing to be able to do that.

For example, if I have a file containing:

VAR_1=${VAR_1}
VAR_2=${VAR_2}

how should I execute envsubst so that it only replaces the reference to ${VAR_1}?

Answer 1

Per the man page:

envsubst [OPTION] [SHELL-FORMAT]

If a SHELL-FORMAT is given, only those environment variables that are referenced in SHELL-FORMAT are substituted; otherwise all environment variables references occurring in standard input are substituted.

Where SHELL-FORMAT strings are "strings with references to shell variables in the form $variable or ${variable}[...] The variable names must consist solely of alphanumeric or underscore ASCII characters, not start with a digit and be nonempty; otherwise such a variable reference is ignored.".

---

Note that the format ${VAR:-default} is not supported. I mentioned HERE some alternatives that support it along with other features.

---

Anyway, back to gettext envsubst:
So, one has to pass the respective variables names to envsubst in a shell format string (obviously, they need to be escaped/quoted so as to be passed literally to envsubst). Example:

input file e.g. infile:

VAR1=${VAR1}
VAR2=${VAR2}
VAR3=${VAR3}

and some values like

export  VAR1="one" VAR2="two" VAR3="three"

then running

envsubst '${VAR1} ${VAR3}' <infile

or

envsubst '${VAR1},${VAR3}' <infile

or

envsubst '${VAR1}
${VAR3}' <infile

outputs

VAR1=one
VAR2=${VAR2}
VAR3=three

Or, if you prefer backslash:

envsubst \$VAR1,\$VAR2 <infile

produces

VAR1=one
VAR2=two
VAR3=${VAR3}

To avoid leakage of the custom-set variables into the executing shell's environment, instead of exporting these variables, some shells such as Bash allow prefixing the command with variable assignments. The values assigned in this way will only be used for the command's execution, and don't affect the shell environment afterwards:

export VAR1="one" VAR2="two" VAR3="three"
...
VAR1="number 1" VAR3="numero 3" envsubst '${VAR1} ${VAR3}' <infile
output:
VAR1=number 1
VAR2=${VAR2}
VAR3=numero 3
echo "VAR1=${VAR1}"
echo "VAR2=${VAR2}"
echo "VAR3=${VAR3}"
output shows that the original environment was preserved:
VAR1=one
VAR2=two
VAR3=three

Answer 2

Although related to docker, the utility envplate should do the job https://github.com/kreuzwerker/envplate

From the readme:

Trivial templating for configuration files using environment keys. References to such keys are declared in arbitrary config files either as:

${key} or

${key:-default value}

gnutext's envsubst only replaces ${key}; if missing is replaced by ''.

Answer 3

Unfortunately 'export' is mandatory, otherwise it won't work. Here is an example.

First, the input file. Note that I only want to change NGINX_HOST and NGINX_PORT and don't want to touch other variables in the input file such as '$uri'.

$ egrep 'server_name|listen|try_files' nginx-default.conf.template 
  server_name ${NGINX_HOST};
  listen ${NGINX_PORT};
    try_files $uri $uri/ /index.php?$args;

Now the empty variables:

$ echo ${NGINX_HOST}
$ echo ${NGINX_PORT}

Set default values to these variables:

$ NGINX_HOST=${NGINX_HOST:-localhost}
$ NGINX_PORT=${NGINX_PORT:-80}
$ echo ${NGINX_HOST}
localhost
$ echo ${NGINX_PORT}
80

Try to use envsubst:

$ envsubst \
    '${NGINX_HOST} ${NGINX_PORT}' \
    < nginx-default.conf.template \
    | egrep 'server_name|listen|try_files'
server_name ;
  listen ;
  try_files $uri $uri/ /index.php?$args;

As you can see, it did not work. The variables must be 'export'-ed for this to work. Here is the second run:

$ export NGINX_HOST=${NGINX_HOST:-localhost}
$ export NGINX_PORT=${NGINX_PORT:-80}
$ envsubst   '${NGINX_HOST} ${NGINX_PORT}' <  nginx-default.conf.template | egrep 'server_name|listen|try_files'
server_name localhost;
  listen 80;
    try_files $uri $uri/ /index.php?$args;

As you can see, now it worked. Hope it helps others.

Answer 4

Before calling envsubst you should use export using single quotes to get back VAR_1 modified. As in:

export VAR_1='somevalue'

For more details, please see:

How to substitute shell variables in complex text files