0

Given I would like create environment variables from files containing newlines and special characters. The clue is that this content can have references to other environment variables which have to be intepreted. Maybe I need more complex tools than just plain bash?

A testfile "testfile.txt" could look like:

* foo
\/-!~ 
FOO=${FOOBAR}

What I have managed myself so far is the following:

#!/bin/bash
FOOBAR="SUCCESS"
file="testfile.txt"
var_name="TEST"
name="$file"
content="$(<$file)"
echo "$content"

Which gives me a context variable as output (so, exactly the contents of the file):

* foo
\/-!~
FOO=${FOOBAR}

But, how can I:

  1. Let ${FOOBAR} be set to "SUCCESS" as defined above but do not touch the rest?
  2. Set an environment variable TEST which would have content as (1)?
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
J. Doe
  • 177
  • an additional aspect here whether it's possible to set the resulting multi-line substituted-values variable as environment variable. – J. Doe Jul 28 '17 at 06:05

2 Answers2

2

This answer is based on and adapted from the accepted answer to https://stackoverflow.com/questions/10683349/forcing-bash-to-expand-variables-in-a-string-loaded-from-a-file

#!/bin/bash
export FOOBAR="SUCCESS"
file="testfile.txt"
var_name="TEST"
name="$file"

content="$(envsubst < "$file")"
echo "$content"

Unlike eval, envsubst only substitutes variables, there is no risk of executing other programs via command substitution or similar. It's a very simple templating tool for shell.

envsubst is part of the GNU gettext internationalisation utilities (on debian, it's in the gettext-base package).

Note that envsubst (being an external command, not a shell built-in) can only see variables that have been exported (but see the allexport standard sh option to make the shell export subsequently defined variables to the environment).

Stéphane Chazelas
  • 544,893
cas
  • 78,579
  • Note that only $X and ${X} are supported. Not ${X#pattern} (even if you have a X#pattern environment variable), ${X-default}... And there's no escaping \$VAR becomes \content-of-$VAR. $$VAR becomes $content-of-$VAR... Probably what the OP wants in the first place but worth pointing out. – Stéphane Chazelas Jul 27 '17 at 11:53
  • yep, it's very limited and very simple. simple variable substitutions only, a very primitive templating tool. also reasonably safe. and, in addition to the problems you mentioned, if the template file contains $FOOBAR but that variable doesn't exist or isn't exported then it gets replaced with the empty string...this can be a feature OR a bug depending on what you want. It can even be both a feature and a bug in the same template file :). Conclusion: For anything but the most basic templating, use awk or perl or python or anything-but-shell. – cas Jul 27 '17 at 12:18
  • but, how can I expose the result as environment variable? – J. Doe Jul 28 '17 at 06:05
  • you can capture the output of anything that prints to stdout with, e.g., content="$(....)". same as I used for envsubst above. Note that almost all languages have some form of easy access to exported environment variables - e.g. perl has a hashed array %ENV that contains the environment, so you'd access the export FOOBAR shell var with $ENV{FOOBAR} – cas Jul 28 '17 at 06:14
0

Try:

eval "content=\$(cat << @@@EOF@@@
$(cat file)
@@@EOF@@@
)"

That will expand the parameter, arithmetic and command substitutions, and \ is used for escaping (of $, ` and itself only) and line-continuation. For instance, on an input like:

* foo
\/-!~ 
FOO=${FOOBAR} $((1+1)) $(uname)
\${FOOBAR}\\foo\
bar

That gives something like:

* foo
\/-!~
FOO=content-of-$FOOBAR 2 Linux
${FOOBAR}\foobar

It should be obvious that it's a command injection vulnerability if the content of the file is not under your full control.

Stéphane Chazelas
  • 544,893