How to add double quotes to every value in env file

Question

I have an .env file and need to copy to an .env.sh file. The command exports to .env.sh file:

scriptPath=$(dirname "$(readlink -f "$0")")
printenv | sed 's/^\(.*\)$/export \1/g' > ${scriptPath}/.env.sh

my .env file:

DB_MYSQL_TYPE=mysql
DB_MYSQL_CONNECTOR=pymysql
DB_MYSQL_USER=tuandc
DB_MYSQL_PASS=mypass
DB_MYSQL_HOST=mysql.host
DB_MYSQL_PORT=3306
DB_MYSQL_DBNAME=vietnam

i 've exported it to .env.sh:

export DB_MYSQL_TYPE=mysql
export DB_MYSQL_CONNECTOR=pymysql
export DB_MYSQL_USER=tuandc
export DB_MYSQL_PASS=mypass~.(?Tweg
export DB_MYSQL_HOST=mysql.host
export DB_MYSQL_PORT=3306
export DB_MYSQL_DBNAME=vietnam

but i want to have double quotes to escape some special characters in DB_MYSQL_PASS:

printenv | sed -e 's/=\(.*\)/="\1/g' -e 's/$/"/g' >> ${scriptPath}/.env.sh

DB_MYSQL_TYPE="mysql"
DB_MYSQL_CONNECTOR="pymysql"
DB_MYSQL_USER="tuandc"
DB_MYSQL_PASS="mypass~.(?Tweg"
DB_MYSQL_HOST="mysql.host"
DB_MYSQL_PORT="3306"
DB_MYSQL_DBNAME="vietnam"

but i want to combine 2 regex above to make .env.sh file complete like this:

export DB_MYSQL_TYPE="mysql"
export DB_MYSQL_CONNECTOR="pymysql"
export DB_MYSQL_USER="tuandc"
export DB_MYSQL_PASS="mypass~.(?Tweg"
export DB_MYSQL_HOST="mysql.host"
export DB_MYSQL_PORT="3306"
export DB_MYSQL_DBNAME="vietnam"

Inside double quotes, $, backslash, backtick (and obviously ") are still a problem (and potentially characters whose encoding contains the encoding of those which in practice happens for backslash and backtick in some locales). Single quotes are a much better choice, though you'd still need to handle ' specially. See also Escape a variable for use as content of another script — Stéphane Chazelas, Mar 28 '22 at 14:22
I wonder if something based on the shell's declare -x -p would be more robust than trying to wrangle the output of printenv? — steeldriver, Mar 28 '22 at 15:32
@steeldriver I don't think declare is present in POSIX but it would work well for bash — SeetheMoar, Mar 28 '22 at 15:42
It ‘ ‘goes without saying’ ’ that values with newline(s) will break pretty much anything you try to do in the realm of post-processing. I guess the lesson here is that printenv, like ls, is (arguably) a useful tool for producing output for display to users, but not for providing input to programs. There; I said it. :-) ⁠ I see now that Stéphane Chazelas said this in his answer (although not in his comment). — G-Man Says 'Reinstate Monica', Apr 06 '22 at 19:17

gilaro · Accepted Answer · 2022-03-29T05:35:03.283

3

$ declare -px

declare is a Bash builtin command that seems appropriate. This command escapes special characters for the shell in existing variable assignments.

edited Mar 29 '22 at 05:35

answered Mar 28 '22 at 17:26

gilaro

56

1

To add to @gilaro 's answer, see this SO answer of me with different flavours by shell. I never understood why printenv did not have an option to quote values. – Mat M Aug 12 '22 at 15:12

score 1 · Answer 2 · answered Mar 28 '22 at 14:23

Easily done with sed, the Stream EDitor:

sed -i.bak -e `s/=/="/;s/$/"/` .env.sh

This does:

-i.bak - edit in place, original file left as .env.sh.bak.
s/=/="/ - replace "=" with "="".
; - command separator
s/$/"/ - match the end-of-line and add a quote.

Read man sed

score 1 · Answer 3 · answered Mar 28 '22 at 18:00

1

Do it in a single phase. With the original file (.env):

printenv | sed 's/\(^[^=]*\)=\(.*\)/export \1="\2"/' > "$scriptPath/.env.sh"

answered Mar 28 '22 at 18:00

DanieleGrassini

2,824

This is what the question asks for, but it would be a better answer if it disclosed that it probably isn’t what the OP really needs. – G-Man Says 'Reinstate Monica' Apr 06 '22 at 18:56

Stéphane Chazelas · Answer 4 · 2022-03-29T08:09:33.497

The standard command to output all the environment variables mapped to shell variables as shell code that can be interpreted to set the same environment variables is export -p. export will take care of quoting the value as appropriate.

In the case bash's export though, you need the posix option to be enabled (like when it's invoked as sh) for that output to be in sh-compatible syntax.

(set -o posix; export -p)

Though note that bash's export does use the "..." quotes for some values, which is not reliable when used with non ASCII text as the code may be interpreted differently when run in a different locale.

Also note that if a shell variable has been exported and not given any value, it will give: export variable.

Doing:

sh -c 'export -p'

Would address the latter, and potentially also the former if your sh is not bash and uses the safer single quotes to quote the values.

If you consider that arbitrary environment variable values and names can contain newline characters (or single quotes or double quotes or non-text for that matters), it should become evident that the output of printenv cannot be post-processed reliably with line-based text utilities.

For example, with printenv here called with one (weirdly named) environment variable:

$ env -i $'$(reboot)\n=\n"$(reboot)"=\n$(reboot)' printenv
$(reboot)
=
"$(reboot)"=
$(reboot)

With the GNU implementations of printenv and awk, you could however do something reliable if limiting for instance to environment variables that are mappable to shell variables and whose name starts with DB_MYSQL_:

printenv -0 | LC_ALL=C gawk -v 'RS=\0' -v q="'" '
  function shquote(s) {
    gsub(q, q "\\" q q, s)
    return q s q
  }
  match($0, /^(DB_MYSQL_[_[:alnum:]]*=)(.*)$/, capture) {
    print "export " capture[1] shquote(capture[2])
  }'

Where the -0 aka --null option of printenv causes printenv to print the environment variables NUL-delimited (and NUL contrary to newline cannot occur in an environment name nor value), and we use single quotes instead of double quotes (with ' in values inserted as \' outside of quotes), amd LC_ALL=C for every byte value to be considered valid text and the [:alnum:] character class to be limited to a-zA-Z0-9.

With zsh, you can do:

vars=(${(Mk)parameters[(R)scalar-export]:#DB_MYSQL_*})
(($#vars)) && export -p $vars

To print the definitions of scalar exported shell variables whose name starts with DB_MYSQL_. zsh's export doesn't use "..." quoting but may use $'...' or leave some non-ASCII characters unquoted, which could also be a problem if the generate code is interpreted in a different locale from that where the code has been generated.

set -o extendedglob
for var (${(Mk)parameters[(R)scalar-export]:#DB_MYSQL_[a-zA-Z0-9_]#})
  print -r export $var=${(Pqq)var}

Where we always use single quotes and skip variable names with non-ASCII alnums would make it safer.

score -1 · Answer 5 · answered Mar 28 '22 at 15:14

-1

Thank you, guys, I have found the solution I concatenate 2 regexes,

printenv | sed -e 's/^\(.*\)$/export \1/g' -e 's/=\(.*\)/="\1/g' -e 's/$/"/g' >> ${scriptPath}/.env.sh

answered Mar 28 '22 at 15:14

Công Tuấn

13

How to add double quotes to every value in env file

5 Answers5