6

I have a variable, e.g. a'b, containing a single quote which I need to replace by two single quotes before writing it to a file: a''b.

The following bash code used to get the job done for me ...

line="a'b"
echo "${line//\'/\'\'}" > out.txt

... until today when I discovered diverging outputs depending on the version of bash being used:

  • GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu): a''b
  • GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu): a\'\'b

I tried to modify the line above in numerous ways but was unable to have it produce the same output a''b in both bash shells.

I ended up using a combination of echo and sed instead,

echo "$(echo $line | sed "s/'/''/g")" > out.txt

but I am still wondering if the job can be gotten done with a more concise pure bash expression. Can it?

Bastian
  • 161
  • the first thing that comes to mind is that you don't need the echo. just printf "%s\n" $line | sed "s/'/''/g" > out.txt will do. BTW, the echo "${line//\'/\'\'}" works as desired in bash 5.0.3 for me. – cas Aug 28 '19 at 08:58
  • Thou should never use variables in format argument of printf. – pLumo Aug 28 '19 at 08:58
  • 2
    are you parsing the output of ls? if so then a) don't do that and b) run man ls and search for --quoting-style and c) run typeset -p QUOTING_STYLE and d) see Why is 'ls' suddenly wrapping items with spaces in single quotes? – cas Aug 28 '19 at 09:02
  • 1
    If you are parsing ls, then this is an XY Problem for at least two reasons. Firstly because there are many better ways to process a list of files than to parse ls. Secondly because the need to double any single-quotes probably only arises because you're trying to parse the output of ls and then (presumably) use the filenames as arguments to some other program. What are you actually trying to achieve that you think parsing ls is part of the solution for? – cas Aug 28 '19 at 09:09
  • @pLumo you are right, and I had replaced the printf call with an echo call by now. Edited the question this way too. – Bastian Aug 28 '19 at 09:21
  • @cas I am actually processing outputs of git calls, e.g. git diff -U0 --no-prefix --no-color --ignore-space-at-eol -b -w ${IN_DIR} – Bastian Aug 28 '19 at 09:23
  • @cas the reason why I'm using echo is, that I am in the actual code doing the following: echo " write(*,*) '$(echo "${line}" | sed "s/'/''/g")'" >> ${OUT_FILE} – Bastian Aug 28 '19 at 09:24

2 Answers2

8

One of the changes between bash-4.3-alpha, and the previous version, bash-4.2-release:

When using the pattern substitution word expansion, bash now runs the replacement string through quote removal, since it allows quotes in that string to act as escape characters. This is not backwards compatible, so it can be disabled by setting the bash compatibility mode to 4.2.

Input:

BASH_COMPAT=4.2
line="a'b"
echo "${line//\'/''}"

Output:

a''b
Kamil Maciorowski
  • 21,864
6

For a method that would work regardless of the version (and shell implementation like ksh93 where it comes from and mksh and zsh which also support it), you can store the pattern and replacement in a variable:

pattern="'"
replacement="''"
printf '%s\n' "${line//$pattern/$replacement}"

Or

q="'"
printf '%s\n' "${line//$q/$q$q}"

(note that in zsh, the $pattern is taken as a fixed string. If you want it to be taken as a wildcard pattern (which doesn't apply here as $pattern doesn't contain wildcard characters), you need to replace $pattern with $~pattern. In other shells, if you want $pattern to be taken as a fixed string, you need to quote it (${line//"$pattern"/$replacement}).

Stéphane Chazelas
  • 544,893
  • Also: "${line//\47/\47\47}" – Léa Gris Jan 22 '21 at 13:18
  • @LéaGris this doesn't work for me in Bash 5.1 but a similar ANSI-quoted string does. "${line//$'\47'/'}". Only had to do this because of a syntax highlight bug in my editor that choked on ${line//"'"/'} – miken32 Apr 23 '22 at 17:53