sed: Substituting text pattern containing backslash

Question

I have an issue with sed that I have been able to recreate with the following simple example.

Consider the following input file (input.txt):

C:\A\quick\brown\fox\ jumps over the lazy dog
C:\A\quick\brown\fox\ ran with the hounds

I want to generate the following output

C:\Animal\ jumps over the lazy dog
C:\Animal\ ran with the hounds

I tried to create a simple shell script, using sed, but it is not performing the required substitution. Here is my script:

FROM_PATTERN="C:\A\quick\brown\fox\"
TO_PATTERN="C:\Animal\"
#FROM_PATTERN="C:\A\quick\brown\fox\"       # Escaping backslash does not help either
#TO_PATTERN="C:\Animal\"       # Escaping backslash does not help either
sed 's/$FROM_PATTERN/$TO_PATTERN/g' input.txt
#sed 's/"$FROM_PATTERN"/"$TO_PATTERN"/g' input.txt       # Quoting the pattern does not help either

---

I am running bash version GNU bash, version 4.4.12(3)-release-(x86_64-unknown-cygwin)

Answer 1

\ is special:

• as a quoting operator in the shell, including inside double quotes where it can be used to escape itself,", $, ` and do line continuations.
• as a regexp operator (both for escaping and introducing new operators) on the left-hand (pattern) side of the s command in sed.
• in the replacement part of the s sed command where it can escape &, itself and newline (or introduce C-style escape sequences such as \n in some sed implementations).

Also note that shell parameter expansion is not performed inside single-quoted strings.

So here, you'd want to:

• use single quotes instead of double quotes around the \ characters
• escape the \ for both the left hand side and right hand side of the s command
• use double quotes in the part where variables must be expanded.

from_regexp='C:\\A\\quick\\brown\\fox\\'
escaped_replacement='C:\\Animal\\'
sed "s/$from_regexp/$escaped_replacement/g" < your-file

Or you could use perl instead of sed where you can do substitutions of fixed strings without having to worry about special characters if you do it like:

from='C:\A\quick\brown\fox\'
to='C:\Animal\'
FROM=$from TO=$to perl -pe 's/\Q$ENV{FROM}\E/$ENV{TO}/g' < your-file

See also How to ensure that string interpolated into `sed` substitution escapes all metachars to work with arbitrary strings.

Answer 2

This was run on Linux machine, not sure if cygwin's behaviour is the same as with bash on linux.

PC1:~> echo 'C:\A\quick\brown\fox\ jumps over the lazy dog' | sed -ne 's|\(C:\\A\).\+\\\+\(.\+\)|\1nimal\2|p'
C:\Animal jumps over the lazy dog
PC1:~> echo 'C:\A\quick\brown\fox\ ran with the hounds' | sed -ne 's|(C:\A).+\+(.+)|\1nimal\2|p'
C:\Animal ran with the hounds
PC1:~>

So I used group capturing, reused the 'A', then append 'nimal' after it. And to make things a bit clearer, I used '|' as the substitute separator to avoid too many forward slashes.

Hope it helps.