solution to insert a special character using sed

Question

I am trying to use sed to replace & by \& in a tex file. I'm using sed -i "s/ & / \\\& /g" bibfile.bib; However, the result in the file is \ & and not \& as I expect. Any help would be welcome, Marc

Answer 1

In Bourne-like shells, using double quotes means you have one extra layer of escaping to do for backslash as \ is used inside double quotes to escape `, ", $ and itself and do line continuation.

Use single quotes instead.

There are also more shells that support '...' as a quoting operator than some supporting "...".

So:

sed -i 's/ & / \\\& /g' bibfile.bib

calls sed with sed, -i, s/ & / \\\& /g¹ and bibfile.bib as four separate arguments.

To send the same arguments using double quotes, you'd need at least:

sed -i "s/ & / \\\\\&" bibfile.bib

Where the first two pairs of \ become one \ each and \& stays \&.

sed -i "s/ & / \\\\\\&" bibfile.bib

Would also work and be cleaner IMO making it more explicit that you want 3 backslashes to be passed to sed, each inserted as \\ within double quotes.

You could also do:

sed -i 's/ \(& \)/ \\\1/g' bibfile.bib

Or:

sed -Ei 's/ (& )/ \\\1/g' bibfile.bib

Your:

sed -i 's/ & / \\\& /g' bibfile.bib

Actually passes sed, -i, s/ & / \\& /g and bibfile.bib as argument to sed, which it interprets as prefixing each " & " with " \" and suffixing with " ".

---

For completeness, in some other shells:

csh/tcsh

in csh/tcsh, \ is not special within '...' nor "..." (except to escape ! for history expansion).

So sed -i "s/ & / \\\& /g" bibfile.bib and sed -i 's/ & / \\\& /g' bibfile.bib would both work there.

fish

In fish, \ is also special inside '...', so you'd need either:

sed -i "s/ & / \\\\\\& /g" bibfile.bib

Or

sed -i 's/ & / \\\\\\& /g' bibfile.bib

The latter still preferred in the general case as there are fewer characters to escape within.

rc

rc has only one form of quoting: '...' (not "...", nor \ nor $'...' nor $"...")

sed -i 's/ & / \\\& /g' bibfile.bib

only there.

See How to use a special character as a normal one in Unix shells? for more details as to what characters are special and how to escape/quote them in various shells.

---

^{¹ and for sed, in the replacement part of its s/pattern/replacement/flags command, the \\ is interpreted as a backslash and \& as a literal & as opposed to its special meaning of expanding to what was matched.}