How to escape special characters in a string?

Question

Assuming $file holding a value of a file name, say Dr' A.tif. In bash programming, how could I escape single quote and any other special character of the $file without removing the special character?

Update on 9 July 2014

As request from @Gilles, following code snippet that doesn't able to handle Dr' A.tif:

files=$(find /path/ -maxdepth 1 -name "*.[Pp][Dd][Ff]" -o -name "*.[Tt][Ii][Ff]")
echo "${files}" > ${TEMP_FILE}
while read file
do
   newfile=$(echo "${file}" | sed 's, ,\\ ,g') ## line 1
done < ${TEMP_FILE}

After I have tried out the answer from @Patrick on line 1, it seems to work for me. But if I have file such as Dr\^s A.tif, printf command doesn't seem help, it shows me Dr\^s\ A.tif. If I manually try it on console like this:

printf "%q" "Dr\^s A.tif"

I will have this output:

Dr\\\^s\ A.tif

Any idea how to handle this?

in what context are you expecting to use $file? or is this issue assigning the string with the special character to $file? — rob, Jul 08 '14 at 10:50
Actually the find command return me an array of file list. And then I loop through this file list into $file variable. — huahsin68, Jul 08 '14 at 10:56
You've been given some correct answers here, but they are probably not the answers to the question that you're really asking. From your comment here, I strongly suspect that you're on the wrong track. We can't help you much because you keep not showing your code. I do however recommend that you read Why does my shell script choke on whitespace or other special characters? as background. Show your script and explain what you want to do. — Gilles 'SO- stop being evil', Jul 08 '14 at 21:18

score 25 · Answer 1 · edited Feb 24 '17 at 00:42

25

You can use the printf builtin with %q to accomplish this. For example:

$ file="Dr' A.tif"
$ printf '%q\n' "$file"
Dr\'\ A.tif

$ file=' foo$bar\baz`'
$ printf '%q\n' "$file"
\ foo\$bar\\baz\`

From the bash documentation on printf:

In addition to the standard format specifications described in printf(1)
and printf(3), printf interprets:
%b       expand backslash escape sequences in the corresponding argument
 %q       quote the argument in a way that can be reused as shell input
 %(fmt)T  output the date-time string resulting from using FMT as a format
          string for strftime(3)

edited Feb 24 '17 at 00:42

wjandrea

658

answered Jul 08 '14 at 12:15

phemmer

71,831

1

This doesn't work with certain cases .e.g. when you need to preserve double-quotes. – user3467349 Nov 07 '15 at 06:08
1

@user3467349 it works just fine with quotes. I'm betting you're trying something like printf '%s' "foo". You need to understand how argument parsing works in the shell first. See https://www.gnu.org/software/bash/manual/bash.html#Shell-Operation #2 happens before #6. – phemmer Nov 07 '15 at 18:49
Could you provide an example of this string printed with printf and no other manipulations It doesn't have a: "" – user3467349 Nov 08 '15 at 04:25
Your issue has nothing to do with printf. Your issue is that you don't want the shell to parse your string. In order to do that, you have to pass your input to the shell in a way where it doesn't even try to parse it. One way to do this would be read -r -p 'input: ' && printf '%q\n' "$REPLY", and provide the input when prompted. – phemmer Nov 08 '15 at 06:48
Provide input while prompted won't work well in a shell script, neither is it speedy even interactively. Hence my above comment about this being a limited / sub-optimal solution. – user3467349 Nov 08 '15 at 10:10
1

As i've said several times now. Not knowing how how to pass raw data to the shell isn't an issue with printf. Perhaps you should ask a question rather than critiquing a solution that has nothing to do with your problem. – phemmer Nov 08 '15 at 15:16
You can use cat <<EOF; (as I wrote below) which seems preferable to me since it will handle edge-cases which your answer doesn't support. I should point out that How to escape special characters in a string? is a generic search-phrase that will cause people to stumble on this question and it makes sense to provide an answer that will cover cases which fit the title (even if it's not exactly what OP was asking). – user3467349 Nov 08 '15 at 15:38
For a more modern version of this answer without printf, see https://stackoverflow.com/a/27817504/442022 . – colan Aug 23 '21 at 17:53

score 4 · Answer 2 · answered Jul 08 '14 at 10:54

Try:-

file=Dr\'\ A.tif
echo $file
Dr' A.tif

or

file="Dr' A.tif"
echo $file
Dr' A.tif

or if the string contains a double quote:-

file='Dr" A.tif'
echo $file
Dr" A.tif

There are good tutorials on escaping and quoting on the net. Start with this one.

l0b0 · Answer 3 · 2024-01-29T04:53:11.573

1

You don't need to escape any file names you are handling in a script. Escaping is only necessary if you want to put a file name as a literal in a script, or to pass several file names as a single input stream to another script.

Since you're looping through the output of find, this is one of the simplest ways (!) to handle every possible path:

while IFS= read -r -d ''
do
    file_namex="$(basename -- "$REPLY"; echo x)"
    file_name="${file_namex%$'\nx'}"
    do_something -- "$file_name"
done < <(find "$some_path" -exec printf '%s\0' {} +)

edited Jan 29 '24 at 04:53

answered Jul 08 '14 at 12:09

l0b0

51,350

I think you need another < before the <( – user202729 Jan 28 '24 at 22:26
Instead of futzing to make cmdsub+basename work for weird names, why not just ${REPLY##*/} -- or even IFS=/ read -r -d '' -a split; ... "${split[${#split[@]}-1]}" ... – dave_thompson_085 Jan 29 '24 at 06:11

score -1 · Answer 4 · edited Apr 24 '15 at 23:53

-1

quick and (very) dirty

find . | sed 's/^/"/' | sed 's/$/"/'

edited Apr 24 '15 at 23:53

chaos

48,171

answered Apr 24 '15 at 23:37

Michael

9

score -2 · Answer 5 · edited Jan 28 '24 at 22:01

-2

A lot of these answers, including the top-voted one using printf "%q", will not work in all cases without additional manipulation. I would suggest the following (example below):

cat <<EOF
2015-11-07T03:34:41Z app[postgres.0000]: [TAG] text-search query doesn't contain lexemes: ""
EOF

edited Jan 28 '24 at 22:01

G-Man Says 'Reinstate Monica'

22,870

answered Nov 07 '15 at 06:07

user3467349

517

Excuse the n00b question but can you elaborate on how you would actually use this to escape characters in a string? If I copy the code you write into my terminal it just prints
2015-11-07T03:34:41Z app[postgres.0000]: [TAG] text-search query doesn't contain lexemes: ""

...nothing is escaped.
– SharkAlley Sep 08 '16 at 05:45
That's literally the point, all of the special characters are interpreted as literal characters not for their special meaning. Try to echo "the above string" instead to see the difference. – user3467349 Sep 12 '16 at 14:11
This answer would be an appropriate answer for e.g. quoting - How to use a special character as a normal one in Unix shells? - Unix & Linux Stack Exchange , but not this question where the "string to be escaped" is the output of a command, not hard coded. – user202729 Jan 28 '24 at 22:32

How to escape special characters in a string?

5 Answers5

Linked