How to grep double quote contains backslashes in single-quote in a file?

Question

I have a file contains a line like content below

xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

In a shell script, I want to grep u2py.DynArray value=b'F\xfeVOC\xfeD_VOC' how to achieve?

I tried grep "u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'" file, nothing returned.

Answer 1

Use the -F option (for fixed-string search, formerly done with the fgrep utility):

grep -F "u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'" file

The . and \ characters are regular expression operators, one that matches any single character, the other used to escape regular expression operators or introduce new ones (in front of x though, the behaviour is unspecified by POSIX and could vary with the implementation).

So without -F, you'd need to escape both.

Since in Bourne-like shells or fish, inside "...", \ also has a special escaping meaning for the shell, you'd also need another round of escaping for it:

grep "u2py\.DynArray value=b'F\\\\xfeVOC\\\\xfeD_VOC'" file

In csh or tcsh, that would be:

grep "u2py\.DynArray value=b'F\\xfeVOC\\xfeD_VOC'" file

The rc shell and derivatives don't support "..." as a quoting operator. You'd need to use single quotes:

grep 'u2py\.DynArray value=b''F\\xfeVOC\\xfeD_VOC''' file

(single quotes are expressed as '' inside single quotes there. See How to use a special character as a normal one? for more information).

Answer 2

Either use:

$ cat <<\END | grep -Ff - file
u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'
END

Or

$ var='u2py.DynArray value=b'"'"'F\xfeVOC\xfeD_VOC'"'"

$ grep -F -- "$var" file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

---

It turns out the problem is with the \ (backslash) not the ' (quotes). But first, a . needs to be quoted to be literal in a regex.

$ grep "u2py\.DynArray value=b'F" file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

As you can see above, the ' is found by grep.
But to find a \x, the change is drastic in bash:

$ grep "u2py\.DynArray value=b'F\\\\x" file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

Why four \? Because the shell converts two \\ to \ and grep receiving two \\ interprets it as one \ as \ is also an special character in regexes.

We can see the two steps with:

$ set -x; grep "u2py\.DynArray value=b'F\\\\x" file ; set +x
+ grep --color=auto 'u2py\.DynArray value=b'\''F\\x' file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>
+ set +x

We can reduce one level of interpretation with the -F option of grep.

$ set -x; grep -F "u2py.DynArray value=b'F\\x" file ; set +x
+ grep --color=auto -F 'u2py.DynArray value=b'\''F\x' file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>
+ set +x

Or, without set -x:

$ grep -F "u2py.DynArray value=b'F\\x" file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

That last level of "interpretation" is difficult to remove.
And all Bourne shells do that. And POSIX require it.

If it is possible to generate on stdout the exact string to search, we can use

grep -Ff - file

To search for the exact "Fixed string" (-F) from the file (-f) standard input (-) inside file.

This might seem to work:

$ printf '%s\n' "u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'"
u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'

But no, the shell is still waiting to remove backslashes:

printf '%s\n' "u2py.DynArray value=b'F\\\\xfeVOC\\\\xfeD_VOC'"
u2py.DynArray value=b'F\\xfeVOC\\xfeD_VOC'

The only robust way to avoid the backslash removal is to use a here document.
Ugly syntax, but works pretty well.

$ cat <<\END
u2py.DynArray value=b'F\xfeVOC\\\\xfeD_VOC'
END

Please note the use of \END (the END is quoted). And the command then becomes:

$ cat <<\END | grep -Ff - file
u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'
END

With a var there is no need for a here document once the var has the correct value:

$ var='u2py.DynArray value=b'"'"'F\xfeVOC\xfeD_VOC'"'"

$ grep -F -- "$var" file
xxx <u2py.DynArray value=b'F\xfeVOC\xfeD_VOC'>

Alternatively could be grep -Fe "$var" file. Thanks @StéphaneChazelas.