Bash =~ regex and https://regex101.com/

Question

Using https://regex101.com/ I built a regular expression to return the first occurrence of an IP address in a string.

RegExp:

(?:\d{1,3}\.)+(?:\d{1,3})

RegExp including delimiters:

/(?:\d{1,3}\.)+(?:\d{1,3})/

With the following test string:

eu-west                       140.243.64.99

It returns a full match of:

140.243.64.99

No matter what I try with anchors etc, the following bash script will not work with the regular expression generated.

temp="eu-west                       140.243.64.99            "
regexp="(?:\d{1,3}\.)+(?:\d{1,3})"
if [[ $temp =~ $regexp ]]; then
  echo "found a match"
else
  echo "No IP address returned"
fi

That looks like a Perl regular expression to me. Bash does not support that. — Kusalananda, Feb 02 '18 at 15:23
The =~ operator is discussed here in the manual where it's written bash uses "extended regular expressions". Extended regexes are described in the regex(7) man page and briefly summarized here. — glenn jackman, Feb 02 '18 at 15:57

ilkkachu · Accepted Answer · 2018-02-02T17:24:15.020

29

\d is a nonstandard way for saying "any digit". I think it comes from Perl, and a lot of other languages and utilities support Perl-compatible REs (PCRE), too. (and e.g. GNU grep 2.27 in Debian stretch supports the similar \w for word characters even in normal mode.)

Bash doesn't support \d, though, so you need to explicitly use [0-9] or [[:digit:]]. Same for the non-capturing group (?:..), use just (..) instead.

This should print match:

temp="eu-west                       140.243.64.99            "
regexp="([0-9]{1,3}\.)+([0-9]{1,3})"
[[ $temp =~ $regexp ]] && echo match

edited Feb 02 '18 at 17:24

answered Feb 02 '18 at 15:28

ilkkachu

138,973

2

Does your GNU grep support \d without -P? – Stéphane Chazelas Feb 02 '18 at 16:00
@StéphaneChazelas, whoops, of course not. It does support \w and \b, which I've learned from Perl, so I got that confused. – ilkkachu Feb 02 '18 at 17:15
3

it's not really fair to say \d or PCRE are "nonstandard". They are quite standard, just a different standard from original regular expressions and extended regular expressions. – erik258 Sep 19 '19 at 16:00
2

@DanielFarrell, the standard in this case is what POSIX specifies, and it doesn't know about \d. Though you're right in that PCRE are rather standard, or in the least well-defined. The annoying issue is that GNU grep (or glibc) supports some PCRE-like atoms, at least \w and \s when interpreting ERE, and in that context they very much are nonstandard. My phrasing may well come partly from that, and the misrecollection that \d was similarly GNU-supported. – ilkkachu Sep 19 '19 at 19:31

Stéphane Chazelas · Answer 2 · 2018-02-02T16:12:34.213

(:...) and \d are perl or PCRE regular expression operators (like in GNU grep -P).

bash only supports extended regular expressions as in grep -E except that for regexps passed literally as in [[ text =~ regexp-here ]] as opposed to as the result of an unquoted expansion (as in [[ text =~ $var ]] or [[ test =~ $(printf '%s\n' 'regexp-here') ]]), it's limited to the POSIX extended regular expression feature set.

So even on systems where the grep -E '\d' would work (GNU EREs have already imported some extensions from perl regexps like \s so future versions might have \d as well), you'd have to use:

regexp='\d'
[[ $text =~ $regexp ]]

in bash for it to work ([[ $text =~ \d ]] wouldn't).

For a shell that supports PCREs, you may want to use zsh instead:

set -o rematchpcre
[[ $text =~ '(?:\d{1,3}\.)+(?:\d{1,3})' ]]

ksh93 also supports its own implementation of perl-like regular expressions (not fully compatible) as part of its pattern matching. There, you'd use:

regexp='~(P)(?:\d{1,3}\.)+(?:\d{1,3})'
[[ $text = $regexp ]]

(note the = instead of =~. You'll want to use temporary variables as the it is very buggy when you don't)

Do not put quotes around regex string. The quotes around the right-hand side of the =~ operator cause it to become a string, rather than a RegularExpression. If you want to use a long or complicated regular expression and avoid lots of backslash escaping, put it in a variable. Source: https://mywiki.wooledge.org/BashPitfalls#if_.5B.5B_.24foo_.3D.2BAH4_.27some_RE.27_.5D.5D — Artfaith, Apr 08 '21 at 08:29

score 1 · Answer 3 · answered Feb 02 '18 at 18:37

The site regex101.com use PCRE (look at the upper left corner) as the default, and it lacks support for "Extended" regex syntax. That is "Perl Compatible Regular Expresions", which come (as is reasonable to expect) from Perl.

PCRE is supported by some tools (like grep -P) under some conditions, but the bash regex support inside the [[…]] idiom is only for extended regex (like grep -E).

In Extended regex, the non-capture (?…) parenthesis does not exist, and the \d is also missing. You need to use simple (…) and [0-9]:

regexp="([0-9]{1,3}\.)+([0-9]{1,3})"

Bash =~ regex and https://regex101.com/

3 Answers3

Linked