Generate strings from regular expression in bash

Question

How in Bash to generate a string given by a regex expression and pipe it to another command?

For example:

> $RANDOM | regex "{abcdef\d}[8]" | grep "1{3}" | less

I did not find which utility can generate strings based on a regex expression special

The regex expression is the input. I want to find a utility that generates strings based on regex.

For example:

$ regex "{abcdef\d}[8]" -n 10
b8789a62
97303666
8b536c28
79590607
6d80ad60
78d36ded
aa001d9a
8826f276
df1e1f7a
854307db

two limits:

the number of times a repetition operator is allowed to generate a given text is limited by some internal (invisible) number
the number of generated strings are limited by a number given by the user.

I don't know of any existing tools which can do this, but there are plenty of libraries which can do this, so you could use one that works with Python or Perl to do the job — muru, Jan 13 '23 at 14:35
Thanks, but isn't there something built into bash? Possibly ineffective, with brute force. I can't install libraries. — Aycon, Jan 13 '23 at 14:41
Regular expressions are strictly used in the opposite direction: you already have a string and you need to check that it matches the pattern. Regular expressions are not applicable for generating strings. — glenn jackman, Jan 13 '23 at 14:57

Kusalananda · Answer 1 · 2023-01-13T14:53:34.270

2

To get a stream of lines, each with eight lower-case hexadecimal digits, you may use

tr -c -d '0-9a-f' </dev/random | fold -w 8

This reads from /dev/random and removes all characters we're not interested in using tr. fold is then used to cut the stream up into lines of eight characters.

Modify the tr selection expression to select any other set of characters. Modify the fold length to get longer or shorter strings. Add a grep stage to the pipeline if you want to pick out strings that fulfil other criteria from the generated ones, for example, use grep 111 to only allow strings that contain three consecutive 1.

edited Jan 13 '23 at 14:53

answered Jan 13 '23 at 14:37

Kusalananda

333,661

How about any regular expression? – Aycon Jan 13 '23 at 14:39
1

@Aycon, regex have different role, to map input string , not to generate something. – Romeo Ninov Jan 13 '23 at 14:44
@Aycon "Any regular expression" does not make sense. You will need to explain a bit further what it is you want to do. It is highly unusual that the regular expression is a parameter in a problem. Is the question really about generating all possible strings that match a given expression, because that seems to be what you are asking? – Kusalananda Jan 13 '23 at 14:47
like this, but in bash terminal: https://www.browserling.com/tools/text-from-regex – Aycon Jan 13 '23 at 14:51
@Aycon, add on the end of the line (after fold command) |grep "{abcdef\d}[8]" and this will filter only strings which map this regex – Romeo Ninov Jan 13 '23 at 14:55
1

@Aycon Note that the web-based tool that you link to in a previous comment has at least two limits: 1) the number of times a repetition operator is allowed to generate a given text is limited by some internal (invisible) number, and 2) the number of generated strings are limited by a number given by the user. In your question you give no limitations whatsoever. If you can give example and pose reasonable limits on things that obviously needs to be limited (like what strings a*b should generate), then this might be an interesting programming exercise. – Kusalananda Mar 22 '24 at 12:20
(cont.) As this would be a possible programming exercise, it's probably not suited for this site, but more in line with the type of questions that StackOverflow may see (given that it is rewritten to highlight what issues you have with its implementation). The answers here will focus on the actual problem, which currently seems to be the generation of randomized hexadecimal numbers of a certain length. – Kusalananda Mar 22 '24 at 12:22
@Kusalananda Yes, I think it would be fair to indicate the appropriate limitations in the question. – Aycon Mar 23 '24 at 09:14
@Kusalananda I don't agree that the issue is programming exercise. The question is whether to point to an existing utility, or to find a workaround to implement this. I gave this regular expression as an example to clarify and specify the question. – Aycon Mar 23 '24 at 09:20
@Kusalananda (cont.) My goals require generating strings for any regular expression. I also accept that restrictions will be required, and that generation may not be done in the most optimal way. – Aycon Mar 23 '24 at 09:21

Stéphane Chazelas · Accepted Answer · 2024-03-22T11:29:21.427

For your particular example of just printing random 32 bit hex numbers, you might as well use od or hexdump on /dev/urandom:

$ od -w4 -An -vtx4 -N40 /dev/urandom
 f0c37f96
 e5dc1143
 ebc9cd59
 7fe573da
 3622c662
 e7bc0013
 5f9e2017
 79a6ccff
 4dd1d41d
 a6957940
$ hexdump -e '"%08x\n"' -n40 /dev/urandom
9f739176
e13a816f
0860c351
42bf86e7
c810ab07
1f42e7b9
d76b7016
10d2ac2c
b5ae6ca5
28089547

Or since you mention bash builtins in comments:

for ((i=0;i<10;i++)) {
  printf "%08x\n" "$((RANDOM<<17^RANDOM<<2^RANDOM))"
}

(that syntax actually coming from ksh (though ksh initially didn't have printf builtin), and is available in zsh and bash as well).

$RANDOM is a 15 bit pseudo-random number. Newer versions of bash also have $SRANDOM as a 32 bit integer which is also more random, so in those, you may prefer:

for ((i=0;i<10;i++)) {
  printf "%08x\n" "$SRANDOM"
}

Now, as a more direct answer to your question, for such a simple regexp, you can use the String::Random module of perl (in a libstring-random-perl package in Debian-based distributions):

$ perl -MString::Random=random_regex -le 'print random_regex("[0-9a-f]{8}") for 1..10'
e785f800
264a131c
c7bcb74e
26d6c169
d5e169df
210ac1c0
616e651d
badf8131
50c15bb5
93e2b2a5

(note that as noted in the documentation, that's not real regexps; doing this thing for arbitrary regexps would be quite hard and not particularly useful).

Generate strings from regular expression in bash

2 Answers2