Finding files and matching patterns with printed filename in bash

Question

I have the following code that lists matches far a pattern ptrn, with the filename being printed before the listing (which uses the context option -C NUM)

find "$fdir" "${isufx[*]}" -type f -exec bash -c  \
  "grep --color -l '$ptrn' '{}'; grep --color -ni ${ictx[*]} '$ptrn' '{}'" \;

I agree it's a monster. I decided to remove the bash -c call, resulting in

  OFS=$IFS
  IFS=$'\n'
  for f in $(find "$fdir" ${isufx[*]} -type f); do
    grep -l "$ptrn" "$f" && grep -ni ${ictx[*]} "$ptrn" "$f"
  done
  IFS=$OFS

Any suggestions on the above? I would like to print the filename before the listing, enclosed between ==> and <==, with an empty line above and below the filename.

After suggestions about avoiding looping over find's output, I have:

  find "$fdir" ${isufx[*]} -type f |
    while read f; do
      grep -l "$ptrn" "$f" && grep -ni ${ictx[*]} "$ptrn" "$f"
    done

That will fork one instance of bash and two of grep (both grepping the same file) for every single file found by find. That's kind of horrifying. Replace it with an awk or perl script that does the pattern matching, the colourisation, and the context lines. Hint: for the context, the script will have to remember NUM lines before the current line, just in case it matches and then print those lines, the current line and the next NUM lines). Using find ... -exec ... + instead of \;, that will fork one process total (not 3 per file), which will make only one pass through all files. — cas, Aug 01 '21 at 12:22
For extra fun, have the script parse $GREP_COLORS (plural), or the older/simpler/deprecated $GREP_COLOR (singular), environment variables if they exist, otherwise default to the same colours as grep. See man grep and search for the section titled "ENVIRONMENT" for details. — cas, Aug 01 '21 at 12:25
Don't use a for loop with find like that. See Why is looping over find's output bad practice? — cas, Aug 01 '21 at 13:59

cas · Accepted Answer · 2021-08-02T01:11:32.943

I'm pretty sure you're not going to like this answer, but here's a correct way of doing this (not the correct way, but one of many). It's here as an example for those who want to know that it's not terribly difficult to write your own custom tools when the standard tools, like grep, don't do quite what you want them to do - this is how unix is meant to be used. (and also because I wanted to find out how difficult it would be to parse and use grep's GREP_COLORS variable...trivially easy, as it turned out)

It's a script that only needs to be forked once by find ... -exec and does everything in one pass through each file, rather then multiple passes. It handles any valid filename, even those containing newlines and other white space.

Save the perl script below as, e.g., context-grep.pl and make it executable with chmod +x context.pl. Then run it like so:

find "$fdir" "${isufx[*]}" -type f -exec ./context-grep.pl {} +

It will use the following environment variables:

$ptrn for the pattern to search for
$NUM for the number of context lines to print (optional, default is 3)
$GREP_COLOR or $GREP_COLORS to use the same colour code as grep (optional, defaults to green).

These can, as usual, be specified on the command line, e.g.

NUM=5 ptrn='foo.*bar' find "$fdir" "${isufx[*]}" -type f -exec ./context-grep.pl {} +

Proper option handling for the script could be done with one of perl's many option processing modules (like Getopt::Std or Getopt::Long) but this script is already getting too long for this site. The whole thing could have been written in perl without needing find with the File::Find module. All three of these modules are core perl library modules and are included with perl.

#!/usr/bin/perl
use strict;
This script should use TERM::TERMCAP to get the actual
colour codes for the current $TERM from the terminfo
database, but I'll just hard-code it to use ANSI colour
codes because almost everything is ansi-compatible these days.
That's good enough for grep, so it's good enough for this.

variable setup and related stuff

my $sgr0 = "\033[m\017";
my $colour = "\033[01;32m"; # default to green
If either of grep's colour env vars are defined, use
them instead. (the newer $GREP_COLORS is checked last,
so has precedence over $GREP_COLOR)
if ($ENV{'GREP_COLOR'}) {
  $colour = "\033[$ENV{'GREP_COLOR'}m";
};
if ($ENV{'GREP_COLORS'}) {
e.g. ms=01;31:mc=01;31:sl=:cx=:fn=35:ln=32:bn=32:se=36
This script really only cares about the ms value
It wouldn't be hard to make it use mc as well to print the
context lines in a different colour than the match line.
my @GC = split /:/, $ENV{'GREP_COLORS'};
  foreach (@GC) {
    if (m/^ms/) {
      my (undef,$c) = split /=/;
      $colour = "\033[${c}m";
      last;
    }
  };
};
my $search=$ENV{'ptrn'};
my @context;
my $NUM=3; # default to 3 lines of context
$NUM = $ENV{'NUM'} if (defined($ENV{'NUM'}));
my $last = -1;
my $first_match=1;

main loop, process the input file(s)

while(<>) {
  chomp;
if ($. <= $last) {
    # current line is an AFTER context line, print it
    printf "%s%s%s\n", $colour, $_, $sgr0;
} elsif (m/$search/) {
    # We've found a match! handle it.
# print filename like head &amp; tail does if this is the
# first match we've found in the current file.
if ($first_match) {
  printf &quot;\n==&gt; %s &lt;==\n\n&quot;, $ARGV;
  $first_match=0;
};

# print the remembered BEFORE context lines
foreach my $l (@context) {
  printf &quot;%s%s%s\n&quot;, $colour, $l, $sgr0;
};

# print current line
printf &quot;%s%s%s\n&quot;, $colour, $_, $sgr0;

# clear the context array 
@context=();

# set $last so we can print the AFTER context lines
$last = $. + $NUM;


} else {
    # remember the last $NUM lines of context
    push @context, $_;                     # add current C line
    shift @context if ($#context >= $NUM); # remove first C line
  };
reset $last, $first_match, and the input record counter
($. - equivalent to awk's NR) on every EOF
if (eof) {
    close(ARGV);
    $last = -1;
    $first_match=1;
  };
};

BUGS: there's absolutely no error handling. Or option processing. or help/usage message. or POD documentation. These are left as an exercise for the reader.

Sample output (the matched line and 2 lines of context around pattern "chomp" from the script itself):

$ NUM=2 ptrn=chomp find . -type f -name '*.pl' -exec ./context-grep.pl {} +
==> ./context-grep.pl <==
while(<>) {
  chomp;
if ($. <= $last) {

I have export GREP_COLOR='0;33' in my ~/.bashrc so the matched line and context lines are printed in yellow. The filename is printed in the terminal's default text colour (white on a black background).

score 0 · Answer 2 · answered Aug 01 '21 at 18:36

0

I put another solution that uses grep without find.

echo ""
grep -rl ${isufx[@]} "$ptrn" $fdir |
  while read f; do
    echo -e $(tput setaf 46)"==> $f <==\n"$(tput sgr0)
    grep -ni ${ictx[@]} "$ptrn" "$f"
    echo ""
  done

answered Aug 01 '21 at 18:36

Pietru

389
1
17

Finding files and matching patterns with printed filename in bash

2 Answers2

This script should use TERM::TERMCAP to get the actual

colour codes for the current $TERM from the terminfo

database, but I'll just hard-code it to use ANSI colour

codes because almost everything is ansi-compatible these days.

That's good enough for grep, so it's good enough for this.

variable setup and related stuff

If either of grep's colour env vars are defined, use

them instead. (the newer $GREP_COLORS is checked last,

so has precedence over $GREP_COLOR)

e.g. ms=01;31:mc=01;31:sl=:cx=:fn=35:ln=32:bn=32:se=36

This script really only cares about the ms value

It wouldn't be hard to make it use `mc` as well to print the

context lines in a different colour than the match line.

main loop, process the input file(s)

reset $last, $first_match, and the input record counter

($. - equivalent to awk's NR) on every EOF