How to output result of a find command that excludes certain file extension

Question

Here is my find command:

find -mtime -$a | xargs zgrep -il ''$b'' | xargs zgrep -il ''$c'' | xargs zgrep -il ''$d'' | xargs zgrep -il 'ST.997' | sort -u

This is an example output of the above command:

example_file.dat.Z
example2_file.dat.Z
example2_file.dat.bak.1234567890.Z
example3_file.dat.Z
example3_file.dat.bak.Z
example4_file.dat.Z
example4_file.dat.bak.qwertyuiop.Z

As you see, it outputs the same files with ".bak" extension in it. What I want is to exclude/not show that file whenever I run my script. Like this example output:

example_file.dat.Z
example2_file.dat.Z
example3_file.dat.Z
example4_file.dat.Z

EDIT: I forgot. Sometimes it is in zip file.

can you post those variables $a , $b etc values? I suppose, that long pipeline can be optimized — RomanPerekhrest, Aug 22 '17 at 08:50
You want to use actual double quotes around your variables, not 2 single quotes: use "$b" not ''$b'' — glenn jackman, Aug 22 '17 at 08:52
@glennjackman I did that but BASH reads it as a string not a variable. It's fine that way. It's not the issue. — WashichawbachaW, Aug 22 '17 at 09:08
@WashichawbachaW It would interpret it as a string (as $b) if you put it in single quotes. The shell would expand the variable to its value if you put it in double quotes. — Kusalananda, Aug 22 '17 at 09:15
Oh thank you. It's working. I don't know why it didn't work before. That's why I used that single quote. — WashichawbachaW, Aug 23 '17 at 03:52
@Wildcard considering your suggestion. I'm just new to Linux and shell scripting. It's the only thing I know for now to run my command. If you have better codes or improvements in my code, I would greatly appreciate it, or if you have references that I could read and learn to improve my script, that would be a very big help. — WashichawbachaW, Aug 23 '17 at 07:34
@WashichawbachaW the answer below by Kusalanda shows how to do it right, for reasons explained in my own answer to the "find looping" question I linked above. Regarding references for shell scripting improvement, check my profile—I have a list there. It doesn't take years of practice to script well, just a keen interest and willingness to learn. Be great :) — Wildcard, Aug 23 '17 at 08:39

Kusalananda · Answer 1 · 2017-08-23T06:05:37.967

Here's my take on this:

$ find . \( -name '*.bak*' -prune \) -o \
      -type f -mtime "-$a" \
      -exec zgrep -iq -e "$b" {} ';' \
      -exec zgrep -iq -e "$c" {} ';' \
      -exec zgrep -iq -e "$d" {} ';' \
      -exec zgrep -iq 'ST.997' {} ';' \
      -print | sort

or

$ find . ! -name '*.bak*' \
      -type f -mtime "-$a" \
      -exec zgrep -iq -e "$b" {} ';' \
      -exec zgrep -iq -e "$c" {} ';' \
      -exec zgrep -iq -e "$d" {} ';' \
      -exec zgrep -iq 'ST.997' {} ';' \
      -print | sort

Each "predicate" given to find will act as a "test" on the things that find finds. If one test fails, the next predicate will not be evaluated (there's an implicit "logical AND" between each, unless otherwise stated) and the next found name is considered instead.

The $ -name "*.bak" -prune $ bit means "if the name matches the given pattern, remove it from the search results". This wole line may be replaced by ! -name '*.bak*'.
-o means "logical OR".
-type f will be true for regular files only.
-mtime "-$a" will be true for files that have a modification timestamp less than $a 24h-periods ago.
Each -exec zgrep -q ... {} ';' will be true if the given pattern was matched in the file, or rather, if zgrep exits with a zero exit status (which zgrep -q does if a match is found, but it won't produce any output). The -e flag to zgrep will force the values of the given variables to act as patterns (this matters if any of them starts with -).
-print will output the name (this predicate is always "true").

If you are unwilling to change the whole command, a hacked fix would be to simpy pass the result of your original command through grep -v '\.bak' or grep -vF '.bak', but as your command currently stands it will have issues with files that have exotic filenames (names containing newlines, for example).

@WashichawbachaW Fair enough. Then just go with what I wrote at the end: pipe your original command through grep -vF '.bak'. — Kusalananda, Aug 22 '17 at 09:05
There is no purpose to the -u flag in sort -u here. I don't think there's any purpose to the sort command at all here, but I know there's no purpose to the -u flag. Still, +1 for doing it right (i.e. using find instead of long xargs pipelines). — Wildcard, Aug 23 '17 at 04:25
@Wildcard I was concentrating on the find bits and didn't even consider the sort, but you're absolutely right; There is no reason to do a sort -u since there will be no duplicated file paths. — Kusalananda, Aug 23 '17 at 05:59

WashichawbachaW · Accepted Answer · 2017-08-23T03:54:43.157

-1

I finally found the answer. After long hours of experiments. haha

Here it is:

find -mtime -$a -type f ! -name "*.bak*" | xargs zgrep -il "$b" | xargs zgrep -il "$c" | xargs zgrep -il "$d" | xargs zgrep -il 'ST.997' | sort -u

edited Aug 23 '17 at 03:54

answered Aug 22 '17 at 09:03

WashichawbachaW

367

2

that long pipeline seems to be redundant – RomanPerekhrest Aug 22 '17 at 09:03
What do you suggest? $a, $b, $c, and $d are variables with different values. – WashichawbachaW Aug 23 '17 at 03:46
regex alternation group – RomanPerekhrest Aug 23 '17 at 04:59
@RomanPerekhrest The problem here is that something like ($a|$b|$c|$d) would have to match on one single line whereas chaining multiple greps together allows for matching the expressions on different lines, which I have assumed is what the OP wanted to do. – Kusalananda Aug 23 '17 at 06:03
Those variables are all needed to find a specific file. If one variable doesn't exist on any file, the command won't output anything. It must be all true for the command to output a file. Yeah it's a little long and redundant, but it's the only thing I know for now. I'm only new to Linux. However, if you @RomanPerekhrest have better code than mine, I would greatly appreciate the help. – WashichawbachaW Aug 23 '17 at 07:07

How to output result of a find command that excludes certain file extension

2 Answers2

Linked