pipe commands inside find -exec?

Question

Let's suppose I want to find all .txt files and search for some string. I would do:

find ./ -type f -name "*.txt" -exec egrep -iH 'something' '{}' \;

What if I want to do a more complex filtering, like this:

egrep something file.txt | egrep somethingelse | egrep other

Inside find -exec? (or similar)

Please keep in mind that I'm searching for a solution that I could easily type when I need it. I know that this could be done with a few lines using a shell script, but that isn't what I'm looking for.

score 20 · Accepted Answer · edited Mar 18 '19 at 17:20

20

If you must do it from within find, you need to call a shell:

find ./ -type f -name "*.txt" -exec sh -c 'grep -EiH something "$1" | grep -E somethingelse | grep -E other' sh {} \;

Other alternatives include using xargs instead:

find ./ -type f -name "*.txt" | 
    xargs -I{} grep -EiH something {} | 
        grep -EiH somethingelse | 
            grep -EiH other

Or, much safer for arbitrary filenames (assuming your find supports -print0):

find ./ -type f -name "*.txt" -print0 | 
    xargs -0 grep -EiH something {} | 
        grep -Ei somethingelse | 
            grep -Ei other

Or, you could just use a shell loop instead:

find ./ -type f -name "*.txt" -print0 | 
    while IFS= read -d '' file; do 
        grep -Ei something "$file" | 
            grep -Ei somethingelse | 
                grep -Ei other
    done

edited Mar 18 '19 at 17:20

Kusalananda

333,661

answered Mar 18 '19 at 16:51

terdon

242,166

2

The first one is exactly what I was looking for. Extremely simple and small enough to type depending on my needs. Thanks. – Foreign Mar 18 '19 at 16:57
3

... and xargs could also be used as xargs -I {} sh -c '...' sh {}, if one wanted to (it makes it possible to run parallel jobs with -P if one wanted to). – Kusalananda Mar 18 '19 at 17:23
I find the first one working for me but I'm confused why there's an extra sh in the very end as in ... sh {} \;. Would you mind clarifying? @terdon – Boson Bear Mar 31 '22 at 14:03
@BosonBear because that will become $0 in the sh script, you can use any arbitrary string. See https://unix.stackexchange.com/a/389706/22222 – terdon Mar 31 '22 at 17:03

trobinson · Answer 2 · 2019-03-20T06:43:59.780

3

Edit: This answer is not preferred, but is left here for comparison and illustration of potentially dangerous pitfalls in bash scripting.

You can put bash (or another shell) as your -exec command:

find -type -f -name "*.txt" -exec bash -c 'egrep -iH something "{}" | egrep somethingelse | egrep other' \;

One of the downsides of doing it this way is that it creates more potential for nested quoting issues as your commands get more complex. If you want to avoid that, you can break it out into a for-loop:

for i in $(find -type -f -name "*.txt"); do
  if egrep -iH something "$i" | egrep somethingelse | egrep other; then 
    echo "Found something: $i"
  fi
done

edited Mar 20 '19 at 06:43

answered Mar 18 '19 at 16:55

trobinson

94

1

The first one is exactly what I was looking for. Extremely simple and small enough to type depending on my needs. Thanks. – Foreign Mar 18 '19 at 16:59
2

That for loop is a very bad idea.Also known as bash pitfall #1. – terdon Mar 18 '19 at 17:04
2

This "{}" in your first command may even lead to code injection. Imagine you got files from me and there's a file literally named " & rm -rf ~ & : ".txt. Luckily for you -type -f is invalid, it just saved your home directory. Fix the typo and try again. :) terdon did it right: find … -exec sh -c '… "$1" …' foo {} \;. – Kamil Maciorowski Mar 18 '19 at 17:55
Thanks for the information! Yeah, the -type -f is a typo I make constantly when using find, and I didn't notice it in my answer. Whoops. terdon's answer is better, but I'll leave this for comparative purposes. – trobinson Mar 18 '19 at 20:20
@terdon: tx for referencing the mywiki.wooledge.org page. It's nice to have a bunch of GPs neatly summarized in one place. – Cbhihe Mar 19 '19 at 08:03
To improve site readability & ease of use, e.g. by new users, do consider at least adding a note at the top of yr post (by editing it) to signal the fact that yr for ... done loop-based answer is not preferred and left for reason "XYZ".... If not unwitting visitors might credit that particular post with upvotes (as seen already). I've seen many SE visitors not read the fine prints (i.e. the comments following an answer). They just stop at the nbr of upvotes. Upvotes are often seen as a seal of approval by some, who tend to accept whatever has apparently been positively sanctioned. – Cbhihe Mar 19 '19 at 08:17

pipe commands inside find -exec?

2 Answers2

Linked