grep and tail -f?

Question

Is it possible to do a tail -f (or similar) on a file, and grep it at the same time? I wouldn't mind other commands just looking for that kind of behavior.

score 105 · Accepted Answer · edited Apr 05 '17 at 09:17

105

Using GNU tail and GNU grep, I am able to grep a tail -f using the straight-forward syntax:

tail -f /var/log/file.log | grep search_term

edited Apr 05 '17 at 09:17

Jeff Schaller

67,283
35
116
255

answered Oct 18 '10 at 03:16

Steven D

46,160

1

This is a solution that works with other implementations of these two utilities, not just the GNU implementation. – Kusalananda Feb 15 '17 at 19:02
tail -F (capital f) will also follow new file created (if file is cycled). -f (small f) will only follow, not trace new cycled files. – Koby Aug 15 '21 at 15:36
This worked for me using the Bash shell, but not using the Fish shell – Flimm Aug 15 '23 at 07:48

score 29 · Answer 2 · edited Feb 05 '13 at 17:01

29

Add --line-buffered to grep, and that may reduce the delay for you. Very useful in some cases.

tail -f foo | grep --line-buffered bar

edited Feb 05 '13 at 17:01

George M

13,959

answered Feb 05 '13 at 16:43

Digit

391

2

That's useful when the output of grep doesn't go to a terminal (redirected to another type of file). line buffering is the default when the output goes to a terminal, so it won't make any difference there. Note that that option is GNU specific. – Stéphane Chazelas Feb 13 '14 at 14:16
2

I believe it is in the tail's output where the buffering causes delays. I use stdbuf utility to invoke it as stdbuf -o0 tail -f foo | grep ..... Same can be applied to grep e.g. if it is being piped to another program, e.g. stdbuf -o0 tail -f foo | stdbuf -i0 -o0 grep bar | another_program – AnyDev Mar 22 '21 at 00:06
1

This will also make a difference when piping grep to yet another grep, and is thus very useful! – phil294 Dec 12 '21 at 07:03

score 11 · Answer 3 · answered Oct 18 '10 at 03:22

It will work fine; more generally, grep will wait when a program isn't outputting, and keep reading as the output comes in, so if you do:

$ (echo foo; sleep 5; echo test; sleep 5) | grep test

Nothing will happen for 5 seconds, then grep will output the matched "test", and then five seconds later it will exit when the piped process does

score 9 · Answer 4 · edited Apr 05 '17 at 09:45

I see all these people saying to use tail -f, but I do not like the limitations of that! My favorite method of searching a file while also watching for new lines (e.g., I commonly work with log files to which are appended the redirected output of processes executed periodically via cron jobs) is:

 tail -Fn+0 /path/to/file|grep searchterm

This assumes GNU tail and grep. Supporting details from the tail manpage (GNU coreutils, mine is v8.22) [https://www.gnu.org/software/coreutils/manual/coreutils.html] :

 -F     same as --follow=name --retry
 -n, --lines=K
         output the last K lines, instead of the last 10; or use -n +K to output
         starting with the Kth.
         If  the first character of K (the number of bytes or lines)
         is a '+', print beginning with the Kth item from the start
         of each file, otherwise, print the last K items in the file.
         K may have a multiplier suffix: b 512, kB 1000, K 1024, MB
         1000*1000, M 1024*1024, GB 1000*1000*1000, G 1024*1024*1024,
         and so on for T, P, E, Z, Y.
With --follow (-f), tail defaults to following the file descriptor,
  which means that even if a tail'ed file is renamed, tail will
  continue to track its end.  This default behavior is  not  desirable
  when  you  really  want  to  track the actual name of the file, not
  the file descriptor (e.g., log rotation).  Use --follow=name in
  that case.  That causes tail to track the named file in a way that
  accommodates renaming, removal and creation.

So, the tail portion of my command equates to tail --follow --retry --lines=+0, where the final argument directs it to start at the beginning, skipping zero lines.

Gilles 'SO- stop being evil' · Answer 5 · 2023-11-20T11:50:20.580

5

You can just pipe the output of tail -f into grep. There are also programs that combine tail -f functionality with filtering and coloring, in particular multitail (examples).

edited Nov 20 '23 at 11:50

answered May 15 '11 at 13:09

Gilles 'SO- stop being evil'

829,060

@Cookie Oh, of course. I meant “pipe tail -f into grep” or “run grep on the output of tail -f”. Weird that nobody caught this in so many years! – Gilles 'SO- stop being evil' Nov 20 '23 at 11:50

score 2 · Answer 6 · answered Feb 07 '11 at 16:32

2

tail -f access | awk '/ADD/{print $0}'

Use the above, i use it usually.

answered Feb 07 '11 at 16:32

mouviciel · Answer 7 · 2010-10-20T04:56:13.100

0

It works. But take care that output is no longer instantaneous: it gets buffered through the pipe.

edited Oct 20 '10 at 04:56

answered Oct 19 '10 at 12:13

mouviciel

1,235

Agreed. To test this, open two windows. Run tail -f in one window, and tail -f logfile | grep pattern in the other window. Lines containing pattern don't always appear in both windows at the same time. I've seen lines appear 30 seconds apart on rare occasions, which was annoying. – Stefan Lasiewski Oct 19 '10 at 19:46
That might have more to do with the system flipping between running one or the other instance of tail. A better test would us tee or something. – Kevin Cantu Oct 19 '10 at 22:08

score 0 · Answer 8 · answered Oct 19 '10 at 22:23

You can use netcat to grep the results of tail -f as new results come in quite easily.

sudo nc -s localhost -l -p 1337 | grep ssh


tail -f /var/log/file.log | nc 127.0.0.1 1337

This sets grep to listen to results for input coming from port 1337.
The second command pipes the output of tail -f to netcat and sends it out localhost 1337. To do it locally you need to switch ttys for each of the two sets of commands, or use something like screen.

grep and tail -f?

8 Answers8

Linked