5

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes.

tail -f general.log | grep Some_word > >(tee -a todel.txt)

tail -f general.log | grep Some_word ; tee  todel.txt

tail -f general.log | grep Some_word | tee -a todel.txt
Gilles 'SO- stop being evil'
  • 829,060
shantanuo
  • 143
  • Perhaps "Some_word" is case sensitive? In which case you should use grep -i. – sandyp Nov 18 '13 at 21:25
  • Are you sure, the required lines really appear there? By default tail -f will give you last 10 lines of input and whatever is written there unless the file is removed and recreated, in which case it quits. GNU tail has the --retry option which will wait until the file is recreated (although I think this option is non-POSIX). – peterph Nov 25 '13 at 23:15

4 Answers4

8

You need to add stdbuf(1) into your pipeline:

tail -f general.log | stdbuf -oL grep Some_word | tee -a todel.txt

This will set grep's stdout stream buffering mode to be line-buffered, otherwise grep waits to get at least 4096 bytes from the stream (this is the default on Linux for buffered i/o).

Alternatively, you can also call grep with --line-buffered:

tail -f general.log | grep --line-buffered Some_word | tee -a todel.txt

See Turn off buffering in pipe and http://www.pixelbeat.org/programming/stdio_buffering/ for in-detail explanations.

Community
  • 1
tkrennwa
  • 3,525
  • 1
  • 15
  • 17
1

The third one should be working fine assuming that you have access to write files in the current directory.

For example: tail -f general.log | grep "Some_word" | tee todel.txt

That is properly-defined tee syntax. If it's not working, you're doing something else wrong.

Also, it's a good idea to put your search string in quotes. If you have a filename matching your search string in the current working directory, it may be considered a file argument instead of a search string.

Magellan
  • 576
  • I think you mean tee instead of tail in your command line example. – dmourati Nov 18 '13 at 06:23
  • Sure enough. Thanks. What I get for multi-tasking. – Magellan Nov 18 '13 at 06:42
  • Using tail -f ... |grep allows one to grep the new lines incoming (-f option to tail). AFAIK grep has no such option. – zhenech Nov 18 '13 at 07:34
  • You're right. Wasn't paying attention. – Magellan Nov 18 '13 at 16:33
  • 1
    Consider a text file with only a few characters in it. If you run tail -f... and pipe it into the grep | tee machinery, you would end up in exactly the scenario of the question: grep would wait for more input before processing it, while tee does not receive anything at all. Of course, if general.log is sufficiently large, the standard approach without altering the output buffer mode to line-buffered would work, but only if you it larger than the buffer size. And then you would need to wait for filling up grep's buffer again. See http://unix.stackexchange.com/a/102505/50954 – tkrennwa Nov 25 '13 at 12:50
  • Good stuff. Thanks. had no idea that the default buffer was that large. – Magellan Nov 25 '13 at 16:51
0
grep "my words" > mytextfile.txt
tail -f access.log > last_log.txt
tail -f access.log | grep "IP address" > hacker.log
Ilja
  • 130
0

It's possible the lines you're looking for are being sent to stderr instead of stdout, in which case ending the line with 2>&1 should fix the problem:

tail -f general.log | grep Some_word | tee -a todel.txt 2>&1
Matthew Cline
  • 3,355