Redirect program output to file with line number limit

Question

Is it possible to redirect program output to file, but with maximum lines limit, for example 4 lines limit:

./program | save 4 out.txt

The ./program will continue giving output (maybe one line or more) every few seconds, but only 4 most recent/last lines will be saved to out.txt. Is there already a save program that could do this? (or I should make it manually)

Answer 1

This could be a good job for the shell's read builtin - as long as your input is text, that is, and doesn't contain any \0NULs. I say this because, though it is often terribly inefficient for file work when compared with other standard utilities, using a shell builtin would likely be better than repeatedly forking another process. And you won't get much efficiency anyway because the only way to do this (that I know of) is to repeatedly open() your output file - unless you can be very sure of the number of bytes per line (which might be assured with a few chained dds in a pipeline, I guess). In any case, the following will work:

###seq used for demo###
seq 3 |( set --
while IFS= read -r l
do    set -- "$@" "$l"
      shift "$(($#>4))"
      printf %s\\n "$@" >/tmp/4_lines
###demo commands###
      printf '\n###4_lines CONTENTS###\n'
      cat </tmp/4_lines
      printf '###END###\n'
###end demo###
done)

So if I do the above as written this is written to the while loop's stdout:

###4_lines CONTENTS###
1
###END###

###4_lines CONTENTS###
1
2
###END###

###4_lines CONTENTS###
1
2
3
###END###

But if I hand seq 20, for example, it prints the above then:

###4_lines CONTENTS###
1
2
3
4
###END###

###4_lines CONTENTS###
2
3
4
5
###END###

...all the way up to...

###4_lines CONTENTS###
16
17
18
19
###END###

###4_lines CONTENTS###
17
18
19
20
###END###

It will go on like that until the input pipe is closed - just round-robining its arg array and overwriting /tmp/4_lines with the array's contents each time an input line is read. If you wanted the lines in reverse order - so the first line is the last line read in, you could change the printf line to:

printf %s\\n ${4+"$4"} ${3+"$3"} ${2+"$2"} "$1" >/tmp/4_lines

...which would print like...

###4_lines CONTENTS###
1
###END###

###4_lines CONTENTS###
2
1
###END###

###4_lines CONTENTS###
3
2
1
###END###

...through...

###4_lines CONTENTS###
19
18
17
16
###END###

###4_lines CONTENTS###
20
19
18
17
###END###

...without risking any difficulties with $IFS and/or unintentional globs on the expansion.

Answer 2

I think you can do what you're asking by running ./program | tail -n 4 > out.txt. If not, then I'm not understanding what you're asking.

Answer 3

The save program could be something like this:

#include<string>
#include<iostream>
#include<fstream>
#include<cstdlib>
using namespace std;
int main(int argc,char *argv[]) {
  if(argc<=2) {
    cerr << "First param must be a number, second must be a file" << endl;
    return -1;
  }
  int max = atoi(argv[1]);
  if(max <= 0) return -2;
  string lines[max];
  int cmax = 1;
  int z = 0;
  while(true) {
    getline(cin,lines[z]);
    ++z;
    z %= max;
    ofstream fo(argv[2]);
    for(int x=0;x<cmax;++x) {
      fo << lines[(x+cmax-z)%cmax] << endl;
    }
    fo.close();
    if(cmax<max) ++cmax;
  }
}

Compile using g++ save.cpp -o save