0

I want to monitor/follow changes of a CSV file (some logging output of a robot). It is a huge file with <TAB> as delimiter and the first row with header information.

So far, I am using the output of tail - f <file.csv>, but it is not formatted per column, and looks like:

yAccelRaw       zAccelRaw       xGyroRaw    ... ... ...
3       256     1   ... ... ...
4       255     3   ... ... ... 
4       255     -6  ... ... ...
3       253     -1  ... ... ...
4       254     2   ... ... ...
5       255     0   ... ... ...
4       255     3   ... ... ...
5       254     3   ... ... ...
5       253     -1  ... ... ...
4       255     3   ... ... ...

With a CSV with 30 or more columns, understanding which value belongs to which column is not really easy. I was wandering if there is a general solution for printing the output formatted as table?

So far, I use tail -f <file.csv> | cut -f5,6 to cut out specific columns and observe their output, but I would prefer the full overview. Also, I tried piping the result to column which doesn't get updated.

Marcus
  • 308

2 Answers2

1

You could use, for example, awk to capture the headers on the first line, and repeat them every 10 lines, and reduce the column size to say max 5 characters with something like this:

tail -n +1 -f file.csv |
awk  'NR==1 { n=split($0,hdr) }
NR%10==1    { for(i=1;i<=n;i++)printf "%5.5s ",hdr[i];printf "\n" }
            { for(i=1;i<=n;i++)printf "%5.5s ",$i;printf "\n" }
'

This would give something like

yAcce zAcce xGyro ...
    3   256     1 ...
    4   255     3 ... 
    4   255    -6 ...
    3   253    -1 ...
    4   254     2 ...
    5   255     0 ...
    4   255     3 ...
    5   254     3 ...
    5   253    -1 ...
    4   255     3 ...
yAcce zAcce xGyro ...
    3   256     1 ...
...

If the width is still too wide to fit in your terminal, you can pipe the result into less -S +F which makes it act like tail -f, but allows you to scroll left and right with the arrows if you pause the tail with Control-C. To continue tailing type F.

meuh
  • 51,383
  • It helps a lot! Extra points for the header repetition :) However, can you expand on the less -S -F idea? I cannot scroll when piping the result to less. Also, the header gets eaten. – Marcus Jun 29 '17 at 22:00
  • It is +F not -F. It makes less do a tail -f initially. You cannot scroll at this point. You have to interrupt with control-c to stop the tailing, but you stay inside less. you can then scroll: check your keybindings in less if you cannot. When you have the view of the columns you want you can type F again to continue tailing with these columns. – meuh Jun 30 '17 at 05:35
  • If I pause less, then scroll, and try to continue, less is "Waiting for data... (interrupt to abort)" indefinetly. My work around is piping to less without +F, scroll and then initiate following with shift+F (that also is non-pausable of course in terms of "continue following after pausing"). I assume there is an issue with following a piped input. – Marcus Jul 04 '17 at 17:36
  • There will be a problem when piping a command into less because the control-c can also be delivered to the command as a signal which will probably stop it. The simplest answer is to put the result in a file and use less on that file, so that a control-c will only be received by less. Or you can pipe from a shell script if you get it to ignore signals with trap '' int. You will have difficulty stopping the script: a useful trick is to use control-z to put it in the background then use a kill command, or use the quit signal (control-\). – meuh Jul 04 '17 at 18:06
  • I found a similar answer in https://unix.stackexchange.com/questions/26826/follow-a-pipe-using-less#139295 ... anyhow, doesn't it make the last part of your answer a bit misleading? – Marcus Jul 05 '17 at 09:36
0

Column does exactly what you're asking for. You can identify tab as the separator with an argument. Why is piping to column not an option here? cat <file.csv> | column -s "\t"

Alex Waibel
  • 1
  • 1
    It's not an option because I want to monitor file changes. I know column for that task, but, as far as I know, I have to take care of updating the file input. – Marcus Jun 29 '17 at 14:28