format to columns for real-time data (pipe)

Question

Is there a way to do column-like filtering on data comming through a pipe?

I am looking for a way how to do a similar thing that column -t does, but without waiting for the input to end so it would work for real-time data comming through a pipe. I know I could force fixed-width columns by awk, but that needs too much setup every time the format changes.

PS: I do not think mimicking column -x on incomplete data is possible. I also think column -t is impossible to replicate perfectly on incomplete data. It is OK if the solution outputs narrower columns at first and then expand them as more lines arrive.

EDIT: Example to illustrate this is NOT a buffering problem:

yes something | cat -n | tr -s '\t' ' ' | column -t

Answer 1

The whole point of column -t is that it aligns fields in columns automatically based on the maximum with of each field in each column.

If your input contains

a b
a  b
a bc

It will output:

a  b
a  b
a  bc

If you add a:

xxxxx b

line to the input, the output becomes:

a      b
a      b
a      bc
xxxxx  b

column needs to read all the lines of input to determine the width of each column and can't start outputting anything before then.

Your only way to work around that is if you know or can guess the maximum width of each column.

For instance, if you know fields are never larger than 10 cells, you can do:

<input tr -s '[:blank:]' '[\t*]' | expand -t 12

To format the output in 12-cell large columns.

(beware some tr implementations including GNU tr don't support multi-byte characters and some expand implementations including GNU expand support neither multi-byte characters nor zero-width or double-width ones)

For a solution with columns whose widths adapt dynamically to the width of new input, you could do something like:

perl -Mopen=locale -MText::CharWidth=mbswidth -lae '
  for (0..$#F) {$l = mbswidth$F[$_]; $l[$_] = $l if $l > $l[$_]}
  print((map {sprintf "%-$l[$_]s  ", $F[$_]} (0..$#F-1)), $F[$#F])'

For instance, on the output of lorem -p 2 | fmt -w 40, that gives:

Rerum  aut  pariatur  nihil
modi.  Exercitationem  ut
animi.  Quibusdam       dolores
voluptates  pariatur        vel
tempora.    Adipisci        expedita  voluptate
dolores     qui             consequatur.  Laboriosam
eum         ea.             Quasi         ab            qui  harum
repudiandae  consequatur     quasi
Nobis        quia            nesciunt      laudantium.
enim         exercitationem
earum.       Pariatur        nesciunt
maiores      natus           nemo          delectus.     Ut
ad           voluptatem.     Consequatur   sint
enim         sequi           aut           est           nihil.  Et     at

Or to reformat only the first 3 columns:

perl -Mopen=locale -MText::CharWidth=mbswidth -lne '
  @F = /(\S+)\s+(\S+)\s+(\S+)\s*(.*)/;
  for (0..$#F) {$l = mbswidth$F[$_]; $l[$_] = $l if $l > $l[$_]}
  print((map {sprintf "%-$l[$_]s  ", $F[$_]} (0..$#F-1)), $F[$#F])'

Giving:

Sit  earum  voluptatem  cum adipisci aut
commodi.  Quia   aut         eaque rerum nihil
aperiam.  Dolor  quia        illo et. Quasi
illum     est    aliquam     consequatur maiores
voluptatibus.  Optio  consectetur  aliquid
Aspernatur     omnis  ex           dolor nemo delectus
sit            quia   ut.          Voluptatum voluptatibus
suscipit       vel    quos.        Quo a at et non
cumque         voluptate  dolorum      nostrum. Eos
ex             est        deleniti     necessitatibus
assumenda      provident  culpa.       Ut
sed            et         labore       ullam voluptatum
impedit.       Tempora    delectus     et rem dicta
debitis        odit       dignissimos.