2

I'm combining live sar samples with paste and trying to format the output with awk live.

It works as expected to format the output, but it doesn't do the formatting live on each sample and instead waits till the full 50 seconds (5 sec samples, 10 of them) completes.

Solution was (stdbuf -o0) to disable buffering on the output stream

stdbuf -o0 paste <(sar -q 1 5) <(sar -r 1 5) | awk '{printf "%8s %2s %7s %7s %7s %8s %9s %8s\n", $1,$2,$3,$4,$5,$11,$12,$13}'

11:53:21 AM runq-sz plist-sz ldavg-1 kbmemfree kbmemused %memused
11:53:22 AM       1     167    0.03    46504    449264    90.62
11:53:23 AM       1     167    0.03    46504    449264    90.62
11:53:24 AM       1     167    0.03    46504    449264    90.62
11:53:25 AM       1     167    0.03    46008    449760    90.72
11:53:26 AM       1     167    0.03    46624    449144    90.60
Average:  1     167    0.03    0.05    90.64     40876   172816

First sar command

sar -q 5 10 | awk '{printf "%-8s %-2s %7s %7s %7s\n", $1,$2,$3,$4,$5}'

01:02:08 AM runq-sz plist-sz ldavg-1
01:02:13 AM       1     160    0.09
01:02:18 AM       1     160    0.08
01:02:23 AM       1     160    0.08

Second sar command

sar -r 5 10 | awk {'printf "%-8s %-2s %9s %9s %8s\n", $1,$2,$3,$4,$5}'

01:19:27 AM kbmemfree kbmemused %memused
01:19:32 AM    113840    381928    77.04
01:19:37 AM    113800    381968    77.05
01:19:42 AM    113840    381928    77.04

Using paste alone works to combine both sar reports in real-time

paste <(sar -q 5 10) <(sar -r 5 10)

01:21:09 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked 01:21:09 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
01:21:14 AM         2       159      0.00      0.03      0.05         0 01:21:14 AM    111416    384352     77.53     12512    142052    948776    191.37    195400    121036        56
01:21:19 AM         2       157      0.00      0.03      0.05         0 01:21:19 AM    111928    383840     77.42     12512    142056    947168    191.05    195060    121036        60
01:21:24 AM         1       156      0.00      0.03      0.05         0 01:21:24 AM    112244    383524     77.36     12528    142060    946784    190.97    194932    121052        72

Trying to format with awk inside paste shows nothing till the sar samples complete

Same things happens when doing awk formatting after the paste combine

paste <(sar -q 5 10 | awk '{printf "%-8s %-2s %7s %7s %7s\n", $1,$2,$3,$4,$5}') <(sar -r 5 10 | awk {'printf "%9s %9s %8s\n", $3,$4,$5}')

01:56:53 AM runq-sz plist-sz ldavg-1    kbmemfree kbmemused %memused
01:56:58 AM       1     157    0.00     99664    396104    79.90
01:57:03 AM       1     157    0.00     99664    396104    79.90
01:57:08 AM       1     157    0.00     99644    396124    79.90
01:57:13 AM       1     157    0.00     99612    396156    79.91
01:57:18 AM       1     157    0.00     99628    396140    79.90
01:57:23 AM       1     157    0.00     99656    396112    79.90
01:57:28 AM       1     157    0.00     99520    396248    79.93
01:57:33 AM       1     157    0.00     99656    396112    79.90
01:57:38 AM       2     157    0.00     99268    396500    79.98
01:57:43 AM       1     157    0.00    100152    395616    79.80
Average: 1      157    0.00    0.01    396122     79.90    17557

If I use the full sar logs instead of samples it works obviously

paste <(sar -q | awk '{printf "%-8s %-2s %7s %7s %7s\n", $1,$2,$3,$4,$5}') <(sar -r | awk {'printf "%9s %9s %8s\n", $3,$4,$5}')

12:00:01 AM runq-sz plist-sz ldavg-1    kbmemfree kbmemused %memused
12:10:01 AM       3     156    0.00     71500    424268    85.58
12:20:01 AM       1     150    0.00    110836    384932    77.64
12:30:01 AM       1     150    0.00    108164    387604    78.18

So, is there anyway to have the paste command update out to awk after each sample so I can watch the formatted output scroll by live?

JacobN
  • 123

1 Answers1

3

The answers in Turn off buffering in pipe provide several techniques which you can use. The principal idea is that commands which are not connected to an interactive terminal (for example, in a pipeline) are using buffering. One command which can run another command with different buffering settings is the GNU coreutils stdbuf command which you can apply to your case, unbuffering each command

paste <(stdbuf -i0 -o0 -e0 sar -q 5 10 | stdbuf -i0 -o0 -e0 awk '{printf "%-8s %-2s %7s %7s %7s\n", $1,$2,$3,$4,$5}') <(stdbuf -i0 -o0 -e0 sar -r 5 10 | stdbuf -i0 -o0 -e0 awk {'printf "%9s %9s %8s\n", $3,$4,$5}')

In the above stdbuf is used to unbuffer the input, output and standard error of each awk and sar command.

As pointed out in the comments, in this case unbuffering output is all that is required, so this can be shortened to

paste <(stdbuf -o0 sar -q 5 10 | stdbuf -o0 awk '{printf "%-8s %-2s %7s %7s %7s\n", $1,$2,$3,$4,$5}') <(stdbuf -o0 sar -r 5 10 | stdbuf -o0 awk {'printf "%9s %9s %8s\n", $3,$4,$5}')
Community
  • 1
user531214
  • 176