finding the highest character length in each column using a loop and awk

Question

I am trying to create a script that goes through a document and finds the highest character length in a column and return it. This script returns 78,78,78,78 when, what im aiming for is 10,11,14,51

for ((i=1;i<=4;i++)); do
  awk -F"|" '{ print length($i) }' contact_d.csv | sort -nr | sed '1!d';
done

contact_d.csv contains this: (please note dummy data) and its a sample

Barrera|Wilkinson|(09) 1466 1886|eu@dignissim.co.uk
Hopkins|Sellers|(07) 3814 2364|faucibus.orci@libero.co.uk
Hunter|Calderon|(01) 3984 0139|Proin@Uttincidunt.ca

Does anyone have any insight as to why the for loop isn't returning what I am aiming for?

Inside your awk scriptlet, i is never assigned a value, so $i is always $0 - hence you will get the lengths of the whole record at every iteration. But shell looping over an awk command is rarely a good option. — steeldriver, Jul 15 '20 at 12:41
For your sample file, what would you like the script to return? 18, 26, 20? — Quasímodo, Jul 15 '20 at 12:46
@steeldriver I tried that just then it just returned 78. my aim with the scrip is to use the loop to go through each feel and find the highest length of each field. I did it manually and the results where as follows 10 11 14 51. I just can't see wy it didnt work? — user419771, Jul 15 '20 at 12:47
ideally to return the row that has that highest character count for each Column @Quasímodo — user419771, Jul 15 '20 at 12:49
@user419771 I don't know what "that" you are referring to - however to pass the value of shell variable $i into awk, you can add -v i="$i". See for example external variable in awk — steeldriver, Jul 15 '20 at 12:55

AdminBee · Accepted Answer · 2020-07-15T14:01:43.193

The following code should work:

awk -F'|' '{for (i=1;i<=NF;i++) {len=length($i); if (len>lval[i]) {lval[i]=len; lpos[i]=FNR;}}} END{for (i in lval) printf("Longest value of column %d: %d (line %d)\n",i,lval[i],lpos[i])}' contact_d.csv

For the above example, it returns

Longest value of column 1: 7 (line 1)
Longest value of column 2: 9 (line 1)
Longest value of column 3: 14 (line 1)
Longest value of column 4: 26 (line 2)

This script will, for every line, loop over all fields (from 1 to NF, the number of fields) and see if the length of the field (temporarily stored in a variable l) is greater than the longest length found so far, which is stored in an array variable lval under the index of the field (=column) number.
On the first line, lval is not yet initialized, and it will behave as if all lval[i] were 0 (in reality, it is more complex than that).
If the length of the field i on the current line is longer than the value stored in lval[i], the script will store the current length of the field in lval[i] and the current line number (which is accessible through the "automatic" variable FNR) into the array variable lpos.
At the end of the file (END condition), it will print the longest length and corresponding position for all columns. I use the construct for (i in lval) which loops over all indices present in the array lval, so I don't have to save the number of columns in an extra variable (as would be necessary for something like for (i=1;i<=ncols;i++) - in the END block, the concept of "number of fields" becomes somewhat ill-defined although in practice awk will often use the corresponding values for the last line of the file when accessed).

Note that it is rarely necessary to call awk in a shell loop; it can do most of the things you would need a loop for by itself.

As for the reason why your original attempt failed, you are trying to feed a shell variable ($i) to an awk script whose code is enclosed in single quotes (as is recommended), but the single quotes turn off the interpretation of shell variables (and even if not, it would not have worked like that).

finding the highest character length in each column using a loop and awk

1 Answers1