How to find line with least characters

Question

I am writing a shell script, using any general UNIX commands. I have to retrieve the line that has the least characters (whitespace included). There can be up to around 20 lines.

I know I can use head -$L | tail -1 | wc -m to find the character count of line L. The problem is, the only method I can think of, using that, would be to manually write a mess of if statements, comparing the values.

Example data:

seven/7
4for
8 eight?
five!

Would return 4for since that line had the least characters.

In my case, if multiple lines have the shortest length, a single one should be returned. It does not matter which one is selected, as long as it is of the minimum length. But I don't see the harm in showing both ways for other users with other situations.

Answer 1

Here's a variant of an awk solution for printing the first found minimum line:

awk '
  NR==1 || length<len {len=length; line=$0}
  END {print line}
'

which can simply be extended by one condition to print all minimum lines:

awk '
  length==len {line=line ORS $0}
  NR==1 || length<len {len=length; line=$0}
  END {print line}'
'

Answer 2

With sqlite3:

sqlite3 <<EOT
CREATE TABLE file(line);
.import "data.txt" file
SELECT line FROM file ORDER BY length(line) LIMIT 1;
EOT

Answer 3

A Perl way. Note that if there are many lines of the same, shortest length, this approach will only print one of them:

perl -lne '$m//=$_; $m=$_ if length()<length($m); END{print $m if $.}' file 

Explanation

• perl -lne : -n means "read the input file line by line", -l causes trailing newlines to be removed from each input line and a newline to be added to each print call; and -e is the script that will be applied to each line.
• $m//=$_ : set $m to the current line ($_) unless $m is defined. The //= operator is available since Perl 5.10.0.
• $m=$_ if length()<length($m) : if the length of the current value of $m is greater than the length of the current line, save the current line ($_) as $m.
• END{print $m if $.} : once all lines have been processed, print the current value of $m, the shortest line. The if $. ensures that this only happens when the line number ($.) is defined, avoiding printing an empty line for blank input.

Alternatively, since your file is small enough to fit in memory, you can do:

perl -e '@K=sort{length($a) <=> length($b)}<>; print "$K[0]"' file 

Explanation

• @K=sort{length($a) <=> length($b)}<> : <> here is an array whose elements are the lines of the file. The sort will sort them according to their length and the sorted lines are saved as array @K.
• print "$K[0]" : print the first element of array @K: the shortest line.

---

If you want to print all shortest lines, you can use

perl -e '@K=sort{length($a) <=> length($b)}<>; 
         print grep {length($_)==length($K[0])}@K; ' file 

Answer 4

Python comes out fairly concise, and the code Does What It Says On The Tin:

python -c "import sys; print min(sys.stdin, key=len),"

The final comma is obscure, I admit. It prevents the print statement adding an additional linebreak. Additionally, you can write this in Python 3 supporting 0 lines like:

python3 -c "import sys; print(min(sys.stdin, key=len, default='').strip('\n'))"

Answer 5

I always love solutions with pure shell scripting (no exec!).

#!/bin/bash
min=
is_empty_input="yes"

while IFS= read -r a; do
    if [ -z "$min" -a "$is_empty_input" = "yes" ] || [ "${#a}" -lt "${#min}" ]; then
        min="$a"
    fi
    is_empty_input="no"
done

if [ -n "$a" ]; then
    if [ "$is_empty_input" = "yes" ]; then
        min="$a"
        is_empty_input="no"
    else
        [ "${#a}" -lt "${#min}" ] && min="$a"
    fi
fi

[ "$is_empty_input" = "no" ] && printf '%s\n' "$min"

Note:

There is a problem with NUL bytes in the input. So, printf "ab\0\0\ncd\n" | bash this_script prints ab instead of cd.

Answer 6

Here a pure zsh solution (it prints all lines with the minimal length, from file):

IFS=$'\n'; print -l ${(M)$(<file):#${~${(o@)$(<file)//?/?}[1]}}

Example input:

seven/7
4for
8 eight?
five!
four

Output is:

4for
four

I think it needs a short explanation :-)

---

First, we set the internal field separator to newline:

IFS=$'\n';

So far so good, now the hard part. print uses the -l flag to print the result separated by newlines instead of spaces.

Now, we start at the inside:

$(<file)

The file is read line by line and treated as array. Then:

${(o@)...//?/?}

The o flag says that the result should be ordered in ascending order, the @ means to treat the result as array too. The part behind (//?/?) is a substitution an replaces all characters with a ?. Now:

${~...[1]}

We take the first array element [1], which is the shortest, in your case its now ????.

${(M)$(<file):#...}

Matching is performed on each array elements separately, and the unmatched array elements are removed (M). Each element that matches ???? (4 characters) stays in the array. So the remaining elements are the ones that have 4 characters (the shortest ones).

Edit: If you need only one of the shortest lines, this modified version prints the first one:

IFS=$'\n'; print -l ${${(M)$(<file):#${~${(o@)$(<file)//?/?}[1]}}[1]}

Answer 7

Try:

awk '{ print length, $0 }' testfile | sort -n | cut -d" " -f2- | head -1

The idea is to use awk to print the length of each line first. This will appear as:

echo "This is a line of text" | awk '{print length, $0}'
22 This is a line of text

Then, use the character count to sort the lines by sort, cut to get rid of the count and head to keep the first line (the one with the least characters). You can of course use tail to get the line with the most characters in this case.

(This was adopted from this answer)

Answer 8

tr -c \\n 1 <testfile |   #first transform every [^\n] char to a 1
grep -nF ''           |   #next get line numbers
paste -d: - testfile  |   #then paste it together with itself
sort  -t: -nk2,2          #then sort on second field

...and the winner is... line 2, it would seem.

2:1111:4for
4:11111:five!
1:1111111:seven/7
3:11111111:8 eight?

But the problem with that is that every line must more than double in length in order for it to work - so LINE_MAX is effectively halved. The cause is that it is using - what, a base 1? - to represent the length of the line. A similar - and perhaps more tidy - approach might be to compress that information in stream. The first idea along those lines that occurs to me is that I ought to unexpand it:

tr -c \\n \  <testfile    |   #transform all [^\n] to <space>
unexpand -t10             |   #squeeze every series of 10 to one tab
grep -nF ''               |   #and get the line numbers
sed    's/:/!d;=;:/;h;:big    #sed compares sequential lines
$P;$!N; /\(:[^ ]*\)\( *\)\n.*\1.*\2/!D     #newest line is shorter or...
        g;/:./!q;b big'   |   #not; quit input entirely for blank line
sed -f - -e q testfile        #print only first occurrence of shortest line

That prints...

2
4for

Another one, just sed:

sed -n '/^\n/D;s/\(.\)\(\n.*\)*/\1/g
$p;h;   s// /g;G;x;n;//!g;H;s// /g
G;      s/^\( *\)\(\n \1 *\)\{0,1\}\n//
D'      <infile >outfile

The syntax is standards compliant - but that is no guarantee that any old sed will handle the \(reference-group\)\{counts\} correctly - many do not.

It basically applies the same regexp to input repeatedly - which can be very beneficial when it is time to compile them. That pattern is:

\(.\)\(\n.*\)*

Which matches different strings in different ways. For example:

string1\nstring2\nstring3

...is matched with s in \1 and '' the null string in \2.

1\nstring2\nstring3

...is matched with 1 in \1 and \nstring2\nstring3 in \2

\nstring2\nstring3

...is matched with \n in \1 and '' the null string in \2. This would be problematic if there was any chance of a \newline occurring at the head of pattern space - but the /^\n/D, and //!g commands are used to prevent this. I did use [^\n] but other needs for this little script made portability a concern and I wasn't satisfied with the many ways it is often misinterpreted. Plus, . is faster.

\nstring2
string1

... match \n and s again in \1 and both get the '' null string in \2. Empty lines don't match at all.

When the pattern is applied globally the two biases - both the left-most standard bias and the lesser right-side \newline bias - are counter-balanced to effect a skip. A few examples:

s/\(.\)\(\n.*\)*/\1:\2/g
s/\(.\)\(\n.*\)*/\2\1:/g
s/\(.\)\(\n.*\)*/\1: /g
s/\(.\)\(\n.*\)*/ :\2/g

...if all applied (not in succession) to the following string...

string1\nstring2

...will transform it to...

s:t:r:i:n:g:1:\nstring2
s:t:r:i:n:g:\nstring21:
s:t:r:i:n:g:1: 
 : : : : : : :\nstring2

Basically I use the regexp to always handle only the first line in any pattern-space to which I apply it. That enables me to juggle two different versions of both a retained shortest-match-so-far line and the most recent line without resorting to test loops - every substitution applied handles the entire pattern-space at once.

The different versions are necessary for literal string/string comparisons - so there must be a version of each line where all characters are guaranteed to be equal. But of course if one or the other should wind up actually being the earliest occurring shortest line in input, then the line printed to output should probably be the original version of the line - not the one I've sanitized/homogenized for comparison's sake. And so I need two versions of each.

It is unfortunate that another necessity is a lot of buffer switching to handle same - but at least neither buffer ever exceeds any more than the four lines needed to stay current - and so maybe it is not terrible.

Anyway, for each cycle the first thing that happens is a transformation on the remembered line - because the only copy actually saved is the literal original - into...

^               \nremembered line$

...and afterward the next input line overwrites any old buffer. If it does not contain at least a single character it is effectively ignored. It would be far easier just to quit at the first occurring blank line, but, well, my test data had a lot of those and I wanted to handle multiple paragraphs.

And so if it does contain a character its literal version is appended to the remembered line and its spaced comparison version is positioned at head of pattern space, like this:

^   \n               \nremembered line\nnew$

Last a substitution is applied to that pattern space:

s/^\( *\)\(\n \1 *\)\{0,1\}\n//

So if the newline can fit within the space needed to contain the remembered line with at least one char to spare then the first two lines are substituted away, else only the first.

Regardless of the outcome the first line in pattern space is always Deleted at end-of-cycle before starting again. This means that if the new line is shorter than the last the string...

new

...is sent back to the first substitution in the cycle which will always strip only from the first newline char on - and so it remains whole. But if it is not then the string...

remembered line\nnew

...will begin the next cycle instead, and the first substitution will strip from it the string...

\nnew

...every time.

On the very last line the remembered line is printed to standard out, and so for the example data given, it prints:

4for

But, seriously, use tr.

Answer 9

With POSIX awk:

awk 'FNR==1{l=$0;next};length<length(l){l=$0};END{print l}' file

Answer 10

Borrowing some of @mikeserv's ideas:

< testfile sed 'h;s/./:/g;s/.*/expr length "&"/e;G;s/\n/\t/' | \
sort -n | \
sed -n '1s/^[0-9]+*\t//p'

The first sed does the following:

• h saves the original line to the hold buffer
• Replace every character in the line with : - this is to remove any danger of code injection
• Replace the whole line with expr length "whole line" - this is a shell expression which may be evaluated
• The e command to s is a GNU sed extension to evaluate the pattern space and put the result back in the pattern space.
• G appends a newline and the contents of the hold space (the original line) to the pattern space
• the final s replaces the newline with a tab

The number of characters is now a number at the start of each line, so sort -n sorts by line length.

The final sed then removes all but the first (shortest) line and the line length and prints the result.

Answer 11

It occurred to me that the whole thing is possible in one sed expression. It ain't pretty:

$ sed '1h;s/.*/&\n&/;G;:l;s/\n[^\n]\([^\n]*\)\n[^\n]/\n\1\n/;tl;/\n\n/{s/\n.*//;x};${x;p};d' testfile
4for
$ 

Breaking this down:

1h            # save line 1 in the hold buffer (shortest line so far)
s/.*/&\n&/    # duplicate the line with a newline in between
G             # append newline+hold buffer to current line
:l            # loop start
s/\n[^\n]\([^\n]*\)\n[^\n]/\n\1\n/
              # attempt to remove 1 char both from current line and shortest line
tl            # jump back to l if the above substitution succeeded
/\n\n/{       # matches if current line is shorter
  s/\n.*//    # remove all but original line
  x           # save new shortest line in hold buffer
}
${            # at last line
  x           # get shortest line from hold buffer
  p           # print it
}
d             # don't print any other lines

The BSD sed in OS X is a bit more finicky with newlines. This version works for both BSD and GNU versions of sed:

$ sed -e '1h;G;s/\([^\n]*\)\(\n\)\(.*\)/\1\2\1\2\3/;:l' -e 's/\(\n\)[^\n]\([^\n]*\n\)[^\n]/\1\2/;tl' -e '/\n\n/{s/\n.*//;x;};${x;p;};d' testfile
4for
$

_{Note this is more of a "because its possible" answer than a serious attempt to give a best practice answer. I guess it means I've been playing too much code-colf}

Answer 12

To get just the first shortest line:

f=file; sed -n "/^$(sed 's/./1/g' $f | sort -ns | sed 's/././g;q')$/{p;q}" $f

To get all the shortest lints, just change {p;q} to p

---

Another method (somewhat unusual) is to have sort do the actual sort by length. It is relatively slow even with short lines, and becomes dramatically slower as the line length increases.
However, I find the idea of sorting by overlapping keys quite interesting. I'm posting it in case others may also find it interesting/informative.

How it works:
Sort by length-variants of the same key – key 1 which spans the entire line
Each successive key variant increments the key length by one character, up to the length of the file's longest line (determined bywc -L)

To get just the first (sorted) shortest line:

f=file; sort -t'\0' $(seq -f "-k1.%0.0f" $(<"$f" wc -L) -1 1) "$f" | head -n1

which is the same as:

f=file.in; 
l=$(<"$f" wc -L)
k=$(seq -f "-k1.%0.0f" $l -1 1) 
sort -st'\0' $k "$f" | head -n1

Answer 13

Another perl solution: store the lines in a hash-of-arrays, the hash key being the line length. Then, print out the lines with the minimum key.

perl -MList::Util=min -ne '
    push @{$lines{ length() }}, $_;
} END {
    print @{$lines{ min keys %lines }};
' sample 

4for

Answer 14

Assuming blank lines are not considered the shortest line and that blank lines might exist, the following pure AWK will work:

awk '
    {
        len   = length;
        a[$0] = len
    }
    !len { next }
    !min { min = len }
    len < min { min = len }
    END {
        for (i in a)
            if (min == a[i])
                print i
    }
' infile.txt

Answer 15

What about using sort?

awk '{ print length($0) "\t" $0 }' input.txt | sort -n | head -n 1 | cut -f2-

Answer 16

With GNU awk

gawk '
    {
         a[length]=$0
    };
    END
    {
        PROCINFO["sorted_in"]="@ind_num_asc";
        for (i in a)
        {
            print a[i]; 
            exit
        }
    }
    ' file

• Read each line into an array indexed by line length.

• Set PROCINFO["sorted_in"] to @ind_num_asc to force array scanning to be ordered by the array index, sorted numerically

• The setting of PROCINFO in the manner above forces the line with the smallest length to be picked up first in the traversal of the array. So print first element from the array and exit

This has the disadvantage of being a nlogn while some of the other approaches are n in time

Answer 17

Mid-level shell tools method, with no sed or awk:

f=inputfile
head -n $(xargs -d '\n' -L 1 -I % sh -c 'exec echo "%" | wc -c' < $f | 
          cat -n | sort -n -k 2 | head -1 | cut -f 1)  $f | tail -1