A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1

Question

Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ?

I have my own bash/awk script, but I am looking for a standard tool, which is found on many/most distros... something more generally available, and ideally has simple command line args, and/or can accept piped input.

Here are some examples of the type of output I am looking for.

    1    Byt  
  173.00 KiB  
   46.57 MiB  
    1.84 GiB  
   29.23 GiB  
  265.72 GiB  
    1.63 TiB  

Here is the bytes-human script (used for the above output)

awk -v pfix="$1" -v sfix="$2" 'BEGIN { 
      split( "Byt KiB MiB GiB TiB PiB", unit )
      uix = uct = length( unit )
      for( i=1; i<=uct; i++ ) val[i] = (2**(10*(i-1)))-1
   }{ if( int($1) == 0 ) uix = 1; else while( $1 < val[uix]+1 ) uix--
      num = $1 / (val[uix]+1)
      if( uix==1 ) n = "%5d   "; else n = "%8.2f"
      printf( "%s"n" %s%s\n", pfix, num, unit[uix], sfix ) 
   }'

Update  Here is a modified version of Gilles' script, as described in a comment to his answer ..(modified to suit my preferred look).

awk 'function human(x) {
         s=" B   KiB MiB GiB TiB EiB PiB YiB ZiB"
         while (x>=1024 && length(s)>1) 
               {x/=1024; s=substr(s,5)}
         s=substr(s,1,4)
         xf=(s==" B  ")?"%5d   ":"%8.2f"
         return sprintf( xf"%s\n", x, s)
      }
      {gsub(/^[0-9]+/, human($1)); print}'

Answer 1

There is nothing like this in POSIX, but there's a number formatting program in modern GNU coreutils: numfmt that at least gets close to your sample output. With GNU coreutils ≥8.24 (2015, so present on all non-embedded Linux except the oldest releases with a very long-term support cycle):

$ numfmt --to=iec-i --suffix=B --format="%9.2f" 1 177152 48832200 1975684956
    1.00B
173.00KiB
 46.58MiB
  1.84GiB

Many older GNU tools can produce this format and GNU sort can sort numbers with units since coreutils 7.5 (Aug 2009, so present on virtually all non-embedded Linux distributions).

---

I find your code a bit convoluted. Here's a cleaner awk version (the output format isn't exactly identical):

awk '
    function human(x) {
        if (x<1000) {return x} else {x/=1024}
        s="kMGTEPZY";
        while (x>=1000 && length(s)>1)
            {x/=1024; s=substr(s,2)}
        return int(x+0.5) substr(s,1,1)
    }
    {sub(/^[0-9]+/, human($1)); print}'

(Reposted from a more specialized question)

Answer 2

As of v. 8.21, coreutils includes numfmt:

numfmt reads numbers in various representations and reformats them as requested.
The most common usage is converting numbers to / from human representation.

e.g.

printf %s\\n 5607598768908 | numfmt --to=iec-i

5.2Ti

Various other examples (including filtering, input/output processing etc) are presented HERE.

---

In addition, as of coreutils v. 8.24, numfmt can process multiple fields with field range specifications similar to cut, and supports setting the output precision with the --format option
e.g.

numfmt --to=iec-i --field=2,4 --format='%.3f' <<<'tx: 180000 rx: 2000000'

tx: 175.782Ki rx: 1.908Mi

Answer 3

Here's a bash-only option, no bc or any other non-builtins, + decimal format and binary units.

# Converts bytes value to human-readable string [$1: bytes value]
bytesToHumanReadable() {
    local i=${1:-0} d="" s=0 S=("Bytes" "KiB" "MiB" "GiB" "TiB" "PiB" "EiB" "YiB" "ZiB")
    while ((i > 1024 && s < ${#S[@]}-1)); do
        printf -v d ".%02d" $((i % 1024 * 100 / 1024))
        i=$((i / 1024))
        s=$((s + 1))
    done
    echo "$i$d ${S[$s]}"
}

Examples:

$ bytesToHumanReadable 123456789
117.73 MiB
$ bytesToHumanReadable 1000000000000 # '1TB of storage'
931.32 GiB                           #  1TB of storage
$ bytesToHumanReadable 
0 Bytes
$ bytesToHumanReadable 9223372036854775807
7.99 EiB

Should perform well on any version of Bash out there (including MSYSGit's Bash for Windows).

Answer 4

Via linux - Is there a command line calculator for byte calculations? - Stack Overflow, I found about GNU Units - though without examples on the SO page; and as I didn't see it listed here, here is a small note about it.

First, check if the units are present:

$ units --check-verbose |grep byte
doing 'byte'

$ units --check-verbose |grep mega
doing 'megalerg'
doing 'mega'

$ units --check-verbose |grep mebi
doing 'mebi'

Given that they are, do a conversion - printf format specifiers are accepted to format the numeric result:

$ units --one-line -o "%.15g" '20023450 bytes' 'megabytes'  # also --terse
    * 20.02345
$ units --one-line -o "%.15g" '20023450 bytes' 'mebibytes' 
    * 19.0958499908447
$ units --one-line -o "%.5g" '20023450 bytes' 'mebibytes' 
    * 19.096

Answer 5

This is a complete rewrite inspired by Peter.O's modified version of Gilles' awk script.

Changes:

• Fixes Peter.O's bug where he looks for a string of >1 character where he should be looking for one >4 characters. Due to that bug, his code doesn't work for ZiB units.
• Removes the very ugly hardcoding of a long string of space-separated unit sizes.
• Adds command line switches to enable/disable padding.
• Adds command line switches to go from base-1024 (KiB) to base-1000 (KB) notation.
• Wraps it all in an easy to use function.
• I place this in the public domain and welcome widespread use.

Code:

bytestohuman() {
    # converts a byte count to a human readable format in IEC binary notation (base-1024), rounded to two decimal places for anything larger than a byte. switchable to padded format and base-1000 if desired.
    local L_BYTES="${1:-0}"
    local L_PAD="${2:-no}"
    local L_BASE="${3:-1024}"
    BYTESTOHUMAN_RESULT=$(awk -v bytes="${L_BYTES}" -v pad="${L_PAD}" -v base="${L_BASE}" 'function human(x, pad, base) {
         if(base!=1024)base=1000
         basesuf=(base==1024)?"iB":"B"

         s="BKMGTEPYZ"
         while (x>=base && length(s)>1)
               {x/=base; s=substr(s,2)}
         s=substr(s,1,1)

         xf=(pad=="yes") ? ((s=="B")?"%5d   ":"%8.2f") : ((s=="B")?"%d":"%.2f")
         s=(s!="B") ? (s basesuf) : ((pad=="no") ? s : ((basesuf=="iB")?(s "  "):(s " ")))

         return sprintf( (xf " %s\n"), x, s)
      }
      BEGIN{print human(bytes, pad, base)}')
    return $?
}

Test Cases (if you want to look at the output):

bytestohuman 1; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 500; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1023; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1024; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1500; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000000; echo "${BYTESTOHUMAN_RESULT}.";

bytestohuman 1 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 500 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1023 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1024 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1500 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000000 no 1000; echo "${BYTESTOHUMAN_RESULT}.";

bytestohuman 1 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 500 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1023 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1024 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1500 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000 yes; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000000 yes; echo "${BYTESTOHUMAN_RESULT}.";

bytestohuman 1 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 500 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1023 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1024 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 1500 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";
bytestohuman 150000000000000000000 yes 1000; echo "${BYTESTOHUMAN_RESULT}.";

Enjoy!

Answer 6

There are a couple of perl modules on CPAN: Format::Human::Bytes and Number::Bytes::Human, the latter one being a bit more complete:

$ echo 100 1000 100000 100000000 |
  perl -M'Number::Bytes::Human format_bytes' -pe 's/\d{3,}/format_bytes($&)/ge'
100 1000 98K 96M

$ echo 100 1000 100000 100000000 |
  perl -M'Number::Bytes::Human format_bytes' -pe 's/\d{3,}/
   format_bytes($&,bs=>1000, round_style => 'round', precision => 2)/ge'
100 1.00k 100k 100M

And the reverse:

$ echo 100 1.00k 100K 100M 1Z |
  perl -M'Number::Bytes::Human parse_bytes' -pe '
    s/[\d.]+[kKMGTPEZY]/parse_bytes($&)/ge'
100 1024 102400 104857600 1.18059162071741e+21

NOTE: the function parse_bytes() was added in version 0.09 (2013-03-01)

Answer 7

Actually, there is a utility that does exactly this. I know cos it was me wot wrote it. It was written for *BSD but ought to compile on Linux if you have the BSD libraries (which I believe are common).

I've just released a new version, posted here:

http://blog.frankleonhardt.com/2015/freebsd-hr-utility-human-readable-number-filter-man-page/

It's called hr, and it will take stdin (or files) and convert numbers to human-readable format in a way that is (now) exactly the same as ls -h and so on, and it can select individual feeds in lines, scale pre-scaled units (e.g. if they're in 512-byte blocks convert them to Mb etc), adjust column padding, and so on.

I wrote it a few years ago because I thought trying to write a shell script, although intellectually interesting, was also utter madness.

Using hr, for example, you can easily get a sorted list of directory sizes (which come out in 1Kb units and need shifting before converting) with the following:

du -d1 | sort -n | hr -sK

While du will produce -h output, sort won't sort by it. The addition of -h to existing utilities is a classic case of not following the unix philosophy: have simple utilities doing defined jobs really well.

Answer 8

@don_crissti's first Answer is good, but can be even shorter using Here Strings, e.g.

$ numfmt --to=iec-i <<< "12345"
13Ki

$ numfmt --to=iec-i --suffix=B <<< "1234567"
1.2MiB

or even

$ numfmt --from=iec-i --to=iec-i --suffix=B <<< "12345Ki"
13MiB

if <<< is not available you can use e.g.

$ echo "1234567" | numfmt --to=iec-i --suffix=B
1.2MiB

Answer 9

I had the same problem and I quickly came up with a simple solution using awk's log() function:

awk '
  BEGIN {
    split("B,kiB,MiB,GiB", suff, ",")
  }

  {
    size=$1;
    rank=int(log(size)/log(1024));
    printf "%.4g%s\n", size/(1024**rank), suff[rank+1]
  }
'

And the precision lost in using float numbers is not that bad since that precision will be lost anyways.

Answer 10

user@host:/usr$ alias duh="du -s -B1 * | sort -g | numfmt --to=iec-i --format='%10f'"
user@host:/usr$ duh

Gives:

 4.0Ki games
 3.9Mi local
  18Mi include
  20Mi sbin
 145Mi bin
 215Mi share
 325Mi src
 538Mi lib

Unfortunately I can't figure out how to get two decimals accuracy. Tested on Ubuntu 14.04.

Answer 11

Here is a way to do it almost purely in bash, just needs 'bc' for the floating point math.

function bytesToHR() {
        local SIZE=$1
        local UNITS="B KiB MiB GiB TiB PiB"
        for F in $UNITS; do
                local UNIT=$F
                test ${SIZE%.*} -lt 1024 && break;
                SIZE=$(echo "$SIZE / 1024" | bc -l)
        done

    if [ "$UNIT" == "B" ]; then
        printf "%4.0f    %s\n" $SIZE $UNIT
    else
        printf "%7.02f %s\n" $SIZE $UNIT
    fi
}

Usage:

bytesToHR 1
bytesToHR 1023
bytesToHR 1024
bytesToHR 12345
bytesToHR 123456
bytesToHR 1234567
bytesToHR 12345678

Output:

   1    B
1023    B
   1.00 KiB
  12.06 KiB
 120.56 KiB
   1.18 MiB
  11.77 MiB

Answer 12

If you can use Python and pip, you can solve this with humanize. (Thanks to Pyrocater for the idea.)

$ pip install humanize
$ bytes=35672345337
$ python -c "import humanize; print(humanize.naturalsize($bytes))"

35.7 GB

$ seq 0 750000 2250000 |python -c $'import sys, humanize\nfor n in sys.stdin: print(humanize.naturalsize(n))'

0 Bytes
750.0 kB
1.5 MB
2.2 MB

Answer 13

pip install humanfriendly and then add a simple function to your default shell (e.g. ~/.bashrc)

function fsize() { humanfriendly --format-size `stat -f '%z' $1` }

Use like this

➜ fsize file.txt
6.17 KB

Answer 14

This is what I'm using:

echo 10000000 | awk '{split("Byt,KiB,MiB,GiB,TiB", unit, ","); (size=$1) ? level=sprintf("%.0d", (log(size)/log(1024))) : level=0; printf "%.2f %s\n", size/(1024**level), unit[level+1]}'

returns 9.54 MiB

Example with path:

stat --format "%s %n" /boot/* | awk '{split("Byt,KiB,MiB,GiB,TiB", unit, ","); (size=$1) ? level=sprintf("%.0d", (log(size)/log(1024))) : level=0; printf "%8.2f %s ", size/(1024**level), unit[level+1]; print $2}'

returns:

   32.00 KiB /boot/EFI
   32.00 KiB /boot/System
   19.78 MiB /boot/bzfirmware
   65.00 Byt /boot/bzfirmware.sha256
    4.60 MiB /boot/bzimage
   65.00 Byt /boot/bzimage.sha256
   11.98 MiB /boot/bzmodules
   65.00 Byt /boot/bzmodules.sha256
  141.09 MiB /boot/bzroot
   69.33 MiB /boot/bzroot-gui
   65.00 Byt /boot/bzroot-gui.sha256
   65.00 Byt /boot/bzroot.sha256
   18.82 KiB /boot/changes.txt
   32.00 KiB /boot/config
  119.44 KiB /boot/ldlinux.c32
   68.00 KiB /boot/ldlinux.sys
    7.79 KiB /boot/license.txt
   32.00 KiB /boot/logs
    1.72 KiB /boot/make_bootable.bat
    3.21 KiB /boot/make_bootable_linux
    2.37 KiB /boot/make_bootable_mac
  146.51 KiB /boot/memtest
   32.00 KiB /boot/previous
   32.00 KiB /boot/syslinux

Answer 15

When you just want a quick conversion without fancy output, or when the numfmt is not available, I find the units handy. Eg define the following in your .bashrc:

convu() { 
  conv=$( units $2 $3 | grep / | cut -f 2 -d " " )
  echo $(( $1 / $conv ))
}

Then you can do this:

$ convu 268435456 bytes megabytes
256

or this:

$ convu $( some command that outputs a value in bytes ) bytes megabytes
256

Additionally if you define unit abbreviations in a file, say ~/.units.bytes then you can define convu instead as

convu() { 
  conv=$( units -f ~/.units.bytes $2 $3 | grep / | cut -f 2 -d " " )
  echo $(( $1 / $conv ))
}

If the file has

bit                     !h!
nibble                  4 bit
nybble                  nibble
byte                    8 bit
word                    2 byte
block                   512 byte
kbyte                   1024 byte
megabyte                1024 kbyte
gigabyte                1024 megabyte
terabyte                1024 gigabyte
petabyte                1024 terabyte
exabyte                 1024 petabyte
zettabyte               1024 exabyte
yottabyte               1024 zettabyte
kilobyte                kbyte
meg                     megabyte
baud                    bit/sec
b                       byte
mb                      megabyte
gb                      gigabyte
kb                      kilobyte

then you can write

$ convu 268435456 b mb
256

Answer 16

The answer to your question is yes.

While the output format isn't exactly to your specification, the conversion itself is easily done by a very standard tool (or two). The ones to which I refer are dc and bc. You can get a segmented report by altering their output radices. Like this:

{   echo 1024 o           #set dc's output radix
    echo 1023 pc          #echo a number then print + clear commands
    echo 1024 pc
    echo 1025 pc
    echo 8000000 pc
} | dc

...which prints...

 1023                    #1 field 1023 bytes
 0001 0000               #2 fields 1k 0b
 0001 0001               #2 fields 1k 1b
 0007 0644 0512          #3 fields 7m 644k 512b or 7.64m

I use dc above because it is a personal favorite, but bc can do the same with different syntax and adheres to the same format rules as specified by POSIX like:

• bc obase

  • For bases greater than 16, each digit shall be written as a separate multi-digit decimal number. Each digit except the most significant fractional digit shall be preceded by a single space. For bases from 17 to 100, bc shall write two-digit decimal numbers; for bases from 101 to 1000, three-digit decimal strings, and so on. For example, the decimal number 1024 in base 25 would be written as:

  01 15 24

  and in base 125, as:

  008 024

Answer 17

Python tools exist

$pip install humanfriendly  # Also available as a --user install in ~/.local/bin

$humanfriendly --format-size=2048
2.05 KB
$humanfriendly --format-number=2048
2,048

I don't see a --binary flag :(, so you'd have to use python directly for binary representation:

$python -c 'import sys, humanfriendly; print(humanfriendly.format_size(int(sys.argv[1]), binary=True))' 2048
2 KiB
$python -c 'import sys, humanfriendly; print(humanfriendly.format_size(int(sys.argv[1]), binary=True))' 2000
1.95 KiB

Answer 18

I made a tool for this finally.

https://github.com/gonejack/hsize

go get -u github.com/gonejack/hsize

> echo 31415 | hsize
31415 => 30.68KB

> hsize 46783 92929211
46783 => 45.69KB
92929211 => 88.62MB

Answer 19

Short and sweet, shell only solution:

convertB_human() {
NUMBER=$1
for DESIG in Bytes KB MB GB TB PB
do
   [ $NUMBER -lt 1024 ] && break
   let NUMBER=$NUMBER/1024
done
printf "%d %s\n" $NUMBER $DESIG
}

It doesn't show the decimal potion.

The let VAR=expression is Korn-ish. Substitute with VAR=$(( expression )) for Born-again-ish.

Answer 20

AFAIK there is no such standard tool to which you can pass text and it returns a human readable form. You may be able to find a package to accomplish the said task for your distro.

However, I do not understand why you may need such a tool. Most packages that give a related output, usually have a -h or equivalent switch for human-readable output.