wc -c not working in a loop in the script

Question

So I'm writing a simple script that checks if the size of the file is less than the given argument and if its true then its suppose to echo "YES"

#!bin/bash

Function ()
{
    cd $1
    lista=`ls`
    for item in $lista
    do
        if [ ! -d $item ]
        then
            size=`wc -c $item | awk '{ print $1 }'`
            if [ $size -lt $2 ]
            then
                echo "$item"
            fi
        else
            Function $item $2
        fi
        cd ..
    done
}

Function $1 $2

And the bash:

bash test2.sh /home/161161 300

However something is wrong with the wc -c because it gives me the error that no such file or exist after running through the second loop.

Any ideas?

May be relevant here: Why does my shell script choke on whitespace or other special characters? — steeldriver, Jun 15 '18 at 00:44
It's hard to tell, because the script as written is very brittle. It should Use More Quotes™, should not parse ls, should use [[ instead of [. For readability it should also use a different name than Function (which is only an uppercase away from a Bash keyword) and should be indented properly. — l0b0, Jun 15 '18 at 03:03
You don't need a script for this, just find /dir -size -300c will list the full filenames; if you really want ony the last branch, which seems unlikely to be useful, add -printf %s or pipe the output through sed 's:.*/::' or awk -F/ '{print $NF}' . If you really want a script, you can avoid the awk for each file by using redirection wc -c <"$file" so the wc output is only the number and no filename. — dave_thompson_085, Jun 15 '18 at 03:55
The wc -c <"$file" will read the contents of each and every file to count the number of bytes. You can get the file size directly with stat -c %s "$file". But as has already been pointed out you can just use find. — Chris Davies, Jun 15 '18 at 06:27
@DavidMathers : BTW, you can simplify your size calculation to size=$(wc -c <$item). If you write it like this, you don't need awk. — user1934428, Jun 15 '18 at 07:13
If you were to indent the loop and the conditional properly, it'd be easier to see that the cd .. is inside the loop, so it will run for every file... — ilkkachu, Jun 15 '18 at 09:06

Kusalananda · Answer 1 · 2018-06-16T21:21:49.737

The issue is that your code is doing cd .. before being done with all the files in a directory. In general, you don't have to cd into directories to get filenames from them, and going back and forth into directories in loops can be confusing. It can also lead to strange issues if you are redirecting output to filenames with relative paths, etc. inside the loops. In this script, you also would not know where (in what directory) the file was found, because you always look in the current directory.

Fixing this by not using cd, and also by not using ls, which allows the script to work with filenames that have spaces and other unusual characters in them:

#!/bin/sh

find_smaller () {
    dir=$1
    size=$2

    for pathname in "$dir"/*; do
        if [ -f "$pathname" ]; then
            # this is a regular file (or a symbolic link to one), test its size
            filesize=$( wc -c <"$pathname" )
            if [ "$filesize" -lt "$size" ]; then
                printf 'Found %s, size is %d\n' "$pathname" "$filesize"
            fi
        elif [ -d "$pathname" ]; then
            # this is a directory (or a symbolic link to one), recurse
            printf 'Entering %s\n' "$pathname"
            find_smaller "$pathname" "$size"
        fi
    done
}

find_smaller "$@"

In the code above, $pathname will be not only the filename of the current file or directory that we're looking at, but also its path relative to the starting directory.

Note also the quoting of all variable expansions. Without quoting the $pathname variable, for example, you would invoke filename globbing if a filename contained characters like * or ?.

See also:

Using bash and its globstar shell option. With this option set, the ** glob pattern matches all the pathnames beneath the given directory. This means we don't have to explicitly walk the directory structure in our script:

#!/bin/bash

dir="$1"
size="$2"

shopt -s globstar

for pathname in "$dir"/**; do
    [ ! -f "$pathname" ] && continue

    filesize=$( wc -c <"$pathname" )

    if [ "$filesize" -lt "$size" ]; then
        printf 'Found %s, size is %d\n' "$pathname" "$filesize"
    fi
done

Rather than writing your own directory tree walker, you may instead use find. The following find command does what your code tries to do:

find /home/161161 -type f -size -100c

As a script:

#!/bin/sh
dir=$1
size=$2
find "$dir" -type f -size -"$size"c

The only slight difference between the explicit directory walker and the find variation is that find (when used as above) will ignore symbolic links, while the shell function above it will resolve symbolic links, possibly causing directory loops to be traversed infinitely, or the same data to be counted more than once.

Using find, the contents of the files will not be read to figure out the file size. Instead a lstat() library call will be made to query the filesystem for the file's size. This is many times faster than using wc -c!

On most (but not all) Unices, you may also use the command line utility stat to get the file size. See the manual for this utility on your system for how to use it (it works differently on Linux and BSD).

stat -c %s instead of wc -c. Or did you want to keep as much of the original code as possible in your first rewrite? — Chris Davies, Jun 15 '18 at 06:45
@roaima I wanted to use wc since that was in the title, also, my stat does not have -c (I'm on OpenBSD and the correct stat call would be stat -f %z filename). We don't know what Unix this is supposed to run on. — Kusalananda, Jun 15 '18 at 06:56
Ah. I hadn't checked POSIX, and just assumed -c was standard. My bad. I might well be on my way (back to) one of the BSDs next refresh. — Chris Davies, Jun 15 '18 at 08:20
That seemed to solve the problem. However now I'm left with even more question. Like how is it possible for the script to do a cd .. before the loop has finished when the cd .. command is written after the loop? — David Mathers, Jun 15 '18 at 20:48
@DavidMathers Look at your code. The cd .. is executed just before the end of the loop. — Kusalananda, Jun 15 '18 at 21:32

wc -c not working in a loop in the script

1 Answers1