Bash Script general questions

Question

The first part of what i am mucking around with simply creates a folder entitled command_manuals that contain text file copies of all the commands available at the bash shell, the filenames which are the command names themselves.Aside from some unusual error messages that do not affect the script from achieving it's goal, this part I have managed to get, but I would prefer to go to the most crippling issues first so I will leave it out for now.

Those without manuals available have empty text files still, which brings me the goal of the second part, which is to delete these "0 size text files" and compile a single list for which I want to use in a later part, that checks the "undocumented man 7" and other parts every time i execute apt-update and uses one of the well known pattern searching packages to see if anything new has been added regarding those commands missing manuals, ra,ra,ra anyway not really important until i get over this drama of part two, which thus far i have successfully written an if condition into the for loop that simply prints out the subset of the folder's contents with a size of zero, but the trouble starts when i try to replay this basic echo command between "then" and "fi" that deletes the empty files.

But this also brings me to an additional question concerning another odd thing i noticed, the basic template script that works as i desired, is missing the standard #!/bin/bash that is always required as the first line for .sh scripts in my previous experience, why?

template_for_if_delete.sh

TOTALnum=$(wc -l < filesizes.txt);
for i in `seq 1 $TOTALnum`;
do
size=$(sed -n ''$i','$i'p' filesizes.txt)
if [ $size -eq 0 ];
then
echo $(sed -n ''$i','$i'p' filenames.txt);
fi
done;

So far for the actual script for the second task i have:

secondpart.sh

#!/bin/bash
cd;
cd command_manuals;
rm filenames.txt;
ls -1 > filenames.txt;
TOTALnum=$(wc -l < filenames.txt);
for i in `seq 1 $TOTALnum`;
do filename=echo $(sed -n ''$i','$i'p' filenames.txt);
size=$(stat -c '%s' $filename);
if [ $size -eq 0 ];
then
rm $(locate $(sed -n ''$i','$i'p' filenames.txt));
fi
done;

which produces the output (in repetition a number of times as per the quantity defined in the for loop above):

Try 'stat --help' for more information.
./secondpart.sh: line 9: [: -eq: unary operator expected
./secondpart.sh: line 8: x86_64-linux-gnu-gold.txt: command not found
stat: missing operand

I then tried to change to complex parameter expansion for the variable 'size' in the if statement:

#!/bin/bash
cd;
cd command_manuals;
rm filenames.txt;
ls -1 > filenames.txt;
TOTALnum=$(wc -l < filenames.txt);
for i in `seq 1 $TOTALnum`;
do filename=echo $(sed -n ''$i','$i'p' filenames.txt);
size=$(stat -c '%s' $filename);
if [ ${#size} -eq 0 ];
then
rm $(locate $(sed -n ''$i','$i'p' filenames.txt));
fi
done;

which produced the concerning output asking me if i want to delete some random unrelated stuff, at which point i decided ok i better post a question before i mess up my virtual machine thats been running so well for me:

./secondpart.sh: line 8: filenames.txt: command not found
stat: missing operand
Try 'stat --help' for more information.
rm: missing operand
Try 'rm --help' for more information.
./secondpart.sh: line 8: file-roller.txt: command not found
stat: missing operand
Try 'stat --help' for more information.
rm: missing operand
Try 'rm --help' for more information./code>
./secondpart.sh: line 8: filesizes.txt: command not found
stat: missing operand
Try 'stat --help' for more information.
rm: missing operand
Try 'rm --help' for more information.
./secondpart.sh: line 8: file.txt: command not found
stat: missing operand
Try 'stat --help' for more information.
rm: cannot remove adammvirtual/Win10UbuntuVM001_apt_sources_file.txt':
 No such file or directory
rm: cannot remove '/home/adamvirtual/mapfile.txt': No such file or directory
rm: remove write-protected regular file '/usr/share/doc/alsa-base/driver/Procfile.txt.gz'?

I pressed ctrl+C to escape the prompt.

Please read and follow the formatting guide - using non-standard formatting is going to make it harder for everyone to read your questions. — l0b0, May 22 '19 at 03:52
Just use four leading spaces instead of manually entered HTML code tags. — Wildcard, May 22 '19 at 05:06
Ok I just thought that the tags were for differentiating between verbose output and input code I don't know how to use a Linux debugging package yet but ok no problem that answered the next question I had — Adam Ledger, May 22 '19 at 05:08
It's very unclear what the actual question is here. The only sentence with a question mark appears unrelated to all the extra information about what you're trying to accomplish. To answer that question, though, see Which shell interpreter runs a script with no shebang? — Wildcard, May 22 '19 at 05:09
As for the rest of it—I recommend rather than trying to store every man page as a text file, you learn to use the man command itself. Start with man less (or this abbreviated guide) to learn just to navigate a single man page, and then read man man. man -w in particular might be useful to you; there are a lot of other goodies. You don't have to reinvent the wheel. — Wildcard, May 22 '19 at 05:12
ok that's all very helpful honestly have no intention of reinventing any wheels I have only been learning for about 12 months, and so the vast majority of the actual purpose to what I am doing is really just becoming competent in writing bash scripts and understanding a Unix system the best of my ability — Adam Ledger, May 22 '19 at 05:17
Cool, that's a worthy goal! Start with the "Shell Scripting Basics" linked from my profile. (It's not self-promotion, by the way; they're all questions and answers on this very site.) :) Also the "Bash Pitfalls," also linked from my profile. But most of the problems in your script will be resolved if you understand how quoting works in the shell. — Wildcard, May 22 '19 at 05:19
Yeah it's certainly been a lot of fun so far! i mean long term i want to get to the point i understand a computer algebra system like the one i am familiar with using for the past decade and change but yeah just competence in this department is realistic for now haha — Adam Ledger, May 22 '19 at 06:35
@Wildcard ok thanks got the links bookmarked, I mean I did raise a meta SE question proposing a set up where by people that are at your level of expertise can be actually make themselves available for private tuition at a rate of their choosing in their chosen field via private chat services that already exist, but the post was immediately voted down and I was told I'm a terrible child more ways or more, which I cant argue with really, but still, I feel guilty it doesn't exist — Adam Ledger, May 22 '19 at 07:59

pLumo · Accepted Answer · 2019-05-22T08:48:17.413

The issue is in ./secondpart.sh: line 8, which should set $filename.

stat and rm have $filename as argument, so they are complaining. These errors are follow-ups.

So you need to fix line 8.

You should almost always have set -euo pipefail at the beginning of your scripts to let them exit on the first error and thus to prevent the follow-up errors.

But in general, your script is overly complicated, using wc, a for loop and sed is not a proper way to read a file line by line.

And instead using stat -c %s to determine if file size is zero, you might want to use the file test operator -s, which does exactly that (actually checking the opposite, if file size is not zero).

Use e.g.:

xargs -a filenames.txt -I{} sh -c '[ -s "$1" ] || rm -i "$1"' xargs-sh {}

or

readarray filenames < filenames.txt
for filename in "${filenames[@]}"; do
    [ -s "$filename" ] || rm -i "$filename"
done

or

while IFS= read -r filename; do
    [ -s "$filename" ] || rm -i "$filename"
done < filenames.txt

In general, having file names line by line in a text file is not a good idea, because file names are allowed to have newline characters in their name. Instead, you should always use null character (\0) as file delimiter.

Great thankyou very much the set -euo pipefail<\code> advice in your first few lines is going to boost my productivity immeasurably for starters, pretty much sitting there watching a lot of unnecessary script's output run its course without that thus far! — Adam Ledger, May 22 '19 at 10:23
Ok thanks yep I'm taking a look at the rest now, with regard to my code being overly complicated, I was thinking along the lines of some kind of customization of the bash history programs, that keep a tally of code length and task specifications that I label in # comments in each script, outputting the average for the "category" the current one I am working in is deemed to be, I'm sure something like this must have been thought of before, am I explaining myself clearly enough here @pLumo? — Adam Ledger, May 24 '19 at 01:34
something like that that also takes computation time into account anyway — Adam Ledger, May 24 '19 at 01:34

Bash Script general questions

1 Answers1