Too many shebang (script declaration) lines --- any way to reduce their amount?

Question

I have a project comprised of about 20 small .sh files. I name these "small" because generally, no file has more than 20 lines of code. I took a modular approach because thus I'm loyal to the Unix philosophy and it's easier for me to maintain the project.

In the start of each .sh file, I put #!/bin/bash.

Simply put, I understand script declarations have two purposes:

1. They help the user recall what shell is needed to execute the file (say, after some years without using the file).
2. They ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

---

When a project starts to grow from say 5 files to 20 files or from 20 files to 50 files (not this case but just to demonstrate) we have 20 lines or 50 lines of script declarations. I admit, even though it might be funny to some, it feels a bit redundant for me to use 20 or 50 instead say just 1 per project (maybe in the main file of the project).

Is there a way to avoid this alleged redundancy of 20 or 50 or a much greater number of lines of script declarations by using some "global" script declaration, in some main file?

Answer 1

Even though your project may now be solely consisting of 50 Bash scripts, it will sooner or later start accumulating scripts written in other languages such as Perl or Python (for the benefits that these scripting languages have that Bash does not have).

Without a proper #!-line in each script, it would be extremely difficult to use the various scripts without also knowing what interpreter to use. It doesn't matter if every single script is executed from other scripts, this only moves the difficulty from the end users to the developers. Neither of these two groups of people should need to know what language a script was written in to be able to use it.

Shell scripts executed without a #!-line and without an explicit interpreter are executed in different ways depending on what shell invokes them (see e.g. the question Which shell interpreter runs a script with no shebang? and especially Stéphane's answer), which is not what you want in a production environment (you want consistent behaviour, and possibly even portability).

Scripts executed with an explicit interpreter will be run by that interpreter regardless of what the #!-line says. This will cause problems further down the line if you decide to re-implement, say, a Bash script in Python or any other language.

You should spend those extra keystrokes and always add a #!-line to each and every script.

---

In some environments, there are multi-paragraph boilerplate legal texts in each script in each project. Be very happy that it's only a #!-line that feels "redundant" in your project.

Answer 2

You misunderstand the point of #!. It's actually a directive to the operating system to run this program under the defined interpreter.

So a script could being #!/usr/bin/perl and the resulting program can be run as ./myprogram and the perl interpreter could be used. Similar #!/usr/bin/python would cause a program to run under python.

So the line #!/bin/bash tells the OS to run this program under bash.

Here's a example:

$ echo $0
/bin/ksh

$ cat x
#!/bin/bash

ps -aux | grep $$

$ ./x
sweh      2148  0.0  0.0   9516  1112 pts/5    S+   07:58   0:00 /bin/bash ./x
sweh      2150  0.0  0.0   9048   668 pts/5    S+   07:58   0:00 grep 2148

So, despite my shell being ksh, the program "x" runs under bash because of the #! line, and we can see that explicitly in the process listing.

Answer 3

On .sh

What is bad is the .sh at the end of the file-name.

Imagine that you rewrite one of the scripts in python.

Well now you have to change the first line to #!/usr/bin/python3 this is not so bad, as you had to change every other line of code in the file as well. However you also have to change the file name from prog.sh to prog.py. And then you have to find everywhere in the other scripts, and all of your user scripts (the ones that you did not even know existed), and change them to use the new script. sed -e 's/.sh/.py/g' may help for the ones you know about. But now your revision control tool is showing a change in lots of unrelated files.

Alternatively do it the Unix way and name the program prog not prog.sh.

On #!

If you have the correct #!, and set the permission/mode to include execute. Then you don't need to know the interpreter. The computer will do it for you.

chmod +x prog #on gnu adds execute permission to prog.
./prog #runs the program.

For non gnu systems read the manual for chmod (and learn octal), maybe read it anyway.

Answer 4

[The hashbang lines] ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

It's probably worth pointing out that this is quite wrong. The hashbang line in no way prevents you from trying to feed the file to an incorrect shell/interpreter:

$ cat array.sh
#!/bin/bash
a=(a b c)
echo ${a[1]} $BASH_VERSION
$ dash array.sh
array.sh: 2: array.sh: Syntax error: "(" unexpected

(or similarly with awk -f array.sh etc.)

---

But in any case, another approach to modularity would be to define shell functions in the files, one function per file, if you like, and then source the files from the main program. That way, you wouldn't need hashbangs: they'd be treated as any other comments, and everything would run using the same shell. The downside would be that you'd need to source the files with the functions (e.g. for x in functions/*.src ; do source "$x" ; done), and they'd all run using the same shell, so you couldn't directly replace one of them with a Perl-implementation.

Answer 5

Is there a way to avoid this alleged redundancy of 20 or 50 or a much greater number of lines of script declarations by using some "global" script declaration, in some main file?

There is - it is called portability or POSIX compliance. Strive to always write scripts in portable, shell-neutral way, source them only from a script that uses #!/bin/sh or when you use /bin/sh interactively (or at very least a POSIX-compliant shell such as ksh).

However, portable scripts don't save you from:

• needing to use features of one of the shells ( ilkkachu's answer is an example)
• needing to use scripting languages more advanced than shells (Python and Perl)
• PEBKAC error: your script falls into hands of user J.Doe who runs your script not as you intended it, for example they run /bin/sh script with csh.

---

If I may express my opinion, "avoiding redundancy" by eliminating #! is wrong goal. The goal should be consistent performance and actually a working script that works, and considers corner cases, follows certain general rules like not-parsing-ls or always-quoting-variables-unless-need-word-splitting.

They help the user recall what shell is needed to execute the file (say, after some years without using the file).

They're really not for user - they're for operating system to run proper interpreter. "Proper" in this case, means OS finds what is in shebang; if the code below it for wrong shell - that's author's fault. As for user memory, I don't think "user" of programs such as Microsoft Word or Google Chrome ever needed to know what language programs are written in; authors may need.

Then again, scripts written portably may eliminate need to even remember which shell you used originally, because portable scripts should work in any POSIX-compliant shell.

They ensure that the script runs only with a certain shell (Bash in that case) to prevent unexpected behavior in case another shell was used.

They don't. What actually prevents unexpected behavior is writing portable scripts.

Answer 6

The reason why you need to put #!/bin/bash at the top of each script is that you're running it as a separate process.

When you run a script as a new process, the OS sees that it's a text file (as opposed to a binary executable), and needs to know what interpreter to execute. If you don't tell it, the OS can only guess, using its default setting (which an administrator can change). So the #! line (plus adding executable permissions, of course) turns a simple text file into an executable that can be run directly, with confidence that the OS knows what to do with it.

If you want to remove the #! line, then your options are:

1. Execute the script directly (as you're doing now), and hope that the default interpreter matches the script - you're probably safe on most Linux systems, but not portable to other systems (e.g. Solaris), and it can be changed to anything (I could even set it to run vim on the file!).

2. Execute the script using bash myscript.sh. This works fine, but you have to specify the path to the script, and it's messy.

3. Source the script using . myscript.sh, which runs the script within the current interpreter process (i.e. not as a sub-process). But this will almost certainly require modifications to your scripts, and makes them less modular / reusable.

In cases 2 and 3, the file doesn't need (and shouldn't have) execute permissions, and giving it a .sh (or .bash) suffix is a good idea.

But none of those options looks good for your project, as you said you want to keep your scripts modular (=> independent and self-contained), so you should keep your #!/bin/bash line at the top of the scripts.

In your question, you also said you're following the Unix philosophy. If that's true, then you should learn what the #! line is really for, and keep using it in every executable script!