How to read the whole shell script before executing it?

Question

Usually, if you edit a scrpit, all running usages of the script are prone to errors.

Example:

sleep 20
echo test

If you execute this script, bash will read the first line (say 10 bytes) and go to sleep. When it resumes, there can be different contents in the script starting at 10-th byte. It might start executing the middle of a different line, in an entirely different if branch. The running script will be broken.

So, how to read the whole shell script before executing it, so that later edits do not affect the running instance?

Answer 1

Yes shells, and bash in particular, are careful to read the file one line at a time, so it works the same as when you use it interactively.

You'll notice that when the file is not seekable (like a pipe), bash even reads one byte at a time to be sure not to read past the \n character. When the file is seekable, it optimises by reading full blocks at a time, but seek back to after the \n.

That means you can do things like:

bash << \EOF
read var
var's content
echo "$var"
EOF

Or write scripts that update themselves. Which you wouldn't be able to do if it didn't give you that guarantee.

Now, it's rare that you want to do things like that and, as you found out, that feature tends to get in the way more often than it is useful.

To avoid it, you could try and make sure you don't modify the file in-place (for instance, modify a copy, and move the copy in place (like sed -i or perl -pi and some editors do for instance)).

Or you could write your script like:

{
  sleep 20
  echo test
}; exit

(note that it's important that the exit be on the same line as }; though you could also put it inside the braces just before the closing one).

or:

main() {
  sleep 20
  echo test
}
main "$@"; exit

The shell will need to read the script up until the exit before starting to do anything. That ensures the shell will not read from the script again.

That means the whole script will be stored in memory though.

That can also affect the parsing of the script.

For instance, in bash:

export LC_ALL=fr_FR.UTF-8
echo $'St\ue9phane'

Would output that U+00E9 encoded in UTF-8. However, if you change it to:

{
  export LC_ALL=fr_FR.UTF-8
  echo $'St\ue9phane'
}

The \ue9 will be expanded in the charset that was in effect at the time that command was parsed which in this case is before the export command is executed.

Also note that if the source aka . command is used, with some shells, you'll have the same kind of problem for the sourced files.

That's not the case of bash though whose source command reads the file fully before interpreting it. If writing for bash specifically, you could actually make use of that, by adding at the start of the script:

if [[ ! $already_sourced ]]; then
  already_sourced=1
  source "$0"; exit
fi

(I wouldn't rely on that though as you could imagine future versions of bash could change that behaviour which can be currently seen as a limitation (bash and AT&T ksh are the only POSIX-like shells that behave like that as far as can tell) and the already_sourced trick is a bit brittle as it assumes that variable is not in the environment, not to mention that it affect the content of the BASH_SOURCE variable)

Answer 2

You simply need to delete the file (ie copy it, delete it, rename the copy back to the original name). In fact many editors can be configured to do this for you. When you edit a file and save a changed buffer to it, instead of overwriting the file it will rename the old file, create a new one, and put the new contents in the new file. Hence any running script should continue without problems.

By using a simple version control system like RCS which is readily available for vim and emacs, you get the dual advantage of having a history of your changes, and the checkout system should by default remove the current file and recreate it with the correct modes. (Beware of hard-linking such files of course).

Answer 3

Use:

{
  ... your code ...
exit
}

Bash will read the whole {} block before executing it, and the exit directive will make sure nothing will be read outside of the code block.

For scripts that are "sourced" rather than executed, use:

{
  ... your code ...
return 2>/dev/null || exit
}

Answer 4

Proof of concept. Here's a script that modifies itself:

cat <<EOF >/tmp/scr
#!/bin/bash
sed  s/[k]ept/changed/  /tmp/scr > /tmp/scr2

# this next line overwites the on disk copy of the script
cat /tmp/scr2 > /tmp/scr
# this line ends up changed.
echo script content kept
EOF
chmod u+x /tmp/scr
/tmp/scr

we see the changed version print

This is because bash loads keeps a file handle to open to the script, so changes to the file will be seen immediately .

If you don't want to update the in-memory copy, unlink the original file and replace it.

One way to do that is by using sed -i.

sed -i '' filename

proof of concept

cat <<EOF >/tmp/scr
#!/bin/bash
sed  s/[k]ept/changed/  /tmp/scr > /tmp/scr2

# this next line unlinks the original and creates a new copy.
sed -i ''  /tmp/scr

# now overwriting it has no immediate effect
cat /tmp/scr2 > /tmp/scr
echo script content kept
EOF

chmod u+x /tmp/scr
/tmp/scr

If you are using an editor to change the script, enabling the "keep a backup copy" feature may be all that's needed to cause the editor to write the changed version to a new file instead of overwriting the existing one.

Answer 5

Wrapping your script in a block {} is likely the best option but requires changing your scripts.

F=$(mktemp) && cp test.sh $F && bash $F; rm $F;

would be the second best option (assuming tmpfs) the disadvantage is it breaks $0 if your scripts use that.

using something like F=test.sh; tail -n $(cat "$F" | wc -l) "$F" | bash is less ideal because it has to keep the whole file in memory and breaks $0.

touching the original file should be avoided so that, last modified time, read locks, and hard links are not disturbed. that way you can leave an editor open while running the file and rsync won't needlessly checksum the file for backups and hard links function as expected.

replacing the file on edit would work but is less robust because it's not enforceable to other scripts/users/or one might forget. And again it would break hard links.