0

I run a shell script at reboot, by having it in my crontab:

@daily apt-get update && apt-get upgrade -y
@reboot bash /root/Start.sh
@reboot /bin/bash -c 'sleep 10 && /bin/mount -a'
...

The shell script is executed, since it launches other .py-scripts successfully, but it does not launch all scripts (all except trouble.py) from within.

However, when I run the shell script myself, with: bash Start.sh

all scripts within the .sh are executed.

This is my shell-script:

sleep 15
python3 /root/.../trouble.py > /root/error_log_bash.txt
python3 /root/.../working1.py &
python3 /root/.../working2.py &

I've tried so far:

  • to change the order within the .sh
  • different amounts of sleep before and after
  • chmod +x the .py script
  • to launch another similar script instead, which works as expected.
  • to check the error_log_bash.txt, but it was empty
  • sudo the launch in the .sh

Since it works with another script I was considering whether the script is the problem. However, since it works manually I find this difficult to imagine. Am I missing something? What could provoke such a behavior? I'm working on debian, as root.

#Edit @Waltinator Thank you for pointing out the differences between cron and regular env. When I run sh /root/Start.sh instead of bash /root/Start.sh with the manual console I receive no error.

Marco
  • 101
  • Impossible to tell without seeing the content of trouble.py. It could be a number of things, the first that springs to mind is that something it does requires a tty .. – tink Nov 29 '23 at 04:01
  • I think you need your sleep in the cron job - not in the script itself; i.e. @reboot sleep 15; bash /root/Start.sh – Seamus Nov 29 '23 at 04:01
  • 2
    I think the location of the sleep is irrelevant. Likely either the path is wrong or some of those require a tty. It's a bad idea to run apt upgrade from cron, things in there frequently want a tty. This looks like an XY problem. There are better ways to do most of this. In particular, mount -a should be done automatically at boot without cron and it may have dependencies that systemd would be better at sorting out. – user10489 Nov 29 '23 at 04:50
  • when removing the sleep from the shell script, behavior is the same. trouble.py starts a websocket, to start recording data. Other scripts do too, also with websockets and run without Problems. But why would the behavior be different when running it with cron and running it manually? - How do I check the tty requirement? – Marco Nov 29 '23 at 04:55
  • This "Jobs run through cron aren't run in the same runtime environment that you have on your desktop." makes me confident, that I'm not crazy. However, I'm not sure how to proceed from here. when running echo "=== id ===";id;echo "=== set ===";set;echo "=== env ===";env | sort;echo "=== alias ===";alias I get a huge output. which I'm also not sure how to find the differences. – Marco Nov 29 '23 at 19:38
  • So, trouble.py (that's run in the foreground, without &) runs ok, but the working.py scripts (that are run in the background with &) don't? Is it correct that you deduced they didn't run since you didn't see them active later? Or did you try to check if they started to begin with? What happens if you have the scripts open some file and write something to it right when they start, just to make a note of the fact they did start? What happens if you put something like sleep 900 at the end of the script, after the background processes are launched? – ilkkachu Nov 30 '23 at 06:48
  • 1
    What I'm wondering here is if it's an issue with starting background processes from cron. Systemd has some features to e.g. kill lingering processes when a user logs out, unless configured to leave them, and I wonder if there might be something similar with cron?? Not that I could replicate an issue like that, but, anyway... – ilkkachu Nov 30 '23 at 07:59
  • @ilkkachu. Thanks. let we clarify. when I run bash Start.sh or sh Start.sh from the terminal the script runs, and is visible with top -c. When I execute the same command with the cronjob only two of the three python scripts are visible with top-c – Marco Dec 01 '23 at 06:31

1 Answers1

1

This is more of a comment than an answer but there are only so many lines available in a comment

  1. It's not at all clear what you're trying to achieve with your situation. You've explained that it's not working, but not what you want instead.
  2. You're missing the first line of your script, which should be used to identify the interpreter for the code you've written. In your case #!/bin/sh could be sufficient.
  3. You're not capturing stderr from the output of trouble.py. That might or might not be a problem but unless you capture it you won't know.
  4. You're not capturing any output at all from your two background jobs.
  5. You're not capturing any error output from the script. For example, if python3 can't be found you won't get any output anywhere useful.

Here's a sample replacement script:

#!/bin/sh
sleep 15
python3 /root/.../trouble.py  >/root/trouble.log 2>&1
python3 /root/.../working1.py >/root/working1.log 2>&1 &
python3 /root/.../working2.py >/root/working2.log 2>&1 &

And here's an alternative that sends all output (stdout and stderr for the script and its three Python programs) to the single file trouble.log

#!/bin/sh
sleep 15
exec >/root/trouble.log 2>&1

python3 /root/.../trouble.py
python3 /root/.../working1.py &
python3 /root/.../working2.py &

Then make the script executable: if it's called runit then chmod a+rx runit would do. And now you can execute the script by path, for example /root/runit

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
  • thank you for your answer! the definition of #!/bin/sh alone did not work. But thanks to the error-log I found out what errors are thrown by the python script. Now I'm working to solve these. – Marco Dec 01 '23 at 06:13
  • I narrowed it down further, and I think it fits well into the fact, that environment variables are not necessarily the same. The SSL-certificates seem to cause a problem.

    trouble.log content:

    error from callback <function main_loop.<locals>.on_error at 0x7fd9bcee8048>: [SSL: CERTIFICATE_VERIFY$

    So I added this to the cronjob definition:

    SSL_CERT_DIR=/etc/ssl/certs```
but it still doesn't work. 
Does this warren't a new question?
    – Marco Dec 01 '23 at 07:00
  • @Marco: the statement "it doesn't work" is no use to anyone - there's no detail from which to give advice. But now you've explained at least some of what's not working I can tell you that you need to read some of the answers on the question How can I run a cron command with existing environmental variables? – Chris Davies Dec 01 '23 at 08:19