How best to start a background resource intensive task in bash?

Question

I'm running some long running tasks (15 minutes up to a day) in Ubuntu 18.04 under WSL2, itself under Win11.

A couple of times my terminal has stopped responding and never came back, but I could see from resource usage that the task was still running.

How can I best start this task so it's a background task that will persist even if my interactive session ends prematurely?

Important requirements are:

The task must be running in the background
I should be able to reconnect to the session if something happens to it
Output should be logged somewhere in case I don't reconnect in time
The task is resource intensive, so the method of starting it should not decrease performance substantially
Ideally I should be able to do all of the above locally as well as over SSH

Just in case it's important, the actual task is a tensorflow job running under python 3.9, within an anaconda environment.

Does this answer your question? Keep processes running after SSH session disconnects or How can I run a command which will survive terminal close? — pLumo, Feb 14 '22 at 21:39
Does this answer your question? How can I run a command which will survive terminal close? — NotTheDr01ds, Feb 15 '22 at 20:30

score 1 · Accepted Answer · answered Feb 14 '22 at 21:35

1

You can use screen for that.

You can simply reconnect to a session if the connection gets lost via screen -r.

Sending the task to background as well as logging can additionally be done with & and >> if necessary.

answered Feb 14 '22 at 21:35

mashuptwice

1 Answers1