I would like to solve the following issue about submitting a job that has been parallelised to a specific node.
Let me start with explaining the structure of my problem
I have two very simple Matlab scripts
1) main.m
clear
rng default
P=2;
grid=randn(4,3);
jobs=1;
2) f.m
sgetasknum_grid=grid(jobs*(str2double(getenv('SGE_TASK_ID'))-1)+1: str2double(getenv('SGE_TASK_ID'))*jobs,:); %jobsx3
result=sgetasknum_grid+1;
filename = sprintf('result.%d.mat', ID);
save(filename, 'result')
exit
What I want to do is:
Run main.m;
then, run f.m 4 times, allowing for parallel execution of 2 tasks at each time
Everything should be executed on node A
Here's how I implement the steps above
1) I save main.m
and f.m
into a folder named My_folder
2) I create the script td.sh
as below and save it into the folder My_folder
#!/bin/bash -l
#$ -S /bin/bash
#$ -l h_vmem=5G
#$ -l tmem=5G
#$ -l h_rt=480:0:0
#$ -cwd
#$ -j y
#$ -N try
date
hostname
J=4 #number tasks
N=2 #number tasks executed in parallel
export SGE_TASK_ID
SGE_TASK_ID=1
n=0
while [ "$SGE_TASK_ID" -le "$J" ]; do
if [ "$n" -eq "$N" ]; then
wait -n # as soon as one task is done, refill it with another
n=$(( n - 1 ))
fi
printf 'Task ID is %d\n' "$SGE_TASK_ID"
/share/.../matlab -nodisplay -nodesktop -nojvm -nosplash -r "main; ID=$SGE_TASK_ID; f; exit" &
SGE_TASK_ID=$(( SGE_TASK_ID + 1 ))
n=$(( n + 1 ))
done
wait
3) I go into the terminal and type ssh username@A
, then cd /.../My_folder
, then bash td.sh
Problem: I get the following error
td.sh: line 26: wait: -n: invalid option
wait: usage: wait [id]
As noticed in the comments below, the issue is that the version of bash on @A is old (the -n option was added to the wait builtin in 4.3) and the sysadmin can't update it. The latest version possible is bash 4.1.
Thus, could you suggest a way to replace wait -n
?
-n
option was added to thewait
builtin in 4.3 – steeldriver Dec 07 '18 at 13:11wait -n
with? – Star Dec 07 '18 at 13:15bash
4.3 or newer are installed elsewhere on the system, or if an admin is happy to upgrade the existingbash
to a later version. – Kusalananda Dec 07 '18 at 13:17#!…
is magic. If you dochmod +b td.sh
then you can run it as a normal executable e.g../my.sh
. And remove the file extension, as when you have to re-implement in python, you don't want to have to rename it. – ctrl-alt-delor Dec 11 '18 at 13:21