2

What is it?

I have set up my own job queuing system which works fine. A user can enter the number of cores (threads) they want to use and the maximum time after which their program will be sigkilled. The queue-system communicates via files. In particular, the following executables and files exist:

Files

root jobq     0 Jun 29 10:53 .commands
root jobq 45591 Jun 29 10:52 log.txt
root jobq     0 Jun 29 10:53 .queue
root jobq    48 Jun 29 10:52 .status
root jobq    30 Jun 28 08:40 .system_buffer

Executables

root jobq    52 Jun 22 08:36 .jobq_exec
root jobq 1100896 Jun 22 08:36 jobq_server
root jobq     271 Jun 22 08:36 jobq_server_start
root jobq  364072 Jun 22 08:36 jobq_server_stop
root jobq  363928 Jun 22 08:36 jobq_status
root jobq  368112 Jun 22 08:36 jobq_stop
root jobq  372512 Jun 22 08:36 jobq_submit

The way it works:

jobq_server_start is a simple shell script, that checks if a process with name jobq_server is running, and if not, will try to start it.

jobq_server_stop will write to the .commands file.

jobq_status will write to the .commands file and read from the .status file.

jobq_submit will write to the .queue file.

.jobq_exec is a shell script that will start a job via taskset, which is passed by command line argument and redirects the outputs

.jobq_stop will write to the .commands file

jobq_server is a binary file, that will read and write to .commands, log.txt, .queue, .status and .system_buffer.

Question

Currently everything is used by one user (jobq), which is shared by different people. I'd like to set up file restrictions, so that

  • each user can only run the jobq_submit, jobq_status and jobq_stop commands as themselves (so they need to login as for example johndoe, not as jobq). If possible, they should not be allowed to edit the text files directly (.commands, .queue).
  • each job will start as the user who submitted it (and as such, can only stop jobs that they started). This will presumably mean that the .job_exec script needs to be altered (see below), to switch to the user. I assume this also means that the server can only be started as root, which is fine.

Script Info

.jobq_exec

#!/bin/bash
cd $1
$5 1> $2 2> $3 &
echo $! > $4 2>&1

usage (this is executed by the server binary)

.job_exec 1:working_directory 2:out_file 3:err_file 4:buffer_file 5:'taskset -ac 1,2,3,4 command cmd_args...'

(added argument numbers for clarity)

Additional info

I'm running Linux Mint 20.3 Una

uname -r
5.4.0-105-generic
infinitezero
  • 207
  • You will want to quote the variables in that script in case any of the file names or paths contain spaces or globbing characters. See Why does my shell script choke on whitespace or other special characters?. Also, please [edit] your question and tell us what operating system this will be used on. – terdon Jun 29 '22 at 11:01
  • There are so many good free job queuing systems... slurm, pbs, sge, etc.. why write one? – user10489 Jun 29 '22 at 11:20
  • 1
    @user10489 Because I tried three days to install pbs and it wouldn't work. Even their support couldn't get it running. So, I wrote it as a simple tool / excercise. Even with testing/bug fixing this took only 1/3 of the time I tried to get pbs to work. – infinitezero Jun 29 '22 at 11:26
  • What about (like other job queing daemons I am thinking of cups in particular) creating a special group let's say jobq, set desired permissions to that group (x for jobq_submit, jobq_status and jobq_stop) and have entitled user members of that group ? – MC68020 Jun 29 '22 at 12:07
  • @infinitezero I suggest trying again with slurm. In my experience, it's a lot easier to get up and running than pbs, and just works. if you're used to pbs/torque style commands, it even has a torque compatibility package so you can run qsub, qhold, qdel, qstat, etc as well as slurm's own sbatch, sstat, sinfo, etc commands. See also https://en.wikipedia.org/wiki/Comparison_of_cluster_software for more alternatives, incl. several with free software licenses. – cas Jun 29 '22 at 14:32
  • pbs and sge are common but they both suck and are dated. Try slurm, it sucks less and is relatively easy to set up. – user10489 Jun 30 '22 at 02:14

0 Answers0