run same command for multiple file at the same time

Question

I need to run same command separately for each file at the same time.

I've tried

for file in *.txt; do ./script <"$file"; done

but It starts the first one and waits until it get finished then goes to the next one.

I need to start them all together.

check out http://www.gnu.org/software/parallel/ – Sundeep Sep 21 '16 at 05:39 — Sundeep, Sep 21 '16 at 05:39

score 4 · Accepted Answer · answered Sep 21 '16 at 07:01

If script doesn't require any input from the user, you could use the shell's job processing features to run it in the background

for file in *.txt; do ./script <"$file" & done

When you append & to a command it's run in the background. Look up job control in the man page for bash (or your preferred shell) for details.

score 3 · Answer 2 · answered Sep 21 '16 at 07:31

3

ls *.txt > list
cat list | parallel './script {}' &
wait

First list out the txt files in list file. parallel command will run the script on each file_name in background. Last wait command will wait for all the background jobs to finish then you can proceed for further commands.

answered Sep 21 '16 at 07:31

SHW

14,786
14
66
101

What about ls -d *.txt | parallel './script {}'? I'm pretty sure you don't need to put parallel in the background only to wait for it. – Chris Davies Sep 21 '16 at 09:15
don't parse ls: printf "%s\n" *.txt | parallel './script {}' – glenn jackman Sep 21 '16 at 19:37
@glennjackman what's the difference in this situation between ls -d *.txt and printf "%s\n" *.txt? Both break on file names containing \n. Both depend on parallel to "do the right thing" with weird characters in the filenames. – Chris Davies Sep 21 '16 at 20:05
You don't need ls or printf for this... parallel alone can do it. – don_crissti Sep 21 '16 at 22:07

run same command for multiple file at the same time

2 Answers2