7

I am having an issue trying to use parallel command on Ubuntu 10.04. I looked up the parallel documentation and few of the commands seem to run. In all cases I just get the command prompt back without any action being taken. e.g. I was trying to compress a bunch of files using bzip2


17:32 farhat HarshaNaveen$ parallel bzip2 ::: *fastq 
17:33 farhat HarshaNaveen$ ls *fastq|parallel bzip2 {}

Neither of these commands worked. Nor was there any error. The example given in the man file worked fine though. 


18:58farhat HarshaNaveen$ parallel sh -c "echo hi; sleep 2; echo bye" -- 1 2 3
hi
hi
hi
bye
bye
bye
18:58farhat HarshaNaveen$

What am I doing wrong?

Jakuje
  • 21,357
Farhat
  • 297
  • 1
    You are probably using moreutils. As you are dealing with fastq-files you definitely want to install GNU Parallel because of the --pipe option. Watch http://www.youtube.com/watch?v=1ntxT-47VPA to see an example with fasta-files. – Ole Tange Jun 21 '11 at 15:12

3 Answers3

3

Your first try is closest to being correct, but why the :::? If you change ::: to --, it will do what you want.

parallel has a specific, unusual structure to its command line. In the first half, you give it the command you want to run multiple times, and the part of the command line that will be the same every time. In the second half, you give it the parts that will be different each time the command is run. These halves are separated by --. Some experimentation shows that if parallel doesn't find the second half, it doesn't actually run any commands.

It's probably worth re-reading the man page carefully. Man pages have a terse, information-dense style that can take some getting used to. Also try reading some pages for commands you're already familiar with.

Jander
  • 16,682
  • Thanks, it is a command that I expect to use quite a bit so it is good to know the intricacies. – Farhat Feb 20 '11 at 05:08
  • The man page actually specifies ::: for the first example. The only difference I had is using bzip2 instead of gzip. – Farhat Feb 20 '11 at 07:14
  • This part of GNU parallel documentation should be updated then http://www.gnu.org/software/parallel/man.html#example__reading_arguments_from_command_line – Farhat Feb 20 '11 at 07:25
  • Ah! I understand now. It looks like there are at least two versions of parallel with different syntax. The one we apparently both have on our systems is from the moreutils package and is not written by GNU. Something's still very strange, though, if you can type man parallel at the command line and get the version with the ::: syntax. – Jander Feb 20 '11 at 07:49
  • Wow. That's a very strange bug to run into. – Farhat Mar 23 '11 at 16:12
3

By default in ubuntu the parallel behaves as Tollef's implementation (why?). You can check /etc/parallel/config. On the command line you can override this setting and switch it into GNU mode with --gnu. With this switch all examples from the man page should work fine.

Darko Veberic
  • 128
0

I assume you are trying to run both bzip2 ::: *fastq and ls *fastq|parallel bzip2 {} in parallel. Any compelling reason you don't just open up two terminal windows and run them each in their own window?

David Oneill
  • 1,208
  • 1
    Depending on how many processors he has, and how many files *fastq matches, he would need quite a few more than two terminal windows. Check out the parallel man page. – Jander Feb 16 '11 at 15:51