Why isn't this bash script working (variable assignment inside for loop)?

Question

I'm pretty new to bash scripting, so apologies if this is obvious!

I'm trying to create a bash script to traverse a bunch of files I have of the format ID1.1.fq.stuff, ID1.2.fq.stuff, ID2.1.fq.stuff, ID2.2.fq.stuff .... etc. The script is meant to find files that are paired (both files for ID1, ID2, and so on) and then submit them both together to a program called STAR for downstream processing.

I made the following bash script:

#/!/bin/sh
module load STAR
current_id = ""
current_file = ""
 for fqfile in `ls path/*`; do
  filename = ${fqfile%%.fq*}
  id = ${filename%.*}
  if $id == $current_id; then
   STAR --readFilesIn $current_file $fqfile --outFileNamePrefix ./$id.bam
  else
   current_id = $id
   current_file = $fqfile
  fi
done

When I run it, I get these errors:

[path to $id, without file extensions]: No such file or directory
current_id: command not found
current_file: command not found

What am I doing wrong?

Thank you!

Welcome to SE. In bash there are no spaces between variable , assignment and value. You need to glue that all together. current_id="$id" for example. Also ls is nog meant to be used in scripts. So basically there are quite some issues with your code at the moment. — Valentin Bajrami, Jul 05 '23 at 19:12
Bash pitfall number one. There are other issues, some already noted, others not yet. E.g. if …; then expects a command in place of …; the command may be [ with arguments. Your shebang is not a shebang. — Kamil Maciorowski, Jul 05 '23 at 19:33
Possible duplicate of Spaces in variable assignments in shell scripts — Stéphane Chazelas, Jul 05 '23 at 19:56
drop the script into https://www.shellcheck.net, read the feedback it gives, go back and test all parts of the script one line at a time — ilkkachu, Jul 05 '23 at 20:36
The first line: #/!/bin/sh should be #!/bin/sh and if you want to make use of extra features in Bash, it should probably be #!/bin/bash or #!/usr/bin/env bash. — Sotto Voce, Jul 05 '23 at 20:42

score 1 · Answer 1 · answered Jul 05 '23 at 19:38

I used bash syntax because the question is tagged with bash

Problems with the original script

Iterating over ls output
Ignoring non-matching globs
Incorrect shebang
Did not use double quotes to prevent globbing and word splitting
Did not quote the right-hand side of == to prevent glob matching

#!/usr/bin/env bash
Instructs bash to immediately exit if any command has a non-zero exit status
set -e
Allows patterns which match no files to expand to a null string, rather than themselves
shopt -s nullglob
module load STAR
Don't put spaces around '=' when assigning variables in bash.
current_id=""
current_file=""
Iterating over ls output is fragile. Use globs
for fqfile in path/.fq; do
  filename="${fqfile%%.fq}"
  id="${filename%.}"
  if [[ $id == "$current_id" ]]; then
   STAR --readFilesIn "$current_file" "$fqfile" --outFileNamePrefix "./$id.bam"
  else
   current_id="$id"
   current_file="$fqfile"
  fi
done

If I remove the first line (set -e) to see error messages, I still see the same error as before:
current_id: command not found current_file: command not found

The first error did go away. — nerd_inthecorner, Jul 05 '23 at 20:25
@nerd_inthecorner that's hard to believe. To check if whatever STAR is is causing your problems - what happens if you put echo in front of module load STAR and STAR --readFilesIn? — Ed Morton, Jul 06 '23 at 10:20

Why isn't this bash script working (variable assignment inside for loop)?

1 Answers1

Instructs bash to immediately exit if any command has a non-zero exit status

Allows patterns which match no files to expand to a null string, rather than themselves

Don't put spaces around '=' when assigning variables in bash.

Iterating over ls output is fragile. Use globs