Issues related to grep command looping for multiple csv files

Question

I made a for loop script as shown below,

for file in *.csv
do
grep raxA $file > new_${file}
done && 
mkdir raxA && mv new_* raxA &&
for file in *.csv
do
grep raxB $file > new_${file}
done && 
mkdir raxB && mv new_* raxB &&
for file in *.csv
do
grep raxC $file > new_${file}
done && 
mkdir raxC && mv new_* raxC

This script works when all the keywords found in the csv file, but it fails when any one of the keywords is missing from the csv files. Could you please help me to make this work, when grep specified keyword misses in the csv files. Thank you in advance.

Answer 1

Sounds like you want something like:

mkdir -p raxA raxB raxC &&
  awk '
    /raxA/ {print > ("raxA/new_"FILENAME)}
    /raxB/ {print > ("raxB/new_"FILENAME)}
    /raxC/ {print > ("raxC/new_"FILENAME)}' ./*.csv

Answer 2

Your problem is connecting the commands with an AND (&&) operator after the grep command. Note the exit status of grep!

From man grep:

EXIT STATUS

Normally the exit status is 0 if a line is selected, 1 if no lines were selected, and 2 if an error occurred.

With an exit status of 1, the following commands are not run. See man bash

command1 && command2

command2 is executed if, and only if, command1 returns an exit status of zero (success).

Further hints:

1. Your for-loops may be combined in one
2. The keywords could become a for-loop and be nested with the file loop
3. First create the directories and directly redirect the output there to skip the mv

Answer 3

As others already pointed the issue with your command, I'm giving you an alternative way of what you are doing with shell-loops that should be avoided.

with awk you can also do search for the patterns and split each pattern matched into multiple files.

awk '{ matched=match($0, /rax[ABC]/); };
     matched{ dirname=substr($0, RSTART, RLENGTH);
              system("mkdir -p " dirname);
              print >(dirname"/new_"FILENAME);
              matched=0;
}' infile

see man awk for the match(), substr(), system() functions.

since your file seems it's a .csv, and you may wanted to looking for the patterns in a specific column rather than in a whole line, add -F, to the command (specifying the input file's field separator) and replace $0 (representing the whole line in awk) with corresponding column number in .csv file, for example to match patterns in first column use $1, second column with $2, third with $3, so on.