Adding suffix to filename during for loop in bash

Question

This has probably been asked but the existing answers don't make sense to me. I am running a loop over multiple CSV files in a folder. This folder has non-csv files also and hence I must specify the csv files explicitly. I am modifying the csv files in several ways using awk, cut etc. After I am done, I want to redirect the output to new csv files with slightly modified names as follows:

TLC_2017.csv > TLC_2017_prepped.csv

Here's a MWE:

for file in TLC*.csv; do
    cut -d, -f2- ${file} > ${file}_prepped
done

The problem is that I am getting new files with names such as TLC_2017.csv_prepped. In other words, the suffix is being added to the file extension and not the name. How do I add the suffix to the filename and not the extension?

If an answer to an existing question solves my problem, kindly don't just link to it but also provide a little bit of explanation so that I can figure it out. Thanks!

Strictly speaking, there's no such thing as a "file extension" in Unix type file systems; that's a concept from MS-DOS that Windows inherited and maintains. Linux / Unix does have conventions for file naming that look like file extensions, and we may informally refer to them as such, but there are very few syntactical restrictions on file names in Linux / Unix file systems. — mpez0, Jul 24 '20 at 13:52

Quasímodo · Accepted Answer · 2020-07-24T12:23:41.393

for file in TLC*.csv; do
    cut -d, -f2- "${file}" > "${file%.*}_prepped.csv"
done

${file%.*} removes everything after the last dot of $file. If you want to remove everything after the first dot, then you would use %%.

Likewise, ${file#*.} (${file##*.}) removes everything before the first (last) dot.

Read more in Parameter Expansion, Bash manual.

And remember, always quote your variables. You can use shellcheck to help you debugging your scripts. It would warn you about unquoted variables.

If looping over files with different extensions, the target file extension cannot be hardcoded as done above, so ${file##*.} is needed. As minimal example, which you can try in an empty test directory, this makes a _prepped copy to every file with an extension:

touch A.file.txt B.file.dat C_file.csv D_file.ko E
for file in *.*; do
    noext=${file%.*}
    ext=${file##*.}
    cp "$file" "${noext}_prepped.${ext}"
done

After execution,

$ ls
A.file_prepped.txt  A.file.txt  B.file.dat  B.file_prepped.dat
C_file.csv  C_file_prepped.csv  D_file.ko  D_file_prepped.ko  E

Adding suffix to filename during for loop in bash

1 Answers1