Reading empty string from CSV column last row in BASH

Question

I'm using the following gawk script to read values from the first column of the csv file file.csv.
I use gawk since I don't want any embedded commas to be ignored.

col=`gawk ' 
BEGIN {
FPAT="([^,]*)|(\"[^\"]*\")+"
}
{print $1 }' file.csv`

However, I noticed that if the empty string/space is in the last row, this method ignores it.

For example, if the file.csv is the following:

col1,col2
"a,a","a,a1" 
"b","b1" 
,"c1"  

The result would be

col1
a,a
b 

instad of

col1
a,a
b

What can I do to fix this issue?

Thank you!

Related post: Reading empty string from CSV file in BASH

Answer 1

As mentioned in the comments under your previous question, this has nothing to do with CSVs or your awk script, it's all about how you're saving the output of a command.

$ printf 'a\nb\n\n'
a
b
$ col=$(printf 'a\nb\n\n')
$ printf '%s' "$col"
a
b$

$ col=$(printf 'a\nb\n\n'; printf x)
$ printf '%s' "$col"
a
b
x$
$ col="${col%x}"
$ printf '%s' "$col"
a
b
$

Note that with the above you're getting the whole output of the command saved in the variable, including the final newline that command substitution would have stripped off. If you want to remove a final newline too then do a subsequent:

$ col="${col%$'\n'}"
$ echo "$col"
a
b
$ printf '%s' "$col"
a
b
$

The reason to remove the x and the \n in 2 steps rather than doing a single col="$(col%$'\n'x}" is that that would fail if the command had produced no output or output that didn't end in a \n because then \nx wouldn't exist in col:

Right:

$ col=$(printf 'a'; printf x)
$ col="${col%x}"
$ col="${col%$'\n'}"
$ printf '%s' "$col"
a$

Wrong:

$ col=$(printf 'a'; printf x)
$ col="${col%$'\n'x}"
$ printf '%s' "$col"
ax$

To learn more about the issue take a look at "Command Substitution" in:

1. The POSIX standard's Shell Execution Environment section where it says:

The shell shall expand the command substitution by executing command in a subshell environment (see Shell Execution Environment) and replacing the command substitution (the text of command plus the enclosing "$()" or backquotes) with the standard output of the command, removing sequences of one or more characters at the end of the substitution.

2. https://mywiki.wooledge.org/CommandSubstitution where it discusses the issue further and provides the workaround I used above.