0

I need to parse a csv file and store the value in a variable. Below is the sample csv file

Sample CSV file

And below is the ./script.sh

#!/bin/bash

D="";

P="";

./xyz --project "$P" --displayname "$D"

For example if we consider the first line:

When i run the script like

./script.sh /home/project.csv

it should store the values: 10V nmos and pmos for MA into $D and 10v_nmos_and_pmos_for_ma into $P.

How can this be done ?

Another scenario is, from the csv file if i call first three projects then it should run in a loop row by row

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
Siddharth Sahoo
  • 15

2 Answers2

1

bash has no built-in support for parsing CSVs. You could use ksh93 instead which supports parsing CSV (at least some form of CSV, the one where a literal " is entered as "" within double quotes) with read -S:

#! /bin/ksh93 -
while IFS=, read -rSu3 P D ignore; do
  ./xyz --project "$P" --displayname "${D#*.}"
done 3< file.csv

Or use perl/python with a proper CSV parsing library that you could tune for the exact format of your csv. Example with perl:

perl -C -MText::CSV -e '
  $c = Text::CSV->new;
  while (($p, $d) = @{$c->getline(STDIN)}) {
    $d =~ s/.*?\.//;
    system "./xyz", "--project", $p, "--displayname", $d;
  }' < file.csv

If you can guarantee that the content of the CSV fields won't contain double-quote, comas or newline characters, with POSIX shells like bash, you could do:

tr -d \" < file.csv | while IFS=, read -r p d ignore; do
  ./xyz --project "$p" --displayname "${d#*.}"
done
Stéphane Chazelas
  • 544,893
  • I am completly new to scripting, i did something like this

    #!/bin/bash

    export IFS="," cat /tmp/test/test_parse/parse.csv | while read a b; #do echo "$a:$b:$c:$d"; do echo $a echo $b done

    But in $a = it saves with 10V nmos and pmos for MA" but i need only the 10V nmos and pmos for MA in $b = it gives with projects.10v_nmos_and_pmos_for_ma which is correct but i need only 10v_nmos_and_pmos_for_ma

    please guide

     – Siddharth Sahoo Nov 11 '16 at 13:14
-2

This assumes very reliable occurrences of ',' and '.' characters in your CSV:

#!/bin/bash
cat "$1" | while IFS='' read -r line; do
    D="$(echo "$line" | cut -d ',' -f 1 | tr -d '"')"
    P="$(echo "$line" | cut -d '.' -f 2)"
    echo 'D: '"$D"
    echo 'P: '"$P"
    ./xyz --project "$P" --displayname "$D"
done
Jan Kyu Peblik
  • 1,829
  • This is not working properly, when i run the script with inputs nothing is getting saved in $D and $P

    i mean i cant see value getting stored the Variable

     – Siddharth Sahoo Nov 11 '16 at 12:48
  • You'll need to make sure the path to your 'xyz' executable is correct ('./xyz' obviously assumes it's in the same directory as the code (here) being run, and of course that your 'xyz' executable is consuming its arguments as expected. You can also substitute foo.csv with "$1" to utilize it exactly as you mentioned (./script.sh /home/project.csv). – Jan Kyu Peblik Nov 11 '16 at 12:54
  • remove the last ./xyz for now

    If i run the below i dont see any value stroring to the variable

    #!/bin/bash

    cat foo.csv | while IFS='' read -r line; do D="$(echo "$line" | cut -d ',' -f 1 | tr -d '"')" P="$(echo "$line" | cut -d '.' -f 2)"

    echo $D echo $P

     – Siddharth Sahoo Nov 11 '16 at 12:56
  • foo.csv is of course a placeholder. You can substitute foo.csv with "$1" to utilize it exactly as you mentioned (./script.sh /home/project.csv). – Jan Kyu Peblik Nov 11 '16 at 12:59
  • I am completely new to scripting, i have added the script this way

    #!/bin/bash

    cat /tmp/home/project.csv | while IFS='' read -r line; do D="$(echo "$line" | cut -d ',' -f 1 | tr -d '"')" P="$(echo "$line" | cut -d '.' -f 2)"

    ./xyz --project "$P" --displayname "$D"

    done

    echo $D echo $P

    Please let me know how to do ??

     – Siddharth Sahoo Nov 11 '16 at 13:06
  • Hey @Jankyupeblik please advice – Siddharth Sahoo Nov 11 '16 at 13:15
  • I've updated the script in the answer. If you paste it directly into a file named 'script.sh' and run chmod +x script.sh, it should work as you desire (by running ./script.sh /home/project.csv [./ assumes script.sh is in the exact directory you're currently at]. – Jan Kyu Peblik Nov 11 '16 at 13:16
  • Thanks a lot this do work

    But as there are lot of rows, so here comes the second scenario

    i need to run /xyz row by row

    for example first loop it should take the first row values and complete one cycle once it is success then it should store the second row values in the variable and run the second cycle.

    Could you please modify this a bit more, it would be lot helpful

     – Siddharth Sahoo Nov 11 '16 at 13:31
  • @SiddharthSahoo It will already do that. – Jan Kyu Peblik Nov 11 '16 at 13:34
  • 1
    A few bad practices here could justify a downvote: UUOC, use of echo for arbitrary data. But also running 9 commands, 4 of which not builtin and 5 pipes for each line of the file, while read can do the cutting itself, and bash expansions can remove the quotes by themselves. At least, you've quoted your variables. kudos for that. – Stéphane Chazelas Nov 11 '16 at 15:20
  • See Why is using a shell loop to process text considered bad practice? – Wildcard Nov 18 '16 at 02:04
  • The only thing more amusing than the unjustified downvotes is the censoring of my comments. :p @Wildcard No, what's bad practice is learning an entire language when you already have a set of tools that does the job. – Jan Kyu Peblik Nov 19 '16 at 02:23
  • @JanKyuPeblik, you mean a pencil and paper? Yeah, why are you using this computing device you're reading on, anyway? You already have a set of tools that does the job. I'll look forward to your stamped letter continuing this conversation. – Wildcard Nov 19 '16 at 02:56
  • I agree with you there, even if it's an idiotic thing for you to say because of the very point you're trying to make. :) – Jan Kyu Peblik Nov 20 '16 at 21:15