0

I have file "report.csv" . This report is generated based on few parameters
which can range from 1-10. So the report.csv has all the parameters printed in
it first and then , the main header and the data. Now I need to process
this file with the header and data and exclude all the parameters. The file looks like this:

para_1: abc
para_2: def
para_3: ghi
.
.
.
para_n: xyz

Customer_Name, Customer_Number, .....
Jon Doe,19923
Adam Plak,19985
.
.
.

I used below command to get the position of the Column header

This gives me the position

grep -n "Customer_Name" report.csv | cut -d : -f1

(let's say) 13. Then I use this command to copy to another file and use the new file

sed '1,13d' report.csv >> report_test.csv

however i need to make it dynamic , like 

$var1 = grep -n "Customer_Name" report.csv | cut -d : -f1
$var2 = d
sed '1,$var1$var2' report.csv >> report_test.csv

can you experts please help achieve this in Shell Script?

Philippos
  • 13,453
Praveen
  • 3
  • Actually, why can't you do everything in sed like sed '1,/Customer_Name/' report.csv >> report_test.csv? – Philippos Jun 24 '19 at 08:49

1 Answers1

0
var1=`grep -n "Customer_Name" report.csv | cut -d : -f1`
var2='d'
sed "1,${var1}${var2}"  report.csv  >> report_test.csv

OR 

var1=$(grep -n "Customer_Name" report.csv | cut -d : -f1)
var2='d'
sed "1,${var1}${var2}"  report.csv  >> report_test.csv

Explanation

What we needed here is command substitution

The command substitution can be done in two ways one is using

  • $(…)
  • `…`

The purpose of command substitution is to evaluate the command which is placed inside the backtick and provide its result as an argument to the actual command.

Difference between the two command substitution methods

The old-style backquotes ` ` treat backslashes and nesting a bit different. The new-style $() interprets everything in between ( ) as a command. The new-style $() applies to all POSIX-conformant shells.

These are good reads to understand command substitution better:

Arushix
  • 1,290
  • when i Do this var1=$(grep -n "Customer Invoice Delivery" b | cut -d : -f1)

    i get below error cm01-dev:@/tmp>sh test_sh.sh test_sh.sh: syntax error at line 1: `var1=$' unexpected

    content of test_sh.sh is

    var1=$(grep -n "Customer_Name" b | cut -d : -f1) echo $var1 var2='d' echo $var2 sed "1,${var1}${var2}" report.csv >> report_test.csv

     – Praveen Jun 24 '19 at 06:05
  • Can you try again, in my case file name was b , I have edited back my answer with your file name report.csv ? So it should be var1=$(grep -n "Customer_Name" report.csv | cut -d : -f1) – Arushix Jun 24 '19 at 06:11
  • it still gives the same error

    test_sh.sh: syntax error at line 2: `var1=$' unexpected

     – Praveen Jun 24 '19 at 06:31
  • problem is assignment of the value of

    grep -n "Customer_Name" report.csv | cut -d : -f1

    in variable var1

     – Praveen Jun 24 '19 at 06:38
  • you can try var1=`grep -n "Customer_Name" report.csv | cut -d : -f1` Use ``` sign to enclose the statement – Arushix Jun 24 '19 at 06:41
  • thanks Arushix , this time it worked. but if possible can you please explain why it worked this time ? – Praveen Jun 24 '19 at 06:51
  • I have added an explanation in the answer – Arushix Jun 24 '19 at 07:06