10

I have a csv file that contains 3 fields per line.

firstname,lastname,url

I'm trying to access the url via the following pipeline:

grep theName file.csv | cut -d, -f 3

then I want to add another pipe and use the results of the cut command in a curl command like so:

grep theName file.csv | cut -d, -f 3 | curl > result.txt

problem is, when i do the above, the curl command throws an error, i assume because curl doesn't have an argument?

how can I use the result of cut to curl the resulting url? Thanks in advance. =)

wdowling
  • 111

2 Answers2

12

Leverage command substitution, $():

curl "$(grep ... | cut -d, -f 3)"

Here $() will be substituted by the STDOUT of the command inside $() i.e. grep ... | cut -d, -f 3, as this is done by the shell first so the curl command would be finally:

curl <the_url>
heemayl
  • 56,300
  • Duh. i should have known that and it didn't even occur to me. Well done. this works perfectly. It won't let me mark this correct yet. I'll try to remember and come back to mark your answer as correct. – wdowling Nov 16 '16 at 05:14
  • @wdowling, should be long enough now. – Wildcard Nov 16 '16 at 05:30
  • I know that few users use anything other than bash these days, but this is a bashism. The portable answer is curl \grep ... | cut -d, -f 3`` - although I much prefer the xargs solution, it's more flexible in general. – reinierpost Nov 16 '16 at 08:55
  • @reinierpost No, $() is defined by POSIX. – heemayl Nov 16 '16 at 08:58
  • OK, thanks, it's a POSIXism then. I'm not sure how many non-POSIX-compliant shs are still out there. I just checked and dash and ash both support it. – reinierpost Nov 16 '16 at 08:59
11

Another solution without substitution:

grep theName | cut -d, -f 3 | xargs curl > result.txt
Ipor Sircer
  • 14,546
  • 1
  • 27
  • 39
  • This answer works just as well as the other answer and i like that the flow doesn't change. My only question is, is xargs supported cross platform? I routinely go back and forth between my linux box and a mac, so I always want to make sure i don't get in the habit of doing something that I can't do on one machine. Thanks for your quick response! – wdowling Nov 16 '16 at 05:16
  • utfg: https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/xargs.1.html – Ipor Sircer Nov 16 '16 at 05:20
  • 1
    @wdowling, it's portable but still a bad habit to get into. See the last paragraph of this answer. – Wildcard Nov 16 '16 at 05:34
  • xargs is very widely available. – reinierpost Nov 16 '16 at 08:54
  • xargs is better than command substitution in case your input has funny characters - it has options to deal with them, e.g. I like to provide -d'\n'. – reinierpost Nov 16 '16 at 08:58
  • @reinierpost URLs don't have funny characters to begin with. – Dmitry Grigoryev Nov 16 '16 at 13:23
  • I'm just commenting on why xargs may be a good habit to get into. For this particular case it doesn't seem to make much of a difference (unless the input is extremely long, command substitution might have a limit there). – reinierpost Nov 16 '16 at 16:44