2

I'm trying to return a boolean if a curl response has a status of 200.

curl https://www.example.com -I | grep ' 200 ' ? echo '1' : echo '0';

This however brings back:

grep: ?: No such file or directory
grep: echo: No such file or directory
grep: 1: No such file or directory
grep: :: No such file or directory
grep: echo: No such file or directory
grep: 0: No such file or directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0

So I'm thinking (and from reading other threads) that grep doesn't support the ternary operator?

Is there a simple one liner to do this without creating a shell script? (also my grep ' 200 ' is loose but I figure I can make that more specific later)

  • 1
    If you just need to return true if found or false if not found, that is exactly what grep returns ... so : curl ...... -I | grep ' 200 ' >/dev/null ; return "$?" ( in bash, $? (return of the last group of commands) gives the return of the last command executed in the pipe, grep here) . bash or grep don't have the ternary (aka "Elvis" ?: ) operator, but awk does, if you reaaaaally want to use it here. – Olivier Dulac Apr 19 '17 at 10:10
  • 2
    Proposal: don't involve grep. I've written an answer discussing what this might look like. curl is already perfectly capable of telling you whether it succeeded or not without stomping all over its output with another program. Besides leaving curl's output clean, it will also be much more stable across curl versions -- you don't have to rely on other version's UI looking the same way you see it in your version. – Daniel Wagner Apr 19 '17 at 11:45

5 Answers5

4

Like a quick study of the grep manual page should reveal, it allows for options, a search pattern (multiple patterns if you specify multiple -e options), and an optional list of file names, falling back to reading standard input like many other Unix filters if no file names are specified. There is nothing about a ternary operator.

If it were supported, it would arguably need to be a feature of the shell, but no shell I know of supports anything like this. It would also clash with the syntax for ? as a wildcard metacharacter which (in isolation) matches any single-character file name.

The usual shorthand idiom in Bourne-compatible shells looks like

curl https://www.example.com/ -I | grep -q ' 200 ' && echo 1 || echo 0

This could accidentally match on a "200" somewhere else in the output. For improved accuracy, try curl https://www.example.com/ -I | head -1 | grep ' 200 ' or perhaps curl https://www.example.com/ -I | awk 'NR==1 { print ($2=="200" ? "1" : "0"); exit }' where the precision of the matching operation is significantly improved.

As an afterthought, if you want to print "0" in the case of invalid URLs and other unexpected errors too, you could set the default output to 0 and change it to 1 when you see 200, then print whatever it ended up as at the end.

curl https://www.example.com/ -I |
awk 'BEGIN { s=0 } NR==1 { s=($2=="200"); exit } END { print s }'
tripleee
  • 7,699
  • Yes, this is well written all makes sense. If the domain didn't resolve though would there also be a way to make that fall into the else? I read the || echo 0 as an else maybe this is more of a null coalesce operation though? Current curl failure brings curl: (6) Could not resolve host: www.example.cm; nodename nor servname provided, or not known (tested with example.cm just as edge case) –  Apr 19 '17 at 05:14
  • Awk is slightly pesky in that if there is no input, there will be no output unless you specifically say so. I'll post an update with slightly different logic. – tripleee Apr 19 '17 at 05:16
  • curl https://www.example.com/ -I | awk 'BEGIN { s=1 } NR==1 { s=1-($2=="200"); exit } END { print s } fails, I get 0 when the response is 200. Rough example HTTP/1.1 200 OK Date: Wed, 19 Apr 2017 05:21:22 GMT Server: Apache/2.2.15 (CentOS). –  Apr 19 '17 at 05:22
  • Sorry, I had 1 and 0 reversed (program exit codes typically use 0 for success). – tripleee Apr 19 '17 at 05:28
  • Okay, that looks to be working. Is there a way to return just the 0 on curl resolution failures? –  Apr 19 '17 at 05:30
  • Um, I don't understand the question, or rather, that's what it already does. It only changes the code to 1 if it sees 200, then prints whatever the code ended up as. – tripleee Apr 19 '17 at 05:32
  • Maybe I'm over testing but if I use a non-existent domain + TLD pair I get curl: (6) Could not resolve host: www.example.cm; nodename nor servname provided, or not known before the 0. The only response I want is 1 for success or 0 for every other occurrence. –  Apr 19 '17 at 05:35
  • curl -I 2>/dev/null to suppress all error output. I believe curl -sI also works (at least for me). – tripleee Apr 19 '17 at 05:36
  • Yes, the sI is exactly what I need with the |....|| pairing. –  Apr 19 '17 at 05:40
4

All the other answers are discussing your proposed solution, bit I would like to suggest that this is an XY problem. What you are probably looking for is a reliable, programmatic way to check whether curl succeeded or not. For that, you should just add -f:

curl -fqI http://www.example.com

This exits with code 0 on success and non-zero on any kind of failure (including server errors that successfully complete an HTTP session but report an HTTP error). In a shell script, for example, you can access this exit code with $? right after the command if you want to store it in a variable, or use the shell's if/&&/|| to conditionally execute other code:

if curl -fqI http://www.example.com
then echo 1
else echo 0
fi

If the goal really is to just print success/not success, most shells have an option to automatically display nonzero exit codes; for example, in zsh with setopt printexitvalue in your ~/.zshrc, you will see something like this (curl output suppressed to highlight the interesting bit):

% curl -fqI http://www.example.com >/dev/null 2>&1 
% curl -fqI http://www.aoeuaoeuaoeuaoeu.com >/dev/null 2>&1
zsh: exit 6     curl -fqI http://www.aoeuaoeuaoeuaoeu.com > /dev/null 2>&1
3
! curl https://www.example.com -I | grep -q ' 200 '; echo "$?"
! curl https://www.example.com -I | grep -q ' 200 '; yes "$?" | sed q
1

You can use awk to process the output as:

curl https://www.example.com -I | awk '/ 200 /{found=1} END{if (found) print 1; else print 0}'

This sets the variable found to 1 (although anything would work in this case) when the pattern / 200 / is seen. Then when the processing is done, the if tests to see if found has been set and reacts accordingly.

  • This also works, could you please add a explanation I haven't seen this awk usage before. Specifically /200/{found=1}. I'd guess if 200 is present found gets set to 1. –  Apr 19 '17 at 05:07
  • For HTTP(S) headers, you actually want to examine the second field of the first line of the output specifically. The END pattern means you look for a match anywhere in the headers. My answer contains a slightly more focused version which exits after reading the first line. – tripleee Apr 19 '17 at 05:13
  • note that this uses curl -I, which uses the HTTP head command. This may not necessarily be the request the writer is expecting and in fact not all webapps implement HEAD .

    It may be better to use curl -D - -o /dev/null as your output source, then use grep as appropriate to search for the HTTP return code.

    Or curl -D - -s -o /dev/null https://www.example.com |head -n1 |grep -w '200'

    – madumlao Apr 19 '17 at 05:16
  • And/or maybe just use -w '%{http_code}'. – tripleee Apr 19 '17 at 05:23
  • I run the server I'm curling so if curl -I isn't returning 200 it is on me to fix. –  Apr 19 '17 at 05:27
  • 1
    The answers here should hopefully be general enough to help others in the future too, so the points about servers not supporting HEAD is useful and relevant even if it's out of scope for your immediate problem. – tripleee Apr 19 '17 at 05:42
1

Here's how I might do it:

if [ "x$(curl -Isqw '%{http_code}' -o /dev/null http://example.com/)" = 'x200' ]; then echo "yep"; else echo "nope"; fi

To break it down:

  1. Set the "silent" and "quiet" flags on curl and redirect the actual response to the bit bucket with -o /dev/null. This stops all curl output whatsoever, which lets us focus on our quarry, the status code. (The -I is a courtesy to the server: we are discarding the response anyway, so we request only the headers of the response)
  2. Ask curl to output the response code with the -w '%{http_code}'flag.
  3. Catch all the output in a single string with the "$()"syntax, and prepend the letter x just to make sure there's always something in the string, even if the command crashes and burns (old habits die hard). The proper thing to do would be to check the return code of the curl command at this point, but since any failures now get neatly mapped to a "not 200" result anyway, we won't bother.
  4. Use the [ command (usually found at /bin/[, which is often a symlink to /bin/test) to check if the resulting string is identical to "x200"
  5. Now we already have the boolean value, but since the question asked for a "string boolean", finally use if to choose what to output. The actual syntax for if may vary from one shell to another; this one will work with sh, bash and their relatives.
Bass
  • 300