0

I have two commands, build and deploy. Currently I run build manually, parse its output using my own eyes, and use a value I find in the output as an argument to deploy. The process looks something like:

$ build
==> amazon-ebs: amazon-ebs output will be in this color.
... hundreds of lines of output ...
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-19601070

$ deploy ami-19601070
... some more output ...

(build is actually Packer, for the astute)

I would like to tie these two steps together in a script. The rough outline would include the following:

  • Run build
  • Make sure the exit status was 0 and that the output contained the string "AMIs were created", otherwise abort
  • Extract the AMI number (ami-19601070) from the output
  • Run deploy ami-19601070

I'm trying to come up with the best way to connect everything together, ideally by using a shell script, but I'm stuck on how to grep the output for two separate patterns while, ideally, still streaming all stdout/stderr messages to the terminal as the commands run. I'm wondering if I should abandon the idea of doing this in a shell script and instead write a small Python script to do it.

smitelli
  • 288

4 Answers4

5

Sounds like a job for tee:

build | tee /some/file
ami_name=$(do_something_to /some/file)
deploy "$ami_name"
Chris Down
  • 125,559
  • 25
  • 270
  • 266
2
 deployArgs=`build | tee /dev/tty  | extractDeployArgs` &&
    deploy "$deployArgs" #won't run unless extractDeployArgs suceeded

tee /dev/tty will print directly to the terminal and pass the output to the next command in the pipeline at the same time.

(Feel free to replace it with some other file (or /dev/fd/"$someFileDescriptor" if you need want the side output to go to $someFileDescriptor))

In more advanced shells (ksh, bash, zsh, but not in dash) you can set -o pipefail to make sure the pipeline fails if any of its links fails (useful if extractDeployArgs can't tell from its input whether build suceeded or not).

Petr Skocik
  • 28,816
  • 1
    xargs deploy has quite inane rules for input delimitation. I'd suggest avoiding it, or using -0 instead. Also, going out to /dev/tty results in some potentially undesirable properties (such as the output defying further redirection) -- I'd suggest switching around FDs instead if you want to go this route. – Chris Down Nov 24 '15 at 17:46
  • @ChrisDown Thanks for the commentary. I improved/simplified it a little, taking your feedback into account. My main point was that saving to a file isn't necessary. – Petr Skocik Nov 24 '15 at 18:39
  • 1
    @Chris Down - quoting aside, xargs -t kind of does exactly what is asked for here. -p is even closer. – mikeserv Nov 25 '15 at 11:39
0

If your interest is in a particular string of output you could run it like:

msg="$(build | grep -e "AMIs were created\|ami-[0-9]*")"

if [ -n "$(echo $msg | grep -e "AMIs were created")" ];then
    ami="$(echo "$msg" | grep -e "ami-[0-9]*" | cut -d ' ' -f 2)"
    deploy "$ami"
else
    exit 1
fi

The first grep selects only the lines in your output that report AMI's were created or ami-#. The output is checked for "AMIs were created", and if present, parses the ami-# out of your relevant output and uses it to deploy.

Centimane
  • 4,490
0
tf=$(mktemp)
build | tee "$tf"
grep -Fq 'AMIs were created' "$tf" && ami=$(grep -o 'ami-[0-9]\+$' "$tf")

# you didn't say if `deploy` can handle multiple args or not.
# uncomment one of the following:
# deploy $ami
# for a in $ami ; do deploy "$a" ; done
cas
  • 78,579