2

I'd often like to quickly take some data, and apply it to a template in bash.

e.g. imagine doing the following

  $ seq 1 2 | eztemplate 'this is a {1} test'
  this is a 1 test
  this is a 2 test

$ eztemplate 'this is a {1} test' hi 'there now' this is a hi test this is a there now test

$ eztemplate /tmp/template /tmp/alphabet | head this is a a test of this system a this is a b test of this system b ...

I've wrote a very simple bash script that does this, but was considering allowing multiple parameters, like CSV style data, per line of data.

Does something better than my little script already exist, given the following?

  • I want it to require only basic unix posix tools, and commonly installed things like perl or awk, but requires no modules be installed additionally by perl, for example
  • It can accept multiple columns of data, per line of data in the data file.
  • Basically is one bash script that doesn't require installing anything else :D
  • A side intent is to allow others that aren't great at bash scripting to have a simple tool to process templates for repeating data

The data and template will vary greatly, but the first example I wanted to do it with was apply 4 ids to a JSON payload

template

{
  "tenantId": "{1}",
  "key": "some.key",
  "value": "some.value"
}

data

my/super/01cbf4e9779649038f0bd753747c8b26
my/test/01cbf4e9779649038f0bd753747c8b26
ez/test/01cbf4e9779649038f0bd753747c8b26
rad/data/df3a47fed39d453f836f9b2196332029

eztemplate

#!/usr/bin/env bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"; PATH="$DIR:$PATH"

function show_help() { ME=$(basename "$0") IT=$(cat <<EOF

replaces vars in a template, one for each line of stdin

e.g.

$ seq 1 2 | $ME 'this is a {1} test' this is a 1 test this is a 2 test

$ $ME 'this is a {1} test' hi 'there now' this is a hi test this is a there now test

$ $ME /tmp/template /tmp/alphabet this is a a test of this system a this is a b test of this system b ...

EOF ) echo "$IT" echo exit }

if [ -z "$1" ] then show_help fi if [ "$1" = "help" ] || [ "$1" = '?' ] || [ "$1" = "--help" ] || [ "$1" = "h" ]; then show_help fi

function process_template(){ DATA=$1 VAR=$2

if [ -f "$DATA" ]; then DATA=$(cat "$DATA") fi

echo "$DATA" | sed "s#{1}#$VAR#g" }

TEMPLATE=$1 if [ -t 0 ] then if [ -f "$2" ]; then # allow first 2 parameters to be files, TEMPLATE and then DATA DATA_FILE=$2 cat "$DATA_FILE" | while read line do process_template "$TEMPLATE" "$line" done else shift; for line in "$@" do process_template "$TEMPLATE" "$line" done fi else

loop over lines from stdin

while IFS= read -r line; do process_template "$TEMPLATE" "$line" done fi

Brad Parks
  • 1,669
  • Since you're using bash which is not POSIX I assume you are OK with bash too, right? Or does it need to be a POSIX shell? – terdon Mar 03 '23 at 17:59
  • Please read why-is-using-a-shell-loop-to-process-text-considered-bad-practice, https://porkmail.org/era/unix/award, correct-bash-and-shell-script-variable-capitalization and I expect running your script through http://shellcheck.net would flag some issues. – Ed Morton Mar 03 '23 at 19:18
  • My impression is that the sample input you provided at the top of your question isn't the actual input format you want to be able to handle (e.g. it's not CSV) so please [edit] your question to include concise, testable sample input and expected output such as you want your script to be able to handle. – Ed Morton Mar 03 '23 at 19:26
  • hey... thanks for the feedback - I'm mostly wondering if there's some tool that does what I'm doing "better", and is preinstalled in linux (or mac), with a non bash script maybe? I don't tend to process tons of data, so I'm not too concerned about using a bash loop, but I get your point :D – Brad Parks Mar 03 '23 at 19:41
  • and the data I want to pass in will be totally arbitrary - most of the time a list of ids, but possibly at time a list of comma separated values, e.g. 54,john,doe. Most of the time they'll be in a file I download from splunk or Google Sheets, and I want to quickly tranform the data into json payloads or something. Internally in my company we have some tools that require individual json payloads, so one example would be applying the values to a json payload format. But honestly it could be anything, and I expect most of the time a single value will do, but sometimes I might use multiple :D – Brad Parks Mar 03 '23 at 19:42
  • 1
    I wrote a simple template engine to use csv data in Powershell for windows. You are welcome to any concepts you can use in creating a tool for Unix and bash. Check it out at github.com/dcressey/Expand-Csv – Walter Mitty May 24 '23 at 13:28
  • Thanks for the ideas @WalterMitty ! My goals are to keep things as "little install required" as possible, as I support devs across the board, some that are new to the game, and some total pros, but everyone likes the "zero install" concept. We do use Docker for lots of things, so I'm considering expanding my horizons to auto load many things transparently through the container. Definitely food for thought! – Brad Parks May 24 '23 at 23:01
  • Yeah, that's why I didn't post it as an answer. – Walter Mitty May 25 '23 at 06:27

1 Answers1

3

For the example you cite, the most natural solution appears to be

$ seq 1 2 | xargs -n1 printf 'this is a %s test\n'

Clearly awk is equal to the task, as well, if desired.

terdon
  • 242,166
J_H
  • 866