6

I have a text (configuration) file, but the program that reads the file unfortunately doesn't allow using any kind of variables. So I'd like to use a preprocessor that replaces a set of placeholders in the config file before passing it to the program.

I can define the format of the variables any way I want (e.g. §SOME_DIR). If I had just a few variables, I would probably use sed:

sed  -e "s*§SOME_DIR*$SOME_DIR*g"  my.conf | target_prog

But the list of variables is pretty long, and it should be easy to configure - so I'd prefer to put the variables in a properties file like

SOME_DIR=...
OTHER_DIR=...
...

and then call

some_replace_tool  my.properties  my.conf  | target_prog

I'm looking for "some_replace_tool". Any ideas?

Chris Lercher
  • 181

5 Answers5

3

I would probably write some script to expand shell varaibles to generate output config file.

Here are some hints:

  1. here-strings (<<<) gets expanded
  2. you can load variables by ". varaibles.conf"
  3. env -i script.sh will run script in clean environment (no extra varaibles)

So script will have to construct a new temporary script which will source the variables and cat here-string with expanded vars. Then to get final output, you will have to run that temporary generated script via env -i.

Michał Šrajer
  • 2,858
2

@fred has something of the right idea, but a much simpler system might be:

sed -ne '/^#/d;s!^\([^=]*\)=\(.*\)$!s@\1@\2g!p' my.proprties > script.sed
sed -f script.sed my.conf | target_prog

This also only generates for lines that have "PROP=VALUE" and ignores comment lines.

Breaking this out:

/^#/d                   - delete any comment lines (to ignore #PROP=VALUE)
s!^\([^=]*\)=\(.*\)$    - match any line with PROP=VALUE and place PROG into group 1 and VALUE into group 2
!s@\1@\2@g              - replace with s@GROUP1@GROUP2@g
!p                      - print the line, to override the -n

You end up with a sed script looking like:

s@PROP1@VALUE1@g
s@PROP2@VALUE2@g
...
Arcege
  • 22,536
2

There are two preprocessors that can claim to be traditional unix preprocessors: cpp (the C preprocessor) and m4. Neither is well-suited to general preprocessing because they replace words anywhere in the text, and in the case of cpp impose a comment syntax. A reasonable general preprocessor would have a macro indicator (i.e. don't expand every occurrence of name, expand only @name (for example)).

There isn't any preprocessor that I'd consider a de facto standard or even merely common. One that's adapted to a variety of input formats and available on many Linux distributions is GPP.

If you want to stick to standard tools, you can use the shell as a preprocessor in some circumstances. Build a string, then run

eval 'preprocessed=$(cat <<EOF)
$template
EOF'

Shell parameters and commands ($var, $(command), `command`) will be expanded. Note that you can't easily prevent the template from invoking external commands or assigning to variables used by the script, so don't use this unless you trust the place where the template comes from.

Gilles 'SO- stop being evil'
  • 829,060
1

Probably the most widespread and portable way for simple macro expansion is the C preprocessor:

cat my.properties my.conf | cpp -P -DSOME_DIR=/a/b -DOTHER_DIR=/c | target_prog

Note that you must use cat here, as cpp only accepts one input file as parameter.

But as you wrote that you have many variables, you may prefer to put them into a file:

#define SOME_DIR /a/b
#define OTHER_DIR /c

Then pass it to cpp as:

cat my.properties my.conf | cpp -P -imacros definition.cpp | target_prog
manatwork
  • 31,277
  • 1
    The C preprocessor is very poorly adapted to general preprocessing. It imposes the C comment syntax, and rewrites all words that are declared as macros. For general preprocessing, having a macro expansion character is a good idea. – Gilles 'SO- stop being evil' Sep 02 '11 at 22:29
0

Update: Extending Arcege's sed-to-sed idea, you can have a more managable layout (tabular, or ragged) for your find/repl templates, which can be inline... and there is no need for an intermediate file: (add a g before the p'<<'EOF' to make multiple changes per line... Note that foo, barbar, etc will be treated by sed as regular expressions. 

I=$'\01'; sed -nre "s/^([^ ]+) +/s$I\1$I/;s/$/$I/p"<<'EOF' |sed -f - file
foo           By the middle of June,
barbar        mosquitos were rampant,
foofoofoo     the grass was tawny, a brown dust
barbarbarbar  haze hung over the valley
EOF

(original post)
You can use sed as an executable script, with a wrapper...

You can define your find and replace text in a seperate text file,
but for this sample it is defined in-line...

This script converts your find/repl pairs into sed expressions and dynamically creates the executable script 

I=$'\01'; set -f
printf "#!/bin/sed -f\n" > ~/bin/script.sed
while IFS= read -r line ;do
    ix=$(expr index "$line" " ")
    printf "s%s%s%s%s%s\n" $I ""${line:0:$((ix-1))}"" $I "${line:$ix}" $I 
done <<'EOF' >> ~/bin/script.sed
$text1 By the middle of June,
$text2 mosquitos were rampant,
$text3 the grass was tawny, a brown dust
$text4 haze hung over the valley
EOF
chmod +x ~/bin/script.sed

This is the test file to be modified.
'$' is okay, as it isn't at the end of a line (for sed), and it won't at any time be exposed to the shell but any token which won't clash with sed is fine. You can even use # 

echo '$text1 $text2 $text3 $text4' > file

Assuming that ~/bin is a $PATH directory, the command to run is:

script.sed  file
Peter.O
  • 32,916