How to add header to multiple files using awk

Question

I would like to add a header line containing whitespaces to multiple files.

Here is what I have so far:

#!/bin/bash
# script name is "add_header.sh"
# ARG1 = HEADER STRING
# ARG2,3,... = ARRAY OF FILES TO ADD HEADER TO, RELATIVE DIRECTORY
HEADER=$1 
shift
for FILE in $@; do
    awk -v HEADER=$HEADER FILE=$FILE 'BEGIN{print HEADER} {print}' FILE > FILE.new
done

Unfortunately, when I run it on my use case, it fails because of the white spaces:

touch file1 file2 file3
./add_header.sh "some header with spaces" file1 file2 file3

which gives following error:

awk: fatal: cannot open file `with' for reading (No such file or directory)
awk: fatal: cannot open file `with' for reading (No such file or directory)
awk: fatal: cannot open file `with' for reading (No such file or directory)

Is there a way to escape white spaces inside a bash variable? I have tried using \ before each space, but the error now changes to:

./add_header.sh "some\ header\ with\ spaces" file1 file2 file3
awk: fatal: cannot open file `with\' for reading (No such file or directory)
awk: fatal: cannot open file `with\' for reading (No such file or directory)
awk: fatal: cannot open file `with\' for reading (No such file or directory)

which means the whitespaces are not being escaped.

Answer 1

#!/bin/sh

header=$1; shift

for pathname do
    { printf '%s\n' "$header"; cat -- "$pathname"; } >"$pathname.new"
done

There is no real need for awk here as we want to concatenate the header and the old file contents. We do this by simply outputting the header string with printf and then using cat to output the file's contents. We redirect the output of both printf and cat to a new file.

Would you really want to do it with awk, then either loop over the files as in the code above, or let awk process each file without an explicit shell loop.

First variation with an explicit shell loop:

#!/bin/sh
header=$1; shift
for pathname do
    header=$header awk 'BEGIN { print ENVIRON["header"] }; 1' "$pathname" >"$pathname.new"
done

The above solution would be the slowest variant of all variants in this answer, as it invokes awk once per file.

Second variation without a shell loop (requires an awk that understands BEGINFILE, like GNU awk does):

#!/bin/sh
header=$1; shift
header=$header awk '
    BEGINFILE { fname = FILENAME ".new"; print ENVIRON["header"] >(fname) }
    { print >(fname) }' "$@"

Third variation (portable variant of the last piece of code):

#!/bin/sh
header=$1; shift
header=$header awk '
    FNR == 1 { fname = FILENAME ".new"; print ENVIRON["header"] >(fname) }
    { print >(fname) }' "$@"

Answer 2

The right way to do this, assuming you have no empty input files, would be:

#!/usr/bin/env bash
header=$1 
shift
awk -v header="$header" '
    FNR==1 { close(out); out=FILENAME ".new"; $0=header ORS $0 }
    { print > out }
' "$@"

The above will work using any awk in any shell on every Unix box. If you can have empty input files it'd need a tweak.

Answer 3

Here is the modified version which works for me:

#!/bin/bash
header=$1
shift
for file in $@; do 
    awk -v HEADER="$header" 'BEGIN{print HEADER} {print}' "$file" > "$file".new
done

I tried to use {print > [FILE].new} inside the awk expression but it did not work. It just printed to stdout. Maybe because new files cannot be created within awk.