Mapping between two files

Question

I am new to scripting and have an requirement like below.

There is a file called jobname.txt with below contents

job1
job2

There is a file called value.txt with below contents

10
20

I need to print output like

update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

How can I do this using shell script or awk?

Answer 1

With paste+awk:

$ paste -d'\n' jobname.txt value.txt \
  |awk '{ print (NR%2?"update_job:":"max_run_alarm:", $0) }'

Basically with the paste command as its name is saying, we paste both files line by line from each but adds a single \newline character between each lines, then we re-process the previous output with awk and we prepend desired "text" for the lines which their line number is module of 2 (lines with even line numbers; for example line numbers 2, 4, 6, ...; the NR contains the line number of a line that awk is read for the processing) we prepend text "update_job:" or for odd line numbers we prepend text "max_run_alarm:", then print the line itself $0.

With just awk:

awk 'NR==FNR{ 
    getline val<ARGV[2]; print "update_job:", $0 ORS "max_run_alarm:", val; next
}
{ exit }' jobname.txt value.txt

Here we process differently, above we said NR is the line number, here you see FNR; NR and FNR has always the same value (i.e, they both contains the line number) but NR keep incrementing for all the lines of all input files, while FNR reset back to 1 as soon as the next input file was read (if any), so by saying NR==FNR we are ensuring that the following block of the code only runs for the first input file.

then we call "getline into a Variable from a File" to read a single line from the second argument we passed as the input (here ARGV[2] return value.txt) and save it into val variable; then the rest is just printing things. ORS prints a line break between each.

or maybe you prefer this one (but I don't like hard-coding fileName):

awk '{ 
    getline val<"value.txt"; print "update_job:", $0 ORS "max_run_alarm:", val
}' jobname.txt

Answer 2

$ paste jobname.txt value.txt |
    awk '{printf "update_job: %s\nmax_run_alarm: %s\n\n", $1, $2}'
update_job: job1
max_run_alarm: 10

update_job: job2
max_run_alarm: 20

or all in awk:

$ awk '
    NR==FNR { jobs[NR]=$1; next }
    { printf "update_job: %s\nmax_run_alarm: %s\n\n", jobs[FNR], $1 }
' jobname.txt value.txt
update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

Both of the above use mandatory POSIX tools and so will work in any shell on every Unix box.

Answer 3

With awk:

awk '{printf "update_job: %s\n", $1 ; getline < "value.txt"; printf "max_run_alarm: %s\n\n", $1 }' jobname.txt

First printf command prints line from first file jobname.txt. Then getline command takes line from file value.txt. Then printf command prints that line.

see http://awk.freeshell.org/AllAboutGetline for when and how to use getline.

Answer 4

In bash, the easiest way to do this is to read both files into two different arrays, then iterate from 1 to to the length of one of the arrays.

bash has a built-in command called mapfile to quickly read files into arrays, so I'll make use of that.

For example, using arrays j and v for jobs and values:

$ cat mapping.sh 
#!/bin/bash
mapfile -t -O 1 j < jobname.txt
mapfile -t -O 1 v < values.txt
for i in $(seq 1 "${#j[@]}") ; do
  printf "update_job: %s\n" "${j[$i]}"
  printf "max_run_alarm: %s\n\n" "${v[$i]}"
done

Sample output:

$ chmod +x ./mapping.sh
$ ./mapping.sh 
update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

In awk:

$ cat mapping.awk
#!/usr/bin/awk -f
FNR==NR { # reading first file
  j[i++] = $0
  next;
}
{ # reading second file
  v[n++] = $0
}
END {
  for (i in j) {
      printf "update_job: %s\n", j[i]
      printf "max_run_alarm: %s\n\n", v[i]
  }
}

Output:

$ chmod +x ./mapping.awk
$ ./mapping.awk jobname.txt values.txt 
update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

Answer 5

 co=`awk 'END{print NR}'  jobname.txt`

for ((i=1;i<=$co;i++)); do awk -v i="$i" 'NR==i{print "update_job: "$1}' jobname.txt; awk -v i="$i" 'NR==i{print "max_run_alarm: "$1}' value.txt;echo -e "\n"; done

output

update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

Answer 6

In bash... utilizing an extra file descriptor, both files can be simultaneously iterated through in a while loop like so:

while read -r j
do
    read -r v <&4
    printf "update_job: %s\nmax_run_alarm: %s\n\n" "$j" "$v"
done <"jobname.txt" 4<"value.txt"

Giving this end result:

update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

Answer 7

Using GNU sed we can do it by running two seds in parallel and pasting the output of one into the pattern space of the other via the R command

$ < values.txt \
  sed -e 's/^/max_run_alarm: /' \
| sed -e '
    s/^/\nupdate_job: /
    1s/.//; R/dev/stdin
' keys.txt

---

One more way is using python3 and zip the iterators of the two file handles:

$ python3 -c 'import sys
keys,values = sys.argv[1:]
p,q = "update_job:","max_run_alarm:"
with open(keys) as k, open(values) as v:
  for a,b in zip(k,v):
    print(p,a+q,b,sep=" ")
' keys.txt values.txt

---

Using perl we can place one file in stdin' other on the command line:

$ perl -ne '
  print for(
    "update_job: $_",
    "max_run_alarm: ".<STDIN>."\n");
' keys.txt < values.txt

---

Output:

update_job: job1
max_run_alarm: 10
update_job: job2
max_run_alarm: 20

Answer 8

If ed is available/acceptable.

The script script.ed

$t.
s/.\{1,\}//
1r value.txt
$m$-3
$-1m$
g/./s/^\(job[0-9]\{1,\}\)$/update_job: \1/\
s/^\([0-9]\{1,\}\)$/max_run_alarm: \1/
,p
Q

---

Then run

ed -s jobname.txt < script.ed

---

• Change the Q to w if you're satisfied with the output and in-place editing will occur.

• Remove the ,p to silence the output.