Awk: isolate a block of code, then iterate through multiple blocks (if they exist)

Question

Can awk † find the nth iteration of a "{" and return everything up to the next "}" character?

[EDIT: yes... solution from Ed Morton at bottom]

† I've been assuming awk is the correct tool for this job. Other ideas are welcome.

I need to isolate blocks of text in hundreds of files. Some files have only one block, but others contain dozens.

sample:

$ cat samp2.txt
//////////////////////////////////
// North Carolina office
// satellite branch
//////////////////////////////////
   {
   first   "John"
   last    "Doe"
   address "163 Main Street"
   age     "25"
   gender  "male"
   }

It may be best to > the current block into a temp file so the script can operate on it before addressing the next. They'll end up in separate files anyway.

I suspect awk can be given an index to find the nth match. The bash script can manage the loop and iteration.

I've gotten close

$ awk '/\{/{flag=1;next}/\}/{flag=0}flag' samp2.txt 
   first   "John"
   last    "Doe"
   address "163 Main Street"
   age     "25"
   gender  "male"

However, since the above operates on the entire file it doesn't work for files containing more than one block (e.g. below). Irrespective of how many blocks in any file, I need every block separated to be processed individually.

Some files contain comments, but many do not--with no standard. I discard them, but the inconsistency means comments can't be relied upon for tracking where we are. The only given is the curly braces (and the line separation).

The text is always newline-separated, but not always a blank line between blocks. The data pairs vary, so this can't be a simple grep 5 lines and proceed solution.

$ cat samp3.txt 
//GROUP1
{
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
}
//The fourth group
{
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
}
{
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
}

---

My above awk statement runs through all groups, mashing them all into one large paragraph.

$ awk '/\{/{flag=1;next}/\}/{flag=0}flag' samp3.txt
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"

I need to tell awk to look for the nth "{" and then dump to the nth "}" separately, like this instead:

first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
 (awk exits, bash script does its thing)
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
 (awk exits, bash script does its thing)
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
 (awk exits, bash script does its thing)
[etc]

The intent is similar to a non-greedy regex match of the nth "{ .+ }" .
With that, there may be a perl solution that's smarter?

TIA.

This code got me what I need. Adapted from Ed Morton's answer.

awk -v n=$LoopVariable -v RS='}' 'NR==n{gsub(/.*\{\r?\n|\n$/,""); print}' $SourceFile

EDITS: Input really helped me isolate my question to what I need. Thank you for that.

---

I've found a few SE questions that seem similar, but if these contain my solution I'm not well-versed enough in awk to see the connection.

Answer 1

I don't see the expected output in your question so I'm not sure but you did say Can awk † find the nth iteration of a "{" and return everything up to the next "}" character? so is this what you're trying to do (using any awk and assuming } and { can't appear anywhere else in your input):

$ awk -v n=2 -v RS='}' 'NR==n{gsub(/.*\{\n|\n$/,""); print}' samp3.txt
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"

If you want to call that in a shell loop:

$ for i in {1..3}; do
    awk -v n="$i" -v RS='}' 'NR==n{gsub(/.*\{\n|\n$/,""); print}' samp3.txt
    echo "-----"
done
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
-----
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
-----
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
-----

but there's almost certainly a better way to do whatever it is you want to do than calling awk multiple times in a loop, for example call awk once to print each block with a terminating } and then read that into a shell array for further processing:

$ readarray -d '}' -t arr < <(awk 'BEGIN{RS=ORS="}"} {gsub(/.*\{\n|\n$/,"")} $0~/[^[:space:]]/' samp3.txt)
$ for i in "${arr[@]}"; do printf '%s\n' "$i"; echo "-----"; done
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
-----
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
-----
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
-----

In reality, though, whatever it is you're doing in the shell loop should probably also be done inside the one call to awk.

Answer 2

My code makes assumptions that may not be true which will mean it may fail in many circumstances. There may also be more efficient solutions that could be used.

Assumption 1 Every GROUP block is seperated by a newline

Assumption 2 You want an action taken at each block

Assumtion 3 Every GROUP block increments (If not, you may end up with a lot of empty files.)

for i in {1..5}; do 
  awk -F"\n" -v RS="" -v inc="GROUP$i" '$0~inc{printf( "%s\n", $0); next}' $inputfile | sed  '/\/\|{\|}/d' > output$i.txt ; 
done

Your example has GROUP1&4, I added a GROUP5 and scripted a for loop to increment from a range of 1-5. This range will be used as a key when going through the GROUP blocks. If you have more groups, you can increase the range accordingly.

awk will be used within the loop to extract the blocks. sed will clean up (awk can do this all in one but I am still learning) after which each block is written to its own output file matching the number of the GROUP block.

Input File

//GROUP1
{
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
}
//GROUP4
{
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
}
{
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
}
//GROUP5
{
first       "Maria"
address     "188 John Street"
last    "Phones"
special     "Supervisors supervisor"
age "35"
gender      "Female"
}

Output

cat output1.txt
first       "John"
address     "124 Main Street"
last    "Jones"
special     "supervisor"
age "35"
gender      "male"
cat output4.txt
first       "John"
address     "125 Main Street"
last    "Jacob"
age "30"
gender      "male"
first       "John"
address     "523 Main Street"
last    "Jingle"
age "40"
gender      "male"
cat output5.txt
first       "Maria"
address     "188 John Street"
last    "Phones"
special     "Supervisors supervisor"
age "35"
gender      "Female"

Answer 3

You were almost there....tweaking your code a bit will get you the individual blocks

awk -v n="$loopVar" '/\{/{f=1;++i;next} /\}/{f=0} i==n&&f' file

Caveats:-

• /\{/ will match an opening brace anywhere.
• Somewhat better is: NF==1&&$1=="{"
• Same for the closing brace as well.
• Before awk, run your input file through dos2unix utility to clear off the carriage returns \r