Sorting an XML file in UNIX with a Bash script?

Question

I am trying to sort my XML file that looks like this by alphabetical order. This is a part of a larger bash script so it needs to work within that script:

<Module>
    <Settings>
        <Dimensions>
            <Volume>13000</Volume>
            <Width>5000</Width>
            <Length>2000</Length>
        </Dimensions>
        <Stats>
            <Mean>1.0</Mean>
            <Max>3000</Max>
            <Median>250</Median>
        </Stats>
    </Settings>
    <Debug>
        <Errors>
            <Strike>0</Strike>
            <Wag>1</Wag>
            <MagicMan>0</MagicMan>
        </Errors>
    </Debug>
</Module>

I want the end result to look like this, I only want the innermost tags to be sorted:

<Module>
    <Settings>
        <Dimensions>
            <Length>2000</Length>
            <Volume>13000</Volume>
            <Width>5000</Width>
        </Dimensions>
        <Stats>
            <Max>3000</Max>
            <Mean>1.0</Mean>
            <Median>250</Median>
        </Stats>
    </Settings>
    <Debug>
        <Errors>
            <MagicMan>0</MagicMan>
            <Strike>0</Strike>
            <Wag>1</Wag>
        </Errors>
    </Debug>
</Module>

I am trying to use sort like this where -t sorts by the > delimeter and then the 4 sorts by the 4th column which would be in the inner but it is not working.

sort -t'>' -k4 file > final.xml

I get funky output that sorts the other columns in with the sorted inner tags.

Any help would be appreciated

Answer 1

[with a generous assist from Kusalananda]

You can do it using the xq wrapper from yq (a jq wrapper for YAML/XML) to leverage jq's sorting capabilities:

$ xq -x 'getpath([paths(scalars)[0:-1]] | unique | .[])
    |= (to_entries|sort_by(.key)|from_entries)' file.xml
<Module>
  <Settings>
    <Dimensions>
      <Length>2000</Length>
      <Volume>13000</Volume>
      <Width>5000</Width>
    </Dimensions>
    <Stats>
      <Max>3000</Max>
      <Mean>1.0</Mean>
      <Median>250</Median>
    </Stats>
  </Settings>
  <Debug>
    <Errors>
      <MagicMan>0</MagicMan>
      <Strike>0</Strike>
      <Wag>1</Wag>
    </Errors>
  </Debug>
</Module>

Explanation:

• paths(scalars) generates a list of all paths, from root to leaf, then array slice [0,-1] removes the leaf node resulting in a list of paths to the deepest non-leaf nodes:

  ["Module","Settings","Dimensions"]
  ["Module","Settings","Dimensions"]
  ["Module","Settings","Dimensions"]
  ["Module","Settings","Stats"]
  ["Module","Settings","Stats"]
  ["Module","Settings","Stats"]
  ["Module","Debug","Errors"]
  ["Module","Debug","Errors"]
  ["Module","Debug","Errors"]
  

• [paths(scalars)[0:-1]] | unique | .[] puts the list into an array so that it may be de-duplicated by unique. The iterator .[] turns it back to a list:

  ["Module","Debug","Errors"]
  ["Module","Settings","Dimensions"]
  ["Module","Settings","Stats"]
  

• getpath() turns the de-duplicated list into bottom-level objects whose contents may be sorted and updated with the |= update-assign operator

The -x option tells xq to convert the result back to XML rather than leaving it as JSON.

Note that while sort works here in place of sort_by(.key) the former implicitly sorts by values as well as keys if the keys are non-unique.

Answer 2

Using any awk, sort, and cut in any shell on every Unix box and assuming your input is always formatted like the sample you provided in your question where the lines to be sorted always have start/end tags and the other lines don't and <s don't appear anywhere else in the input:

$ cat tst.sh
#!/usr/bin/env bash
awk '
BEGIN { FS="<"; OFS="\t" }
{
    idx = ( (NF == 3) && (pNF == 3) ? idx : NR )
    print idx, $0
    pNF = NF
}
' "${@:--}" |
sort -k1,1n -k2,2 |
cut -f2-

$ ./tst.sh file
<Module>
    <Settings>
        <Dimensions>
            <Length>2000</Length>
            <Volume>13000</Volume>
            <Width>5000</Width>
        </Dimensions>
        <Stats>
            <Max>3000</Max>
            <Mean>1.0</Mean>
            <Median>250</Median>
        </Stats>
    </Settings>
    <Debug>
        <Errors>
            <MagicMan>0</MagicMan>
            <Strike>0</Strike>
            <Wag>1</Wag>
        </Errors>
    </Debug>
</Module>

The above uses awk to decorate the input to sort so that we can just run sort once on the whole file and then use cut to remove the number that awk added. Here are the intermediate steps so you can see what's happening:

awk '
BEGIN { FS="<"; OFS="\t" }
{
    idx = ( (NF == 3) && (pNF == 3) ? idx : NR )
    print idx, $0
    pNF = NF
}
' file
1       <Module>
2           <Settings>
3               <Dimensions>
4                   <Volume>13000</Volume>
4                   <Width>5000</Width>
4                   <Length>2000</Length>
7               </Dimensions>
8               <Stats>
9                   <Mean>1.0</Mean>
9                   <Max>3000</Max>
9                   <Median>250</Median>
12              </Stats>
13          </Settings>
14          <Debug>
15              <Errors>
16                  <Strike>0</Strike>
16                  <Wag>1</Wag>
16                  <MagicMan>0</MagicMan>
19              </Errors>
20          </Debug>
21      </Module>

awk '
BEGIN { FS="<"; OFS="\t" }
{
    idx = ( (NF == 3) && (pNF == 3) ? idx : NR )
    print idx, $0
    pNF = NF
}
' file | sort -k1,1n -k2,2
1       <Module>
2           <Settings>
3               <Dimensions>
4                   <Length>2000</Length>
4                   <Volume>13000</Volume>
4                   <Width>5000</Width>
7               </Dimensions>
8               <Stats>
9                   <Max>3000</Max>
9                   <Mean>1.0</Mean>
9                   <Median>250</Median>
12              </Stats>
13          </Settings>
14          <Debug>
15              <Errors>
16                  <MagicMan>0</MagicMan>
16                  <Strike>0</Strike>
16                  <Wag>1</Wag>
19              </Errors>
20          </Debug>
21      </Module>

Alternatively, using GNU awk for sorted_in:

$ cat tst.awk
BEGIN { FS="<" }
NF == 3 {
    rows[$0]
    f = 1
    next
}
f && (NF < 3) {
    PROCINFO["sorted_in"] = "@ind_str_asc"
    for (row in rows) {
        print row
    }
    delete rows
    f = 0
}
{ print }

If you don't have GNU awk you can use any awk and any sort for that same approach:

$ cat tst.awk
BEGIN { FS="<" }
NF == 3 {
    rows[$0]
    f = 1
    next
}
f && (NF < 3) {
    cmd = "sort"
    for (row in rows) {
        print row | cmd
    }
    close(cmd)
    delete rows
    f = 0
}
{ print }

but it'll be much slower then the first 2 solutions above as it's spawning a subshell to call sort for every block of nested lines.

Answer 3

Answered as asked: pure(ish) bash solution (still calls sort however). Produces specified output from example input. Fragile, of course, as any solution that treats XML as line-oriented must be.

#!/bin/bash
function FunkySort(){
    local inputfile="$1"
    local -a linestosort=()
    local line ltchars
    while IFS= read -r line; do
        # strip all but less-than characters
        ltchars="${line//[^<]}"
        # if we guess it is "innermost" tag
        if [ ${#ltchars} -gt 1 ]; then
            # append to array
            linestosort+=("${line}")
        else
            # if non-innermost but have accumulated some of them
            if [ ${#linestosort} -gt 0 ]; then
                # then emit accumulated lines in sorted order
                printf "%s\n" "${linestosort[@]}" | sort
                # and reset array
                linestosort=()
            fi
            printf "%s\n" "$line"
        fi
    done < "$inputfile"
}
FunkySort "test.xml" >"test.out"