13

I have the following json

[root@mdfdevha1 ~]#  echo "$Group_ID"
[ {
  "id" : "e27206c0-aeb6-43db-acda-c4ba43233071",
  "name" : "A1",
  "path" : "/A1",
  "subGroups" : [ ]
}, {
  "id" : "89f3bd6a-33a9-4e02-9fe3-eae660c5a6cf",
  "name" : "Admin_UserGroup",
  "path" : "/Admin_UserGroup",
  "subGroups" : [ ]
}, {
  "id" : "cdc2bce5-c3bb-4b88-bdaf-d87b8bb6c644",
  "name" : "Group104",
  "path" : "/Group104",
  "subGroups" : [ ]
}, {
  "id" : "a0d749f2-ab6c-4c27-ad55-3357eaab9527",
  "name" : "Group105",
  "path" : "/Group105",
  "subGroups" : [ ]
}, {
  "id" : "fbf99c34-d50d-408b-8d19-9713f9af3e3a",
  "name" : "Group106",
  "path" : "/Group106",
  "subGroups" : [ ]
}, {
  "id" : "ebd8336f-4017-4fb1-8035-153ae1d9ba37",
  "name" : "Group201",
  "path" : "/Group201",
  "subGroups" : [ ]
}, {
  "id" : "38f4aef7-caf0-4430-9e61-1ae7026e872f",
  "name" : "Group202",
  "path" : "/Group202",
  "subGroups" : [ ]
}, {
  "id" : "436a0f4a-8b1b-4d7d-a014-fcec3513644e",
  "name" : "Group203",
  "path" : "/Group203",
  "subGroups" : [ ]
}, {
  "id" : "41962c5f-e7e9-4748-b81f-e3f1880b78de",
  "name" : "Sure_Groups",
  "path" : "/Sure_Groups",
  "subGroups" : [ {
    "id" : "593dfe69-1ed8-4649-bde4-a277166333f8",
    "name" : "Test1",
    "path" : "/Sure_Groups/Test1",
    "subGroups" : [ ]
  } ]
}, {
  "id" : "6856b69b-9113-46e1-90c6-f34548625278",
  "name" : "UG_1",
  "path" : "/UG_1",
  "subGroups" : [ ]
}, {
  "id" : "6496a0fe-b41f-4f0f-9eb9-5ef749c9130a",
  "name" : "UG_12",
  "path" : "/UG_12",
  "subGroups" : [ ]
}, {
  "id" : "71a5f5ae-bf91-4cdf-ab3c-c09ca15080d6",
  "name" : "UG_1456",
  "path" : "/UG_1456",
  "subGroups" : [ ]
}, {
  "id" : "385ea518-1d40-45f7-afcd-c0488ff02e97",
  "name" : "UG_26",
  "path" : "/UG_26",
  "subGroups" : [ {
    "id" : "a4064e3a-e2e3-47bb-99b8-9f7fadb0bc20",
    "name" : "Test1",
    "path" : "/UG_26/Test1",
    "subGroups" : [ ]
  } ]
}, {
  "id" : "9c5efedc-b901-4dcf-bbc8-8ddeaa5d84f7",
  "name" : "UG_266",
  "path" : "/UG_266",
  "subGroups" : [ ]
}, {
  "id" : "c5eb3064-752c-4f7c-b4f1-ac59f50397dd",
  "name" : "Usergroup_01",
  "path" : "/Usergroup_01",
  "subGroups" : [ ]
}, {
  "id" : "d39dc10c-558b-433e-82b4-e01a8f1d8998",
  "name" : "Usergroup_02",
  "path" : "/Usergroup_02",
  "subGroups" : [ ]
} ]

How to get particular data with awk or sed .I need to get data where name="Admin_UserGroup"?

EDIT #1

Thanks to Hossein Vatani for his answer and here are the final commands:

$ /opt/keycloak/bin/kcadm.sh get groups -r T0_Realm > Group.json
$ GROUP_ID_TEMP=$(grep -B1 -A0 '"name" : "Admin_UserGroup"' Group.json)
$ GROUP_ID=$(echo $GROUP_ID_TEMP | cut -d : -f2 | awk -F\" '{print $2}')
Rui F Ribeiro
  • 56,709
  • 26
  • 150
  • 232
subodh
  • 291
  • 2
    I'd expect a python interpreter to be present on most systems today. If it does, your problem has a trivial solution. – mustaccio Aug 01 '18 at 12:28
  • from python i already done it but in server python wont be available . – subodh Aug 01 '18 at 14:57
  • 2
    This is marked as a duplicate, but it's not a duplicate, since the question is about awk and sed. – David Roussel Aug 24 '18 at 08:02
  • Here is another way: cat json|xargs|cut -c 4-|awk 'BEGIN{RS="}, {";ORS="\n";FS=", ";FS=", "; OFS=";"} { gsub (" : ", "=",$0);print $1,$2,$3}'|while read line do eval $line [ $name == "Admin_UserGroup" ] && echo $id $path done

    Step by step: xargs - put everything in one line cut -c 4- - cuts first 3 chars which are { [ awk.'{BEGIN ... RS="}, {"; - Tell to awk }, { threat as record separator. ORS="\n"; - Tell to awk we want \n to be new record separator.

    – Petras L Oct 10 '18 at 10:30
  • FS=", "; - Tell to awk we have field separator as ,. OFS=";" - Tell to awk we want to have field separator as ;. gsub (" : ", "=",$0) - replace : into = print $1,$2,$3 - print only 3 first fields while read line ... - read line by line our records eval $line - execute them (assign values to variables) $name == "Admin_UserGroup" ] && echo $id $path - check if variable $name has appropriate value and && print what we want – Petras L Oct 10 '18 at 10:30

5 Answers5

16

Using jq:

$ printf '%s\n' "$Group_ID" | jq '.[] | select(.name == "Admin_UserGroup")'
{
  "id": "89f3bd6a-33a9-4e02-9fe3-eae660c5a6cf",
  "name": "Admin_UserGroup",
  "path": "/Admin_UserGroup",
  "subGroups": []
}

This selects all objects in the array whose name key corresponds to a value of Admin_UserGroup.

Kusalananda
  • 333,661
  • 3
    For me jq cant be use as client wont allow to install any third party api in the production environment – subodh Aug 01 '18 at 09:37
  • 5
    @SubodhJoshi Well, the client will get code that is unstable instead then. – Kusalananda Aug 01 '18 at 09:41
  • means if same can be achieve through sedor awkit would be unstable ? – subodh Aug 01 '18 at 09:42
  • 11
    JSON is structured data, by using tools that are unaware of the structure (awk, sed) you run the risk of creating regex matching that's very brittle and will fail because it's not taking any care to respect that the JSON data is structured. – slm Aug 01 '18 at 09:45
  • 5
    @SubodhJoshi JSON is a structured document format in the same genre as XML. Parsing such documents with anything other than a dedicated parser is error prone. JSON has rules for how data is encoded and quoted, where keys goes and where newlines are allowed, etc. – Kusalananda Aug 01 '18 at 09:48
  • @slm While I agree with your comment applied to sed, I don't agree with respect to awk, which is far from restricted to regex. – Pac0 Aug 01 '18 at 10:37
  • @Pac0 - fair point. – slm Aug 01 '18 at 10:37
  • 2
    @Pac0 I agree that awk is not restricted to regular expressions. I do not agree that awk is well suited to do JSON parsing in. – Kusalananda Aug 01 '18 at 10:55
8

According to your consideration for basic commands, if your file has structure exactly as shown, grep may help you:

$ grep -B2 -A3 '"name" : "Admin_UserGroup"' File

but if all in one line:

$ sed -E 's/\},\s*\{/\},\n\{/g' File | grep  '"name" : "Admin_UserGroup"'

Using your original example, you'd use my solution like so:

$ echo "$Group_ID" | grep ...
slm
  • 369,824
7

You can use JSON.awk:

awk -f JSON.awk -v file1.json file2.json

https://github.com/step-/JSON.awk

  • 4
    To be a complete (self-contained) Answer, the code should be here in the Answer box, not a link. – Jeff Schaller Aug 01 '18 at 10:26
  • 2
    do one need to copy in answers the whole source of every tool that can perform jobs ? – Pac0 Aug 01 '18 at 10:40
  • 1
    @Pac0 - yes, you need to put JSON.awk on every system where you'll want to use this. – slm Aug 01 '18 at 10:45
  • 2
    @Pac0 or, if you meant that as a reply to my comment, answers should contain the necessary information for someone to take it to their system and use it. Systems (should) have awk installed, or else it's turtles all the way down to hand-writing your own compiler. See our Help Center regarding Why and how are some answers deleted? and Why are answers no longer being accepted from my account? both of which discourage link-only answers. – Jeff Schaller Aug 01 '18 at 11:03
  • @JeffSchaller yes, that was concerning your comment. Thank you for the answer and the links. I just made a (pending) edit to incoporate the source in the answer. I was afraid it was a bit too long, but I forgot that code blocks do use scroll bars. – Pac0 Aug 01 '18 at 11:12
  • @Pac0 - i rejected the edit b/c that AWK.json file is a 259 line script. – slm Aug 01 '18 at 12:43
  • The OP rejected jq because it is a "third party api". I think that JSON.awk is just as much of a "third party api", albeit in script form instead of c. – Digital Trauma Aug 01 '18 at 21:40
  • While this solution may work it has only one committer and 16 commits. If you look for a stable long term solution, this may not be the best option. – patrik May 19 '20 at 13:31
  • patrik, but it is in source control, and has automated tests. I don't see any automated tests for the other suggested approaches to parsing json with awk. – David Roussel Jun 01 '20 at 10:29
2

With recent versions of ksh93, you can parse json data into a ksh93 compound variable with read -m json:

print -r -- "$Group_ID" | IFS= read -rm json v
for ((i = 0; i < ${#v[@]}; i++)); do
  if [[ ${v[i].name} = Admin_UserGroup ]]; then
    print -r -- "${v[i]}"
  fi
done

It outputs:

(
        id=89f3bd6a-33a9-4e02-9fe3-eae660c5a6cf
        name=Admin_UserGroup
        path=/Admin_UserGroup
        typeset -C subGroups
)

Instead of print -r -- "${v[i]}", you can print ${v[i].path} or any other information from that compound variable.

  • I am getting this error -> -bash: syntax error near unexpected token('` – subodh Aug 01 '18 at 15:55
  • @SubodhJoshi, yes, as I said, it's for the ksh93 shell (recent versions: ksh93v- and newer), not the bash shell. Most of bash's syntax and features have been copied from ksh, however it's far from being a full clone. In particular bash doesn't support ksh's compound variables. – Stéphane Chazelas Aug 01 '18 at 16:00
  • Do i have to install a new library ? – subodh Aug 01 '18 at 16:07
0
$ awk '{if($0~/"AdminUserGroup"/) {NR=NR-1;print $0; NR=NR+2;print $0; } }'file1

In this way we can print info of lines near the searched string. Also I think instead of printing it we can put all those inside an an array also:

$ awk '{if($0~/"AdminUserGroup"/) {NR=NR-1;arr[$i]=(print $0);i=$((i+1)); \
    NR=NR+2; arr[$i]=(print $0); } }'file1.

Then we can use array details.

slm
  • 369,824