37

Is there a way to print an entire array ([key]=value) without looping over all elements?

Assume I have created an array with some elements:

declare -A array
array=([a1]=1 [a2]=2 ... [b1]=bbb ... [f500]=abcdef)

I can print back the entire array with

for i in "${!array[@]}"
do
echo "${i}=${array[$i]}"
done

However, it seems bash already knows how to get all array elements in one "go" - both keys ${!array[@]} and values ${array[@]}.

Is there a way to make bash print this info without the loop?

Edit:
typeset -p array does that!
However I can't remove both prefix and suffix in a single substitution:

a="$(typeset -p array)"
b="${a##*(}"
c="${b%% )*}"

Is there a cleaner way to get/print only the key=value portion of the output?

Dani_l
  • 4,943

6 Answers6

32

I think you're asking two different things there.

Is there a way to make bash print this info without the loop?

Yes, but they are not as good as just using the loop.

Is there a cleaner way to get/print only the key=value portion of the output?

Yes, the for loop. It has the advantages that it doesn't require external programs, is straightforward, and makes it rather easy to control the exact output format without surprises.


Any solution that tries to handle the output of declare -p (typeset -p) has to deal with a) the possibility of the variables themselves containing parenthesis or brackets, b) the quoting that declare -p has to add to make it's output valid input for the shell.

For example, your expansion b="${a##*(}" eats some of the values, if any key/value contains an opening parenthesis. This is because you used ##, which removes the longest prefix. Same for c="${b%% )*}". Though you could of course match the boilerplate printed by declare more exactly, you'd still have a hard time if you didn't want all the quoting it does.

This doesn't look very nice unless you need it.

$ declare -A array=([abc]="'foobar'" [def]='"foo bar"')
$ declare -p array
declare -A array='([def]="\"foo bar\"" [abc]="'\''foobar'\''" )'

With the for loop, it's easier to choose the output format as you like:

# without quoting
$ for x in "${!array[@]}"; do printf "[%s]=%s\n" "$x" "${array[$x]}" ; done
[def]="foo bar"
[abc]='foobar'

# with quoting
$ for x in "${!array[@]}"; do printf "[%q]=%q\n" "$x" "${array[$x]}" ; done
[def]=\"foo\ bar\"
[abc]=\'foobar\'

From there, it's also simple to change the output format otherwise (remove the brackets around the key, put all key/value pairs on a single line...). If you need quoting for something other than the shell itself, you'll still need to do it by yourself, but at least you have the raw data to work on. (If you have newlines in the keys or values, you are probably going to need some quoting.)

With a current Bash (4.4, I think), you could also use printf "[%s]=%s" "${x@Q}" "${array[$x]@Q}" instead of printf "%q=%q". It produces a somewhat nicer quoted format, but is of course a bit more work to remember to write. (And it quotes the corner case of @ as array key, which %q doesn't quote.)

If the for loop seems too weary to write, save it a function somewhere (without quoting here):

printarr() { declare -n __p="$1"; for k in "${!__p[@]}"; do printf "%s=%s\n" "$k" "${__p[$k]}" ; done ;  }  

And then just use that:

$ declare -A a=([a]=123 [b]="foo bar" [c]="(blah)")
$ printarr a
a=123
b=foo bar
c=(blah)

Works with indexed arrays, too:

$ b=(abba acdc)
$ printarr b
0=abba
1=acdc
ilkkachu
  • 138,973
  • Note that the output of your printf ...%q... variant is not suitable for reinput to the shell if the array has a @ key as %q doesn't quote it and a=([@]=value) is a syntax error in bash. – Stéphane Chazelas May 23 '17 at 07:02
  • @StéphaneChazelas, apparently. "${x@Q}" quotes that too, since it quotes all strings (and looks nicer). added a note about using that. – ilkkachu May 23 '17 at 07:26
  • Yes, copied from mksh. Another operator of yet a different shape which cannot be combined with most others. Again, see zsh with its variable expansion flags (that again predates bash's by decades and with which you can choose the quoting style: ${(q)var}, ${(qq)var}...) for a better design. bash has the same issue as mksh in that it doesn't quote the empty string (not an issue here as anyway bash doesn't support empty keys). Also, when using quoting styles other than single quote (${var@Q} resorts to $'...' for some values) it's important that the code be reinput in the same locale. – Stéphane Chazelas May 23 '17 at 09:20
  • @StéphaneChazelas, I think you mean an unset value, not an empty string? (x=; echo "${x@Q}" does give '', unset x; echo "${x@Q}" gives nothing.) Bash's @Q seems to prefer $'\n' over a literal newline, which may actually be good in some situations (but I can't tell what others prefer). Of course having a choice there wouldn't be bad. – ilkkachu May 23 '17 at 09:36
  • Oh yes sorry, I hadn't realised that. That's a difference from mksh then. The $'...' syntax is a potential problem in things like LC_ALL=zh_HK.big5hkscs bash -c 'a=$'\''\n\u3b1'\''; printf "%s\n" "${a@Q}"' which outputs $'\n<0xa3><0x5c>' and 0x5c alone is backslash so you'd have a problem if that quote was interpreted in a different locale. – Stéphane Chazelas May 23 '17 at 10:07
16
declare -p array
declare -A array='([a2]="2" [a1]="1" [zz]="Hello World" [b1]="bbb" [f50]="abcd" )'

2 fork

Maybe this:

printf "%s\n" "${!array[@]}"
a2
a1
f50
zz
b1

printf "%s\n" "${array[@]}" 2 1 abcd Hello World bbb

printf "%s\n" "${!array[@]}" "${array[@]}" | pr -2t a2 2 a1 1 f50 abcd zz Hello World b1 bbb

3 forks

or this:

paste -d= <(printf "%s\n" "${!array[@]}") <(printf "%s\n" "${array[@]}")
a2=2
a1=1
f50=abcd
zz=Hello World
b1=bbb

No fork

to be compared to

for i in "${!array[@]}";do printf "%s=%s\n" "$i" "${array[$i]}";done
a2=2
a1=1
f50=abcd
zz=Hello World
b1=bbb

Execution times comparission

As last syntax don't use fork, they could be quicker:

time printf "%s\n" "${!array[@]}" "${array[@]}" | pr -2t | wc
      5      11      76
real    0m0.005s
user    0m0.000s
sys     0m0.000s

time paste -d= <(printf "%s\n" "${!array[@]}") <(printf "%s\n" "${array[@]}") | wc 5 6 41 real 0m0.008s user 0m0.000s sys 0m0.000s

time for i in "${!array[@]}";do printf "%s=%s\n" "$i" "${array[$i]}";done | wc 5 6 41 real 0m0.002s user 0m0.000s sys 0m0.001s

But this affirmation doesn't stay true if the array becomes big; if reducing forks is efficient for small process, using dedicated tools is more efficient for bigger process.

for i in {a..z}{a..z}{a..z};do array[$i]=$RANDOM;done

time printf "%s\n" "${!array[@]}" "${array[@]}" | pr -2t | wc 17581 35163 292941 real 0m0.150s user 0m0.124s sys 0m0.036s

time paste -d= <(printf "%s\n" "${!array[@]}") <(printf "%s\n" "${array[@]}") | wc 17581 17582 169875 real 0m0.140s user 0m0.000s sys 0m0.004s

time for i in "${!array[@]}";do printf "%s=%s\n" "$i" "${array[$i]}";done | wc 17581 17582 169875 real 0m0.312s user 0m0.268s sys 0m0.076s

Remark

As both (forked) solutions use alignment, none of them will work if any variable contains a newline. In this case, the only way is a for loop.

More robust and detailed answer at StackOverflow

  • While looking clever, both ways are less efficient than a for. Which is a shame, really. – Satō Katsura May 22 '17 at 16:50
  • @SatoKatsura I agree, but if slower, syntax using pr is shorter... I'm not sure about pr syntax stay slower, even with big arrays! – F. Hauri - Give Up GitHub May 22 '17 at 17:14
  • For last example (paste -d), why haven't do so: printf "%s=%s\n" "${!array[@]}" "${array[@]}"? The same result, but more simple. – MiniMax May 22 '17 at 18:35
  • 2
    @MiniMax Because it doesn't produce the correct result (same elements, wrong order). You'd need to zip arrays ${!array[@]} and ${array[@]} first for that to work. – Satō Katsura May 22 '17 at 19:30
  • @SatoKatsura Yes, I missed it. – MiniMax May 22 '17 at 19:48
  • 1
    That last snippet with paste is longer than the for loop in the question written on one line for i in "${!array[@]}"; do echo "$i=${array[$i]}" ; done, but requires two subshells and an external program. How is that neater? The solution with pr also breaks if there are many elements, as it tries to paginate the output. You'd need to use something like | pr -2t -l"${#array[@]}" which is starting to get hard to remember compared to the simple loop, and again, is longer than it. – ilkkachu May 22 '17 at 21:45
  • @ilkkachu Then, while we are at it, using echo is wrong too, since $i can begin with a dash. :) printf should be used instead. – Satō Katsura May 23 '17 at 04:12
  • 1
    In bash, cmd1 | cmd2 means 2 forks, even if cmd1 or cmd2 or both are builtin. – Stéphane Chazelas May 23 '17 at 06:28
  • @StéphaneChazelas You're right: <(builtin) is a fork, but exertnal pr is a fork too. I will re-count... – F. Hauri - Give Up GitHub May 23 '17 at 06:42
6

Bash 5.1 allows a very straight forward way to display associative arrays by using the K value as in ${arr[@]@K}:

$ declare -A arr
$ arr=(k1 v1 k2 v2)
$ printf "%s\n" "${arr[@]@K}"
k1 "v1" k2 "v2" 

From the Bash 5.1 description document:

hh. New `K' parameter transformation to display associative arrays as key-value pairs.

It is well explained in the Bash Reference Manual → 3.5.3 Shell Parameter Expansion:

${parameter@operator}

K
Produces a possibly-quoted version of the value of parameter, except that it prints the values of indexed and associative arrays as a sequence of quoted key-value pairs (see Arrays).

fedorqui
  • 7,861
  • 7
  • 36
  • 74
  • Note that those are not separated out. The only useful application of that operator is in something like typeset -A "copy=(${arr[@]@K})". Which works now that bash has eventually copied zsh's arr=(k1 v1 k2 v2) form of associative array definition, but still can't be used like zsh's copy=("${(@kv)arr}") form because of that quoting and the fact that the elements are not separated out. – Stéphane Chazelas Dec 10 '20 at 11:56
  • @StéphaneChazelas oh, true, they all appear together. The first case you mention is interesting, note you can also use now ${arr[@]@A} since it produces something like declare -A arr=([k1]="v1" [k2]="v2" )`. That is, the definition of the associative array. Not sure, though, how to use it apart from within a process substitution or something – fedorqui Dec 10 '20 at 14:38
3

If you're looking for a shell with better associative array support, try zsh.

In zsh (where associative arrays were added in 1998, compared to 1993 for ksh93 and 2009 for bash), $var or ${(v)var} expands to the (non-empty) values of the hash, ${(k)var} to the (non-empty) keys (in the same order), and ${(kv)var} to both keys and values.

To preserve the empty values, like for arrays, you need to quote and use the @ flag.

So to print the keys and values, it's just a matter of

printf '%s => %s\n' "${(@kv)var}"

Though to account for a possibly empty hash, you should do:

(($#var)) &&  printf '%s => %s\n' "${(@kv)var}"

Also note that zsh uses a much more sensible and useful array definition syntax than ksh93's (copied by bash):

typeset -A var
var=(k1 v1 k2 v2 '' empty '*' star)

Which makes it a lot easier to copy or merge associative arrays:

var2=("${(@kv)var1}")
var3+=("${(@kv)var2}")
var4=("${@kv)var4}" "${(@kv)var5}")

(you can't easily copy a hash without a loop with bash, and note that bash currently doesn't support empty keys or key/values with NUL bytes).

See also zsh array zipping features which you'll typically need to work with associative arrays:

keys=($(<keys.txt)) values=($(<values.txt))
hash=(${keys:^values})
1

Since typeset does what you want why not just edit its output?

typeset -p array | sed s/^.*\(// | tr -d ")\'\""  | tr "[" "\n" | sed s/]=/' = '/

gives

a2 = 2  
a1 = 1  
b1 = bbb 

Where

array='([a2]="2" [a1]="1" [b1]="bbb" )'

Verbose but it's pretty easy to see how the formatting works: just execute the pipeline with progressively more of the sed and tr commands. Modify them to suit pretty printing tastes.

Nadreck
  • 402
  • 1
    That kind of a pipeline is bound to fail the moment some of the keys or values of the array contain any of the characters you're replacing, like parenthesis, brackets or quotes. And a pipeline of seds and tr's isn't even much simpler than a for loop with printf. – ilkkachu May 22 '17 at 21:19
  • Also, you do know that tr translate character-by-character, it doesn't match strings? tr "]=" " =" changes "]" to a space and an = to an =, regardless of position. So you could probably just combine all three tr's to one. – ilkkachu May 22 '17 at 21:22
  • Very true about some of the non-alphanumeric characters gumming this up. However anything that has to deal them gets an order of magnitude more complex and less readable so unless there's a really good reason to have them in your data feed and that's stated in the question I assume they're filtered out before we got here. Should always have your explicit caveat tho. I find these pipelines to be simpler, for example and debugging purposes, than a printf glob that either works perfectly or blows up in your face. Here you make one simple change per element, test it, then add 1 more. – Nadreck May 23 '17 at 01:10
  • My bad! Got my _tr_s and _sed_s totally mixed up! Fixed in the latest edit. – Nadreck May 23 '17 at 01:12
1

One more option is to list all the variables and grep for the one you want.

set | grep -e '^aa='

I use this for debugging. I doubt that it is very performant since it lists all the variables.

If you were doing this often you could make it a function like this:

aap() { set | grep -e "^$1="; }

Unfortunately when we check performance using time:

$ time aap aa aa=([0]="abc") . real 0m0.014s user 0m0.003s sys 0m0.006s

Therefore, if you were doing this very often, you'd want @F.Hauri's NO FORKS version because it is so much faster.

xer0x
  • 121