How to read a properties file into an associative array?

Question

I'd like to read the properties in a properties file into an associative array. How can I do it?

Specifics on what to parse: hash and equal signs. Everything else is a bonus.

Properties file content example:

# comment
a=value-a
b=http://prefix.suffix:8080/?key=value
c=password_with\\backslash-and=equals

I'd like this bash associative array to be constructed out of that file:

 declare -A props='(
  [a]="value-a"
  [b]="http://prefix.suffix:8080/?key=value"
  [c]="password_with\\backslash-and=equals" )'

(expected output of declare -p on that associative array, note that ${props[c]} contains only one backslash, "\\" is '\').

Answer 1

Use a real parser like perl's Config::Properties module. I would do the whole script in perl, but if you have to use bash, you could do something like:

typeset -A props
while IFS= read -rd '' key && IFS= read -rd '' value; do
  props[$key]=$value
done < <(
  perl -MConfig::Properties -l0 -e '
   $p = Config::Properties->new();
   $p->load(STDIN);
   print for $p->properties' < file.properties
)

(also works with zsh).

Implementing a full parser in bash would be a lot of work and mean re-inventing the wheel. You can implement a good subset with a simple while read loop though as the read builtin expects an input format that is very similar to those properties files:

typeset -A props
while IFS=$':= \t' read key value; do
  [[ $key = [#!]* ]] || [[ $key = "" ]] || props[$key]=$value
done < file.properties

(also works with ksh93 and zsh, the two other Bourne-like shell supporting associative arrays).

That handles:

• prop = value
• prop: value
• prop value
• comments at the start of the line (! and # with optional leading blanks)
• backslash escaping (as in foo\:\:bar=value for keys containing delimiters or foo=\ bar or the password_with\\backslash-and=equals in your sample).
• line continuation with backslash

However, if we check against the specification

• That doesn't handle \n, \r, \uXXXX... sequences

• LF is the only recognised line delimiter (not CR nor CRLF).

• FF is not recognised as a whitespace (we can't just add it to $IFS as depending on the shell and version, \f will not necessarily be recognised as an IFS-whitespace character¹).

• for an input like foo: bar = , that stores bar in ${props[foo]} instead of bar = (foo: bar:baz: is OK though). That's only a problem when the value of the property contains one (unescaped) delimiter (: optionally surrounded by SPC/TAB characters, = optionally surrounded by SPC/TAB characters or sequence of one or more SPC/TAB characters) and it is at the end.

• it treats as comments lines that start with \! or \#. Only a problem for properties whose name starts with ! or #.

• in

    prop=1\
     2\
     3
  

we get 1 2 3 instead of 123: the leading spaces are not ignored in the continuation lines as they should be.

---

^{² IFS whitespace characters, per POSIX are the characters classified as [:space:] in the locale (which generally includes \f but doesn't have to) and that happen to be in $IFS though in ksh88 (on which the POSIX specification is based) and in most shells, that's still limited to SPC, TAB and NL. The only POSIX compliant shell in that regard I found was yash. ksh93 and bash (since 5.0) also include other whitespace (such as CR, FF, VT...), but limited to the single-byte ones (beware on some systems like Solaris, that includes the non-breaking-space which is single byte in some locales)}

Answer 2

Here is howto in Bash4+

#!/usr/bin/env bash

declare -A properties

# Read with:
# IFS (Field Separator) =
# -d (Record separator) newline
# first field before separator as k (key)
# second field after separator and reminder of record as v (value)
while IFS='=' read -d $'\n' -r k v; do
  # Skip lines starting with sharp
  # or lines containing only space or empty lines
  [[ "$k" =~ ^([[:space:]]*|[[:space:]]*#.*)$ ]] && continue
  # Store key value into assoc array
  properties[$k]="$v"
  # stdin the properties file
done < file.properties

# display the array for testing
typeset -p properties

file.properties:

# comment
a=value-a
b=http://prefix.suffix:8080/?key=value
c=password_with\\backslash-and=equals

d e=the d e value
  # comment

The output of this script from the supplied data sample:

declare -A properties=(["d e"]="the d e value" [c]="password_with\\\\backslash-and=equals" [b]="http://prefix.suffix:8080/?key=value" [a]="value-a" )

Answer 3

For the most common subset of that dataformat, you can use a short function, using bash variable expansion and regexp matching.

Note: This expects lines to be in ^key = value$ format, or ^#.*$ and ^!.*$ for comments. Adapt the code, or pre-process your data otherwise

$ cat /tmp/propdata 
k1 = v1
# A comment
k2 = v2and some s=t=u=f=f
! Another comment
k3 = v3

$ unset DATA
$ declare -A DATA

$ props(){ while read line || [[ -n $line ]]; do
[[ "$line" =~ ^#|^! ]] && continue;
if [[ "${line% =*}" ]]; then DATA[${line% =*}]="${line#*= }" ; fi ;
done < $1 ; }

$ props /tmp/propdata

$ echo "${DATA[k3]}"
v3
$ echo "${DATA[k2]}"
v2and some s=t=u=f=f

Edit: Updated to trim the spaces around the "=" for key and value

Edit2: Filters comments now too.

Answer 4

declare -A properties
function readPopertyFile
{
    while read line || [[ -n $line ]]; do
        key=`echo $line | cut -s -d'=' -f1`
        if [ -n "$key" ]; then
            value=`echo $line | cut -d'=' -f2-`
            properties["$key"]="$value"
        fi
    done < $1
}

Usage:

readPopertyFile "file.properties"

Will read the properties into an associative array variable named properties.

* Works in bash. Don't know about other shells.

* Won't handle multi-line properties.