15

In python we can decorate functions with code that is automatically applied and executed against functions.

Is there any similar feature in bash?

In the script I'm currently working on, I have some boilerplate that test the required arguments and exit if they don't exist - and display some messages if the the debugging flag is specified.

Unfortunately I have to reinsert this code into every function and if I want to change it, I'll have to modify every function.

Is there a way to remove this code from each function and have it applied to all functions, similar to decorators in python?

nfarrar
  • 391

6 Answers6

14

That would be a lot easier with zsh that has anonymous functions and a special associative array with function codes. With bash however you could do something like:

decorate() {
  eval "
    _inner_$(typeset -f "$1")
    $1"'() {
      echo >&2 "Calling function '"$1"' with $# arguments"
      _inner_'"$1"' "$@"
      local ret=$?
      echo >&2 "Function '"$1"' returned with exit status $ret"
      return "$ret"
    }'
}

f() {
  echo test
  return 12
}
decorate f
f a b

Which would output:

Calling function f with 2 arguments
test
Function f returned with exit status 12

You can't call decorate twice to decorate your function twice though.

With zsh:

decorate()
  functions[$1]='
    echo >&2 "Calling function '$1' with $# arguments"
    () { '$functions[$1]'; } "$@"
    local ret=$?
    echo >&2 "function '$1' returned with status $ret"
    return $ret'
  • Stephane - is typeset necessary? Would it not declare it otherwise? – mikeserv Apr 23 '14 at 13:12
  • 1
    @mikeserv, eval "_inner_$(typeset -f x)" creates _inner_x as an exact copy of the original x (same as functions[_inner_x]=$functions[x] in zsh). – Stéphane Chazelas Apr 23 '14 at 13:21
  • I get that - but why do you need two at all? – mikeserv Apr 23 '14 at 13:27
  • 1
    You need a different context otherwise you wouldn't be able to catch the inner's return. – Stéphane Chazelas Apr 23 '14 at 13:36
  • Ok, but the function can catch its own return in a trap. I didn't demonstrate that here either, but I did in the links. And you don't necessarily need that either if you just call the generator instead of returning in the first place. – mikeserv Apr 23 '14 at 13:50
  • 2
    I don't follow you there. My answer is an attempt as a close map of what I understand python decorators to be – Stéphane Chazelas Apr 23 '14 at 14:25
  • It's very good, of course. In fact, the best of my own tricks come primarily from reading your answers here, and in mailing lists elsewhere. But - I just mean, why do we need three functions - _gen(), _catch(), _fn() when we can just _gen(), _fn() trap? – mikeserv Apr 23 '14 at 14:32
  • 1
    You know what? Nevermind, your link, and a little more thinking, and I get it - the _innner() part anyway. I doubt I'd use typeset to do it, but it's bash friendly anyway. I do see that you did very closely map the python decorator - which is exactly what was asked. Thanks for your time. – mikeserv Apr 23 '14 at 14:51
  • I will hazard though, that with a combination of the methods in both of our answers that it could be possible to decorate a function as many times as you liked. – mikeserv Apr 23 '14 at 14:57
6

I've already discussed the hows and whys of the way the below methods work on several occasions before so I won't do it again. Personally, my own favorites on the topic are here and here.

If you're not interested in reading that but still curious just understand that the here-docs attached to the function's input are evaluated for shell expansion before the function runs, and that they are generated anew in the state they were when the function was defined every time the function is called.

DECLARE

You just need a function that declares other functions.

_fn_init() { . /dev/fd/4 ; } 4<<INIT
    ${1}() { $(shift ; printf %s\\n "$@")
     } 4<<-REQ 5<<-\\RESET
            : \${_if_unset?shell will ERR and print this to stderr}
            : \${common_param="REQ/RESET added to all funcs"}
        REQ
            _fn_init $(printf "'%s' " "$@")
        RESET
INIT

RUN IT

Here I call upon _fn_init to declare me a function called fn.

set -vx
_fn_init fn \
    'echo "this would be command 1"' \
    'echo "$common_param"'

#OUTPUT#
+ _fn_init fn 'echo "this would be command 1"' 'echo "$common_param"'
shift ; printf %s\\n "$@"
++ shift
++ printf '%s\n' 'echo "this would be command 1"' 'echo "$common_param"'
printf "'%s' " "$@"
++ printf ''\''%s'\'' ' fn 'echo "this would be command 1"' 'echo "$common_param"'
#ALL OF THE ABOVE OCCURS BEFORE _fn_init RUNS#
#FIRST AND ONLY COMMAND ACTUALLY IN FUNCTION BODY BELOW#
+ . /dev/fd/4

    #fn AFTER _fn_init .dot SOURCES IT#
    fn() { echo "this would be command 1"
        echo "$common_param"
    } 4<<-REQ 5<<-\RESET
            : ${_if_unset?shell will ERR and print this to stderr}
            : ${common_param="REQ/RESET added to all funcs"}
        REQ
            _fn_init 'fn' \
               'echo "this would be command 1"' \
               'echo "$common_param"'
        RESET

REQUIRED

If I want to call this function it will die unless the environment variable _if_unset is set.

fn

#OUTPUT#
+ fn
/dev/fd/4: line 1: _if_unset: shell will ERR and print this to stderr

Please note the order of the shell traces - not only does the fn fail when called when _if_unset is unset, but it never runs in the first place. This is the most important factor to understand when working with here-document expansions - they must always occur first because they are <<input after all.

The error comes from /dev/fd/4 because the parent shell is evaluating that input before handing it off to the function. It's the simplest, most efficient way to test for requisite environment.

Anyway, the failure is easily remedied.

_if_unset=set fn

#OUTPUT#
+ _if_unset=set
+ fn
+ echo 'this would be command 1'
this would be command 1
+ echo 'REQ/RESET added to all funcs'
REQ/RESET added to all funcs

FLEXIBLE

The variable common_param is evaluated to a default value on input for every function declared by _fn_init. But that value is also changeable to any other which will also be honored by every function similarly declared. I'll leave off the shell traces now - we're not going into any uncharted territory here or anything.

set +vx
_fn_init 'fn' \
               'echo "Hi! I am the first function."' \
               'echo "$common_param"'
_fn_init 'fn2' \
               'echo "This is another function."' \
               'echo "$common_param"'
_if_unset=set ;

Above I declare two functions and set _if_unset. Now, before calling either function, I'll unset common_param so you can see they will set it themselves when I call them.

unset common_param ; echo
fn ; echo
fn2 ; echo

#OUTPUT#
Hi! I am the first function.
REQ/RESET added to all funcs

This is another function.
REQ/RESET added to all funcs

And now from the caller's scope:

echo $common_param

#OUTPUT#
REQ/RESET added to all funcs

But now I want it to be something else entirely:

common_param="Our common parameter is now something else entirely."
fn ; echo 
fn2 ; echo

#OUTPUT#
Hi! I am the first function.
Our common parameter is now something else entirely.

This is another function.
Our common parameter is now something else entirely.

And if I unset _if_unset?

unset _if_unset ; echo
echo "fn:"
fn ; echo
echo "fn2:"
fn2 ; echo

#OUTPUT#
fn:
dash: 1: _if_unset: shell will ERR and print this to stderr

fn2:
dash: 1: _if_unset: shell will ERR and print this to stderr

RESET

If you need to reset the function's state at any time it is easily done. You need only do (from within the function):

. /dev/fd/5

I saved the arguments used to initially declare the function in the 5<<\RESET input file-descriptor. So .dot sourcing that in the shell at any time will repeat the process that set it up in the first place. It's all pretty easy, really, and pretty much fully portable if you're willing to overlook the fact that POSIX doesn't actually specify the file-descriptor device node paths (which are a necessity for the shell's .dot).

You could easily expand on this behavior and configure different states for your function.

MORE?

This barely scratches the surface, by the way. I often use these techniques to embed little helper functions declarable at any time into the input of a main function - for instance, for additional positional $@arrays as needed. In fact - as I believe, it must be something very close to this that the higher order shells do anyway. You can see they're very easily programmatically named.

I also like to declare a generator function which accepts a limited type of parameter and then defines a single-use or otherwise scope-limited burner-function along the lines of a lambda - or an in-line function - that simply unset -f's itself when through. You can pass a shell function around.

mikeserv
  • 58,310
  • What's the advantage of that extra complexity with file descriptors over using eval? – Stéphane Chazelas Apr 22 '14 at 06:37
  • @StephaneChazelas There's no added complexity from my perspective. In fact, I see it the other way round. Also, the quoting is far easier, and .dot works with files and streams so you don't run into the same kind of argument list problems you might otherwise. Still, it's probably a matter of preference. I certainly think it's cleaner - especially when you get into evaling eval - that's a nightmare from where I sit. – mikeserv Apr 22 '14 at 07:18
  • @StephaneChazelas There is one advantage though - and it's a pretty good one. The initial eval and the second eval need not be back to back with this method. The heredocument is evaluated on input, but you don't have to .dot source until you're good and ready - or ever. This enables you a little more freedom in testing its evaluations. And it provides the flexibility of state on input - which can be handled other ways - but it's far less dangerous from that perspective than is eval. – mikeserv Apr 22 '14 at 07:22
2

I think one way to print information about function, when you

test the required arguments and exit if they don't exist - and display some messages

is to change bash builtin return and/or exit in the beginning of every script (or in some file, that you source every time before executing program). So you type

   #!/bin/bash
   return () {
       if [ -z $1 ] ; then
           builtin return
       else
           if [ $1 -gt 0 ] ; then
                echo function ${FUNCNAME[1]} returns status $1 
                builtin return $1
           else
                builtin return 0
           fi
       fi
   }
   foo () {
       [ 1 != 2 ] && return 1
   }
   foo

If you run this you will get:

   function foo returns status 1

That's may be easily updated with debugging flag if you need, somewhat like this:

   #!/bin/bash
   VERBOSE=1
   return () {
       if [ -z $1 ] ; then
           builtin return
       else
           if [ $1 -gt 0 ] ; then
               [ ! -z $VERBOSE ] && [ $VERBOSE -gt 0 ] && echo function ${FUNCNAME[1]} returns status $1  
               builtin return $1
           else
               builtin return 0
           fi
       fi
    }    

This way statement will be executed only when variable VERBOSE is set ( at least that's how I use verbose in my scripts). It certainly doesn't solve the problem of decorating function, but it can display messages in case function returns non-zero status.

Similarly you can redefine exit, by replacing all instances of return, if you want to exit from script.

EDIT: I wanted to add here the way I use to decorate functions in bash, if I have a lot of them and nested ones as well. When I write this script:

#!/bin/bash 
outer () { _
    inner1 () { _
        print "inner 1 command"
    }   
    inner2 () { _
        double_inner2 () { _
            print "double_inner1 command"
        } 
        double_inner2
        print "inner 2 command"
    } 
    inner1
    inner2
    inner1
    print "just command in outer"
}
foo_with_args () { _ $@
    print "command in foo with args"
}
echo command in body of script
outer
foo_with_args

And for the output I can get this:

command in body of script
    outer: 
        inner1: 
            inner 1 command
        inner2: 
            double_inner2: 
                double_inner1 command
            inner 2 command
        inner1: 
            inner 1 command
        just command in outer
    foo_with_args: 1 2 3
        command in foo with args

It can be helpful to someone that has functions and wants to debug them, to see in which function error occurred. It is based on three functions, which can are described below:

#!/bin/bash 
set_indentation_for_print_function () {
    default_number_of_indentation_spaces="4"
    #                            number_of_spaces_of_current_function is set to (max number of inner function - 3) * default_number_of_indentation_spaces 
    #                            -3 is because we dont consider main function in FUNCNAME array - which is if your run bash decoration from any script,
    #                            decoration_function "_" itself and set_indentation_for_print_function.
    number_of_spaces_of_current_function=`echo ${#FUNCNAME[@]} | awk \
        -v default_number_of_indentation_spaces="$default_number_of_indentation_spaces" '
        { print ($1-3)*default_number_of_indentation_spaces}
        '`
    #                            actual indent is sum of default_number_of_indentation_spaces + number_of_spaces_of_current_function
    let INDENT=$number_of_spaces_of_current_function+$default_number_of_indentation_spaces
}
print () { # print anything inside function with proper indent
    set_indentation_for_print_function
    awk -v l="${INDENT:=0}" 'BEGIN {for(i=1;i<=l;i++) printf(" ")}' # print INDENT spaces before echo
    echo $@
}
_ () { # decorator itself, prints funcname: args
    set_indentation_for_print_function
    let INDENT=$INDENT-$default_number_of_indentation_spaces # we remove def_number here, because function has to be right from usual print
    awk -v l="${INDENT:=0}" 'BEGIN {for(i=1;i<=l;i++) printf(" ")}' # print INDENT spaces before echo
    #tput setaf 0 && tput bold # uncomment this for grey color of decorator
    [ $INDENT -ne 0 ] && echo "${FUNCNAME[1]}: $@" # here we avoid situation where decorator is used inside the body of script and not in the function
    #tput sgr0 # resets grey color
}

I tried to put as much as possible in comments, but here is also the description: I use _ () function as decorator, the one I put after declaration of every function: foo () { _. This function prints the function name with the proper indentation, depending how deep function is in other function (as a default indentation I use 4 number of spaces). I usually print this in grey, to separate this from usual print. If function is needed to be decorated with arguments, or without, one can modify the pre-last line in decorator function.

In order to print something inside function, I introduced print () function that prints everything that is passed to it with proper indent.

The function set_indentation_for_print_function does exactly what it stands for, calculating indentation from ${FUNCNAME[@]} array.

This way has some flaws, for example one can't pass options to print like to echo, e.g. -n or -e, and also if function returns 1, it is not decorated. And also for arguments, passed to print more than terminal width, that will be wrapped on screen, one won't see the indentation for wrapped line.

The great way to use these decorators is to put them into separate file and in each new script to source this file source ~/script/hand_made_bash_functions.sh.

I think the best way to incorporate function decorator in bash, is to write decorator in body of each function. I think it is much easier to write function inside function in bash, because it has option to set all variables global, not like in standard Object-Oriented Languages. That makes it as if you're putting labels around your code in bash. At least that helped me for a debugging scripts.

0

Maybe the decorator examples in http://sourceforge.net/projects/oobash/ project can help you (oobash/docs/examples/decorator.sh).

0

To me this feels like the simplest way to implement a decorator pattern inside bash.

#!/bin/bash

function decorator {
    if [ "${FUNCNAME[0]}" != "${FUNCNAME[2]}" ] ; then
        echo "Turn stuff on"
        #shellcheck disable=2068
        ${@}
        echo "Turn stuff off"
        return 0
    fi
    return 1
}

function highly_decorated {
    echo 'Inside highly decorated, calling decorator function'
    decorator "${FUNCNAME[0]}" "${@}" && return
    echo 'Done calling decorator, do other stuff'
    echo 'other stuff'
}

echo 'Running highly decorated'
# shellcheck disable=SC2119
highly_decorated
  • Why are you disabling these ShellCheck warnings? They seem correct (certainly the SC2068 warning should be fixed by quoting "$@"). – dimo414 Apr 13 '20 at 21:23
0

I do a lot (perhaps too much :) ) metaprogramming in Bash, and have found decorators invaluable for re-implementing behavior on the fly. My bash-cache library uses decoration to transparently memoize Bash functions with minimal ceremony:

my_expensive_function() {
  ...
} && bc::cache my_expensive_function

Obviously bc::cache is doing more than just decorating, but the underlying decoration relies on bc::copy_function to copy an existing function to a new name, so that the original function can be overwritten with a decorator.

# Given a name and an existing function, create a new function called name that
# executes the same commands as the initial function.
bc::copy_function() {
  local function="${1:?Missing function}"
  local new_name="${2:?Missing new function name}"
  declare -F "$function" &> /dev/null || {
    echo "No such function ${function}" >&2; return 1
  }
  eval "$(printf "%s()" "$new_name"; declare -f "$function" | tail -n +2)"
}

Here's a simple example of a decorator that times the decorated function, using bc::copy_function:

time_decorator() {
  bc::copy_function "$1" "time_dec::${1}" || return
  eval "${1}() { time time_dec::${1} "'"\$@"; }'
}

Demo:

$ slow() { sleep 2; echo done; }

$ time_decorator slow

$ $ slow
done

real    0m2.003s
user    0m0.000s
sys     0m0.002s
dimo414
  • 1,797