bash: exported function not visible, but variables are

Question

Over the years I've collected sort of a library of bash functions the shell and scripts refer to. To decrease the import boilerplate, I'm exploring options how to reasonably include the library in scripts.

My solution has two parts - first importing configuration (env vars), followed by sourcing the library of functions.

~/bash_envs: (the configuration)

export SOME_VAR=VALUE
export SHELL_LIB=/path/to/library.sh
convenience funtion, so scripts who source env_vars file (ie this file) can
simply call it, instead of including the same block in each file themselves.
function _load_common() {
    # import common library/functions:
    source $SHELL_LIB
}
export -f _load_common
marker var used to detect whether env vars (ie this file) have been loaded:
export __ENV_VARS_LOADED_MARKER_VAR=loaded

Now following code is ran from scripts:

if [[ "$__ENV_VARS_LOADED_MARKER_VAR" != loaded ]]; then  # in our case $__ENV_VARS_LOADED_MARKER_VAR=loaded, ie this block is not executed
    USER_ENVS=/home/laur/bash_envs
if [[ -r "$USER_ENVS" ]]; then
    source "$USER_ENVS"
else
    echo -e "\n    ERROR: env vars file [$USER_ENVS] not found! Abort."
    exit 1
fi

fi
_load_common

This produces _load_common: command not found exception. Why is that? Note __ENV_VARS_LOADED_MARKER_VAR=loaded is nicely exported and visible which is why there's no reason to source $USER_ENVS; yet _load_common() function is not found, albeit it being exported from the same place as __ENV_VARS_LOADED_MARKER_VAR.

Answer 1

The problem

Observe:

$ bash -c 'foobar () { :; }; export -f foobar; dash -c env' |grep foobar
$ bash -c 'foobar () { :; }; export -f foobar; bash -c env' |grep foobar
BASH_FUNC_foobar%%=() {  :
$ bash -c 'foobar () { :; }; export -f foobar; ksh93 -c env' |grep foobar
BASH_FUNC_foobar%%=() {  :
$ bash -c 'foobar () { :; }; export -f foobar; mksh -c env' |grep foobar
$ bash -c 'foobar () { :; }; export -f foobar; zsh -c env' |grep foobar
BASH_FUNC_foobar%%=() {  :
$ bash -c 'foobar () { :; }; export -f foobar; busybox sh -c env' |grep foobar
BASH_FUNC_foobar%%=() {  :

Environment variables are a feature of the Unix operating system. Support for them goes all the way down to the kernel: when a program calls another program (with the execve system call), one of the parameters of the call is the new program's environment.

The built-in command export in sh-style shells (dash, bash, ksh, …) causes a shell variable to be used as an environment variable which is transmitted to processes that the shell calls. Conversely, when a shell is called, all environment variables become shell variables in that shell instance.

Exported functions are a bash feature. Bash “exports” a function by creating an environment variable whose name is derived from the name of the function and whose value is the body of the function (plus a header and a trailer). You can see above how the name of the environment variable is constructed: BASH_FUNC_ then the name of the function then %%.

This name is not a valid name for a shell variable. Recall that shells import environment variables as shell variables when they start. Different shells have different behaviors when the name of an environment variable is not a valid shell variable. Some pass the variable through to their subprocesses (above: bash, ksh93, zsh, BusyBox), while others only pass the exported shell variables to their subprocesses (above: dash, mksh), which effectively removes the environment variables whose name is not a valid shell variable (non-empty sequence of ASCII letters, digits and _).

Originally, bash used an environment variable with the same name as the function, which would mostly have avoided this problem. (Only mostly: function names can contain characters that are not allowed in shell variable names, such as -.) But this had other downsides, such as not allowing to export a shell variable and a function with the same name (whichever one was exported last would overwrite the other in the environment). Critically, bash changed when it was discovered that the original implementation caused a major security hole. (See also What does env x='() { :;}; command' bash do and why is it insecure?, When was the shellshock (CVE-2014-6271/7169) bug introduced, and what is the patch that fully fixes it?, How was the Shellshock Bash vulnerability found?) A downside of this change is that exported functions no longer go through some programs, including dash and mksh.

Your system probably has dash as /bin/sh. It's a very popular choice. /bin/sh is used a lot, so the chances are high that there was a call to sh somewhere in the call path from the original instance of bash that ran export -f _load_common to the instance of bash that tried to use the function. __ENV_VARS_LOADED_MARKER_VAR passed through because it has a valid variable name, but BASH_FUNC__load_common%% didn't pass through.

The solution

Don't use exported functions. They have little use in the first place, and for you they are completely useless. The only advantage of exporting functions is to call bash without requiring that instance of bash to read the definition of the function from somewhere, for example to define a function in a script and to pass it to a bash instance invoked from find -exec or xargs or parallel. But in your case, you already have code to read the function definition. So just read the function definition unconditionally. Remove export -f _load_common, remove __ENV_VARS_LOADED_MARKER_VAR, and just call source "$USER_ENVS".