If-else Parameter substitution

Question

I'm running a series of experiments with bash and want to store the log files in a directory whose name is based on the experiment configuration. Some items in the configuration are boolean (true/false). Take the following configuration as an example:

batch_size=16
fp16=false
bf16=true
checkpoint_activations=true

I'd like to store the log file from the experiment with above configuration as input in a directory with the following name:

output_dir="experiment_bs${batch_size}_dt${fp16 if fp16=true else bf16}_${cp if checkpoint_activations=true else empty}"

Of course, I could declare helper variables:

data_type=""
"${fp16}" && data_type=fp16
"${bf16}" && data_type=bf16
"${cp}" && cp="_cp" || cp=""
output_dir="experiment_bs${batch_size}_dt${data_type}${cp}"

But I find this somewhat clunky and hope that parameter substitutions could be useful here. "${bf16:+bf16}" won't help in my case because this will always print "bf16" regardless of its boolean value as long as it's defined.

Are there any parameter substitutions that could be applied to this use-case? Or is there an even better in-line solution to this problem?

Note: there's an application-specific reason why I don't directly use data_type in my configuration.

Answer 1

You can put any bash commands you want inside $(...), so you could write:

output_dir="experiment_bs${batch_size}_dt$([[ $fp16 = true ]] && echo $fp16 || echo $bf16)_$([[ $checkpoint_activation = true ]] && echo $cp || echo empty)"

Although for legibility I might write instead:

printf -v output_dir "experiment_bs%s_dt%s_%s" \
  "$batch_size" \
  "$([[ $fp16 = true ]] && echo "$fp16" || echo "$bf16")" \
  "$([[ $checkpoint_activation = true ]] && echo "$cp" || echo empty)"

Given your sample inputs...

batch_size=16
fp16=false
bf16=true
checkpoint_activations=true

...both of the above produce the value:

experiment_bs16_dttrue_empty

Answer 2

In zsh, you could define a ? function (and an alias to prevent it being treated as a glob pattern) that implements a form of ternary ? condition if-yes if-no operator, reminiscent of C's condition ? if-yes : if-no:

alias "?='?'"
'?'() if eval $1; then print -r -- $2; else print -r -- $3; fi
output_dir=experiment_bs${batch_size}dt$(? $fp16 fp16 bf16)$(? $cp cp)

With zsh 6.0+ (not released yet as of 2024-02-06), you can change it to:

alias "?='?'"
'?'() if eval $1; then REPLY=$2; else REPLY=$3; fi
output_dir=experiment_bs${batch_size}dt${|? $fp16 fp16 bf16}${|? $cp cp}

to avoid the forking of a process to get the result and allow values to end in newline characters (a feature called valsub (value substitution) copied from mksh).

Note that that ternary operator evaluates the code in the first argument to decide whether to return $2 or $3, so those $fp16/$cp are expected to contain either true or false. Change to $(? '[[ $fp16 = true ]]' fp16 bp16) to check whether $fp16 contains true or anything else.

See also this discussion on the zsh mailing list for some builtin approaches at a ternary operator. And this Q&A about valsubs with details about it and alternatives.

Answer 3

If fp16 is a configuration variable, then I wouldn't do "${fp16}" && data_type=fp16 since it turns that configuration variable into a command. Even if we discount the possibility of someone putting something like reboot there, even a typo would cause some odd-looking error messages (like "tru: command not found", or whatever).

Then again, perhaps that just works as a reminder to validate the values your script gets, e.g. with a checker function like:

checkbool() {
    case $1 in
        true|false) return 0;; 
        *) echo >&2 "'$1' is an invalid boolean (must be 'true' or 'false'";
           exit 1;;
    esac
}
checkbool "$fp16"
checkbool "$bf16"
# ...

Also consider if fp16 and bf16 even make sense as independent variables?

In:

"${fp16}" && data_type=fp16
"${bf16}" && data_type=bf16

if both fp16 and bf16 are true, the latter takes precedence. And if neither is set, data_type is left empty, which may or may not be valid. I'm not sure of your exact scenario, but I am left wondering if it'd be better to just have data_type as the configuration variable directly. Okay, the post says there's a reason to not use data_type directly, but thinking about what happens if both or none of the settings are enabled might still make sense.

Anyway, if you want the parameter expansions like "${bf16:+bf16}" to work, you need to use an empty value as falsy, and any non-empty string as truthy. Then you could do e.g. data_type="${enable_fp16:+fp16}", but even that seems hard to use since I don't think there's a good way to get the empty string to turn into the default value, without leaking some other value there. E.g. the opposite "${enable_fp16:-bf16}" would turn the empty string into bf16, but it'd also return the string yes as-is.

And if you were to use empty/non-empty values within the script, would you want to expose that bit internal detail in the configuration to the user? Or would be just better for usability to write out the conditionals to turn the config values into what ever values the script actually needs, clunky or not?

I would go with something like this, which perhaps feels verbose but didn't take that too long to write, really:

# config
batch_size=16
fp16=false
bf16=true
checkpoint_activations=true
## code
# this treats anything that's not 'true' as falsy
if   [[ $fp16  = true && $bf16 != true ]]; then
    data_type=fp16
elif [[ $fp16 != true && $bf16  = true ]]; then
    data_type=bf16
else
    echo >&2 "exactly one of fp16 and bf16 must be 'true'"
    exit 1
fi
cp=
if [[ $checkpoint_activations = true ]]; then
    cp=_cp
fi
# (maybe the value of $batch_size should also be checked, whatever
output_dir="experiment_bs${batch_size}_dt${data_type}${cp}"

Of course one could also check on each assignment to data_type if it was already set, instead of checking the value of each input variable on each condition. (As done above, adding a third variable would require changes to both existing conditions too.)

If you want to go the concise way, the two-way choice function from Stéphane's answer would also work in Bash with minor modifications. Though I'd still rather explicitly check the value, so something like this maybe:

choose() if [[ $1 = true ]]; then printf "%s\n" "$2"
         else printf "%s\n" "$3"
         fi
data_type=$(choose "$fp16" fp16 bf16)
# etc.

Of course, the decision between verbose and explicit vs. compact and concise code is always up to the programmer.