Wednesday 24 November 2021

BASH inline functions or macros

Sometimes you want to write bash macros instead of bash functions.

You want to write some bash which can declare local variables in the caller context.

Maybe it parses some data and creates a bunch of hashes with a known prefix (that's as much namespace management as bash gives you) and populates a named list with the names of those hashes.

And then you need to use that twice so you want to factor it into a function. Only you can't these aren't meant to be global variables.

So you have a few solutions, all using source in one way or another

You could put it into a separate file and source it whenever you need it.

You could define it in a string or here document and source it when needed (but you loose bash syntax support in the editor).

You can call the function using command substitution and have it execute declare -p on the variables you want to export.

e.g. source <( my-function args )

Or to enjoy all the side effects directly, you can export the function body using declare -f and strip the first line (which is the function declaration) my masking it with a comment, and then execute the rest directly, using source.

First iteration

Hint: this one is reliable.
function-body() {
echo -n '#'
declare -f "$1"
}

And to invoke it:

source <( function-body my-function ) arg1 arg2 arg3

Example

my-function()
{
echo "my-function $*";
echo a=$1;
a=$1
}
test-my-function() {
local a
source <( function-body my-function ) "$@"
echo "test says a=$a"
}

$ a=nothing ; test-my-function something ; echo "shell says a=$a"
my-function something
a=something
test says a=something
shell says a=nothing

Maybe  source <( function-body my-function ) "$@"  is a bit verbose when calling a function inline every time.

Bugs

The first iteration is fine which makes sense as eval is not used:
$ v='$thing' ; a=nothing ; test-my-function '$v $v' ; echo "shell says a=$a"
my-function $v $v
a=$v $v
test says a=$v $v
shell says a=nothing

Second iteration

Hint: this one is not reliable.
Let's create a function to invoke the function inline. But that's not going to work for the same reason, so let's invoke something that looks like a function. How about this? $inline my-function args

The printf '\x23' is an easy way to avoid # taking effect in the wrong level of the eval context.

declare -- inline="eval eval \"\$_inline\" <<<"
declare -- _inline="source <( read && printf '\x23' && declare -f \$REPLY )"

Example

my-function()
{
echo "my-function $*";
echo a=$1;
a=$1
}
test-my-function() {
local a
$inline my-function args "$@"
echo "test says a=$a"
}
$ a=nothing ; test-my-function something ; echo "shell says a=$a"
my-function something
a=something
test says a=something
shell says a=nothing

Note the double-eval mechanism so that <<< can be used to read the function name on stdin while the remaining arguments are applied to the body as $@ for the lifetime of the body (thanks, bash)

Bugs

The second iteration surprisingly appears to work fine despite the double eval:
$ v='$thing' ; a=nothing ; test-my-function '$v $v' ; echo "shell says a=$a"
my-function $v $v
a=$v $v
test says a=$v $v
shell says a=nothing

but bafflingly, if thing is defined then suddenly a double evaluation is exposed:
$ thing=xxx; v='$thing' ; a=nothing ; test-my-function '$v $v' ; echo "shell says a=$a"
my-function xxx xxx
a=xxx
test says a=xxx
shell says a=nothing
how was '$v' preserved when thing wasn't defined?

Third Iteration

Hint: this one is not reliable.
It's a shame that a fork has to be incurred each time for the command substitution to generate the code to be sourced from the function body.

Maybe, rather than $inline my-function ... we could do $my-function except that variable naming has stricter rules than function naming.

But still, that's a minor inconvenience

We can extract the function body using the current technique and assign that to a variable to be sourced, and declare another variable as syntactic sugar to source it.

It gets a little more awkward trying to find a file descriptor or device name from which to source the function without blatantly destroying stdin (which to be fair, we may be doing in the other examples -- it needs testing), so skipping that for now:

make-macro() {
declare -g "_$1" "$1"
{ read -r ; IFS="" read -d '' -r _$1 ; } < <( declare -f "$2" )
printf -v "$1" 'eval source /dev/stdin <<<"$_my_function"'
}

Example

my-function()
{
echo "my-function $*";
echo a=$1;
a=$1
}
test-my-function() {
local a
$my_function "$@"
echo "test says a=$a"
}

$ make-macro my_function my-function
$ a=nothing ; test-my-function something ; echo "shell says a=$a"
my-function something
a=something
tast says a=something
shell says a=nothing

Bugs

Third iteration has arguments subject to one extra evaluation:
$ v='$thing' ; a=nothing ; test-my-function '$v $v' ; echo "shell says a=$a"
my-function $thing $thing
a=$thing
test says a=$thing
shell says a=nothing

To prevent argument evaluation, source must be invoked first, and any eval done within what is sourced.

Bugs

We aren't being careful to make sure we don't destroy stdin. Maybe the function needs stdin. 

Writing hygenic macros is hard and using eval right is harder. The arguments are subject to a varying amount of eval evaluation.

eval had been required so that <<< could be used to paste a here-string and there is no way around that using source from a string with a command defined in a variable, as <<< is not part of a command list

we could work around this by sourcing from an actual file instead of a here doc.

Fourth Iteration

Hint: this one is reliable.
We can avoid here-strings and here-docs and the associated redirections without the need for a file to source, by declaring a temporary file as a here doc, like this:

exec {_source_}<<<'source /dev/stdin <<<"${!1}" "${@:2}"'

Now we can refer to that here-string as /dev/fd/${_source_} and read it as often as we like
source /dev/fd/${_source_} f '$v $v'
my-function $v $v
a=$v $v

So the fuller solution based on the third iteration is
make-macro() {
declare -g _macro_
test -z "$_macro_" && exec {_macro_}<<<'source /dev/stdin <<<"${!1}" "${@:2}"'
declare -g "_$1" "$1"
{ read -r ; IFS="" read -d '' -r _$1 ; } < <( declare -f "$2" )
printf -v "$1" 'source /dev/fd/%d %s' "${_macro_}" "_$1"
}

Example

my-function()
{
echo "my-function $*";
echo a=$1;
a=$1
}
test-my-function() {
local a
$my_function "$@"
echo "test says a=$a"
}

make-macro my_function my-function
thing=xxx ; v='$thing' ; a=nothing ; test-my-function '$v $v' ; echo "shell says a=$a"
my-function $v $v
a=$v $v
test says a=$v $v
shell says a=nothing

And that's enough nastyness for one day