Filtering stderr

This helper function will take $1 as a simple command to be used on stderr, on the rest of the command. The filtered output is emitted on stderr.

e.g. stderr "sed -e s/^/tar: /" tar -xvzf -

The function is short, but obscure, making use of a few tricks

stderr() {
  { set -- $_ "$@" ; } {_}>&1

  { eval '"${@:3}"' "$1>&-" ; } 2>&1 >&${1} | eval '$2' ">&2" "$1>&-" 


  set -- $1 ${PIPESTATUS[0]} "${@:2}"
  eval "exec $1>&-"
  return $2
}

An explanation is here:

stderr() {
  { set -- $_ "$@" ; } {_}>&1

The first line uses the temporary variable _ (underscore), which cannot generally be relied upon, but is safe enough in this context. This variable is used to avoid this helper leaving any imprint. Trampling on variables or declaring any local variables could affect destroy transparency and potentially affect the rest of the script.

So _ becomes a copy of stdout; and then inside the { ... } we update the function arguments so that this copy of stdout is now argument 1.

The function arguments are the only lexically scoped variables in bash. We can set them here in this function knowing that they will not have any other affect anywhere else.

So $1 now refers to a copy of stdout, $2 is now the filter to be applied to stderr, and "${@:3}" ($3 and onwards) is the command to be filtered.

We want to run the command with the $1 copy of standard out closed, in case the command spawns other processes that might inherit this copy and leave it open. Its a private copy, and as bash doesn't support close-on-exec we must close it.

We want to do this: "${@:3}" $1>&- but bash can't take a parameter variable on the left hand side of a redirector (not even as ${!1}) so we must use eval. We put the command in single quotes to prevent it being interpolated at all prior to eval, but the redirector is in double quotes so that the interpolated string is passed to eval; thus: eval '"${@:3}"' "$1>&-"

We want to run this command with stdout passed to the spare copy we made in $1  because we will redirect stderr to stdout to be fed into the filter. This redirection specification is: 2>&1 >&${1} (variables are allowed on the right hand side of a redirector).

However we can't append these redirectors to the previous one which already closed $1, so we use a brace scope { ... ; }  in which $1 is closed.

This gives us so far: { eval '"${@:3}"' "$1>&-" ; } 2>&1 >&${1} which has stdout going to our copy of stdout, and stderr going to actual stdout ready to pipe to the next stage.

The next stage also wants to close $1 for the same reason as before, and is:

eval '$2' ">&2" "$1>&-"

So the whole invocation is:

{ eval '"${@:3}"' "$1>&-" ; } 2>&1 >&${1} | eval '$2' ">&2" "$1>&-"

We now want to close $1  for the rest of the script without losing the exit code. As we have finished calling other commands we could save $? in a local variable, but I use the function arguments again to save as $2. Note that PIPESTATUS[0] holds the result of the first stage of the pipeline.

  set -- $1 ${PIPESTATUS[0]} "${@:2}"
  eval "exec $1>&-"
  return $2
}

And there it is.