Thursday, 12 September 2013

using sed to split a stream into 2 streams

An expensive file listing operation needs to invoke an action on the listed files.

xargs is normally the candidate for that, but what when there are multiple file types with varied actions?

Normally I would pipe into a bash scriptlet like this

... | while read "$file" ; do if [ $(expr "$file" :  "$pattern" ) = "0" ] ; then ... ; else ...

but it lacks the bulk appeal of xargs which can reduce the number of command invocations by thousands of times for a large file list.

So here I make use of sed, and bash's >( ... ) construct to open a subshell and substitute a magic filename that refers a file descriptor that writes to the input of the subshell. (The substituted filename is typically something like /dev/fd/63). The newline can be entered on a terminal session with ^V ^J. It is also essential that there are no spaces between the ' and >( and also between the ) and ', otherwise the sed script will be presented to sed as multiple arguments instead of one argument.

... | sed -e '/\.ko$/{w'>( xargs strip --strip-debug )'
;d}' | xargs strip

This allows kernel objects to be stripped of debug only but other objects to be stripped entirely.

An alternative would be to use tee and a separate grep

... | tee >( grep '\.ko$' | xargs strip --strip-debug ) | grep -v '\.ko$' | xargs strip

1 comment:

  1. The compact example can be made to work without the need for ^V^J by inverting the match and using sed -n:

    ... | sed -ne '/\.ko$/!{p;d};w'>( xargs strip --strip-debug ) | xargs strip