Monday 2 March 2015

Packed binary in bash

Using bash, and other accessible commands, I wanted to output some based 64 encode packed binary string representations of 64 bit decimal numbers, in reverse byte order.

I used bc to convert the arbitrarily long decimal number into hex
bc <<< "obase=16; $N"
$ N=123456789
$ bc <<< "obase=16; $N"
I then split this into hex digit pairs to represent one byte each, in reverse order, and prefixing with \x to produce a printf format string.  I do this by appending a space to the line, and then move two (or one, if two not available) digits that precede the space to instead append to the end of the string, (with \x prefix). And then finally, remove the space which is now a leading space.
sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
e.g. the following pattern space is iterated
Another method of separation might have been to pad with 0 to an even length, and then split off pairs of digits from the front. I then use all of that as the argument to printf which interpolates the characters, and then pipe to base64.
printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64
$ N=123456789
$ printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64
An alternate method for fixed with fields that don't need byte order fixings:
Convert to hex using printf:
printf -v hex '%08x' $dec
printf '%b' "\x${hex:0:2}" "\x${hex:2:2}" "\x${hex:4:2}" "\x${hex:6:2}"
To convert a string of space separated hex to characters:
input=" $input"
printf -v input '%b' ${input// /\\x}