Wednesday, July 22, 2015

Convert pdf to monochrome

From StackExchange

gs \
 -sOutputFile="BW-$1" \
 -sDEVICE=pdfwrite \
 -sColorConversionStrategy=Gray \
 -dProcessColorModel=/DeviceGray \
 -dCompatibilityLevel=1.4 \
 -dBATCH "$1"

Writes sample.pdf to BW-sample.pdf

Tuesday, July 7, 2015

Marriage of alloca and asprintf

It all comes down to not being able to return variable length arrays in C.

I think that asprintf is the way sprintf should work, but not everyone has it, and so folk either cart around an asprintf implementation, or enjoy getting the tedium of snprintf(NULL, 0, ...) or the worse error-prone tedium of adding up buffer sizes.

A single line  asprintf and corresponding  free() make a much simpler 1-2 lines of use.

But while I'm not being portable it would be nice to be able to have asprintf use alloca as it's allocator...

alloca returns space from the current function stack frame (not scope) and is automatically freed when the function returns. But if alloca can't allocate enough memory, then who knows what might happen.

So the plan is to alloca a buffer of the right size, and then sprintf into that.

This 1 line macro has the same syntax as the official asprintf.

#define aasprintf(PTR, ...) sprintf( (*(PTR)) = alloca(snprintf(NULL, 0, __VA_ARGS__)), __VA_ARGS__)

and must be used with two lines:

char* x;
aasprintf(&x, "a %s b %d c\n", "<>", -1);

But I'd prefer a single line use:

char* x = aasprintf("a %s b %d c\n", "<>", -1);

So the plan is to alloca a buffer of the right size, and the right size is measured with snprintf(NULL, 0, ...). So we define a helpful macro:

#define measure_printf(...) snprintf(NULL, 0, __VA_ARGS__)

Once the size is measured, we can allocated with alloca and call sprintf(alloc, ...) to populate our alloc'd buffer. usual warning that macro arguments might get evaluated twice, but I don't see any other way around that without using a function call to stash the values, and require the heap or static buffers to hold them.

#define aasprintf(...) sprintf(alloca(measure_printf(__VA_ARGS__)), __VA_ARGS__);

However, that will return whatever sprintf returns, which is the size, not the buffer. We need a temporary variable to hold the alloca pointer, and maybe return via a special GNU macro, like this:

#define aasprintf(...) ({ char* x=alloca(measure_printf(__VA_ARGS__)); sprintf(x, __VA_ARGS__); x })

but that also requires us to come up with a variable name that will never be one of the __VA_ARGS__. And relying on alloca is bad enough without insisting on GNU extensions.

Maybe a tuple, like this:

#define aasprintf(...) ( char* x=alloca(measure_printf(__VA_ARGS__)), sprintf(x, __VA_ARGS__), x ) 

It's not a GNU extension but x still might clash with one of the __VA_ARGS__ and now we are declaring a variable in the middle of a scope.

It is simpler to define a wrapper to snprintf (or vsprintf ) that returns the buffer instead of the size. (Although perhaps a simple assembler push/pop would have saved the value nicely).

static char* a_sprintf(char* v, char* format, ...)
    if (! v) return NULL; // However alloca is undefined if it fails...
    va_list args;
    va_start(args, format);
    vsprintf(v, format, args);
    return v;

Which gives this winning combination which returns the string instead of the length (more useful):

#define measure_printf(...) snprintf(NULL, 0, __VA_ARGS__)
#define aasprintf(...) a_sprintf(alloca(measure_printf(__VA_ARGS__)), __VA_ARGS__);
/* sprintf wrapper that returns the buffer address. 
   Does not check size, intended to be used on a 
   properly allocated buffer as part of aasprintf */
char* a_sprintf(char* v, char* format, ...)
    if (! v) return NULL; // However alloca is undefined if it fails...
    va_list args;
    va_start(args, format);
    vsprintf(v, format, args);
    return v;

It doesn't warn you or fail sensibly if you try to print too much, but on the other hand it's 1 line to use and no need to worry about freeing.

Which is more likely?
With snprintf that you either:
  • get the snprintf buffer arithmetic wrong
  • or don't cope with allocating too much memory
  • or forget to free the memory
or with aasprintf
that you
  • allocates too much memory
There's certainly a lot less to get wrong, although admitrdly you can't get it all right either - but there's asprintf for that.

sprintf was designed to get wrong, and snprint was designed to also get the arithmetic wrong.

And for completeness, asprintf (not tested):

int asprintf(char **v, const char* format, ...)
    int len = 0;
    if (! v) return -1;
    va_list args;
    va_start(args, format);
    len = vsnprintf(NULL, 0, format, args);
    *v = malloc(len); 
    if (! *v) return -1;
    len = vsnprintf(*v, len, format, args);
    return len;

Monday, March 2, 2015

Packed binary in bash

Using bash, and other accessible commands, I wanted to output some based 64 encode packed binary string representations of 64 bit decimal numbers, in reverse byte order.

I used bc to convert the arbitrarily long decimal number into hex
bc <<< "obase=16; $N"
$ N=123456789
$ bc <<< "obase=16; $N"
I then split this into hex digit pairs to represent one byte each, in reverse order, and prefixing with \x to produce a printf format string.  I do this by appending a space to the line, and then move two (or one, if two not available) digits that precede the space to instead append to the end of the string, (with \x prefix). And then finally, remove the space which is now a leading space.
sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
e.g. the following pattern space is iterated
Another method of separation might have been to pad with 0 to an even length, and then split off pairs of digits from the front. I then use all of that as the argument to printf which interpolates the characters, and then pipe to base64.
printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64
$ N=123456789
$ printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64

Friday, February 13, 2015

Finding xterm Terminal Window Size for serial console resize

I use a networked serial console displaying in an xterm (TERM=xterm), and to my frustration it can't cope when I resize the xterm, instead either garbling the output or just using the original portion.

I don't know why this should be.
# shopt | grep checkwinsize
checkwinsize on

Various combinations of # kill -SIGWINCH $$ and # kill -s WINCH $$ had no effect at all, and # stty size proved to be 0 0, and thus not very useful.

With a little help from the source to xterm's resize command and also this python interpretation I came up with this 1-liner for bash, to read the xterm width and height into LINES and COLUMNS and set these in the tty driver ready for vi or other programs to pick up:

IFS=$';\x1B[' read -p $'\x1B7\x1B[r\x1B[999;999H\x1B[6n\x1B8' -d R -rst 1 _ _ LINES COLUMNS _ </dev/tty \
&& stty cols $COLUMNS rows $LINES

It's worth looking at how this works.

$'...' is a bash quoted string that allows character entities to be expressed in hexadecimal, thus $'\x1B' is the character named ESC.

The string emitted as a prompt instructs the xterm to save the current cursor position, move the cursor to 999,999 which should be beyond the terminal bounds and so instead move to the bottom right corner; it then instructs the xterm to report the current cursor position, and then restore the previous cursor position.

The current cursor position is returned as a string (without newline) in this form: \x1B[lines;columnsR and so this can be read into two variables LINES and COLUMNS that bash uses. But how to do that?

The bash read function will emit the prompt, and then read the response. As the response is terminated in R instead of a newline, -d R is passed to read.

Other useless characters are \x1B [and ; so we put these into IFS causing read to use these to split the input. This gives us 2 empty strings, which we read into the bash underscore variable which gets overwritten every line anyway, so our read variable specification is _ _ LINES COLUMNS _ causing LINES and COLUMNS to take the 3rd and 4th values, with a final _ to take any potential junk that otherwise would have been appended to COLUMNS. Raw mode and silent mode are advised obviously, hence -r -s and a timeout of 1 second is set in case an xterm isn't in use and there will be no response.

So why not define this as a function...

resize() {
  IFS=$';\x1B[' read -p $'\x1B7\x1B[r\x1B[999;999H\x1B[6n\x1B8' \
                     -d R -rst 1 _ _ LINES COLUMNS _ < /dev/tty &&
  stty cols $COLUMNS rows $LINES

and depending when/how you invoke it you may wish to not loose the previous value of $?, saved here without the use of local variables (which through dynamic scoping might affect the called function).

just() {
  set -- $? "$@"
  return $1

so now you can invoke: just resize without affecting $?, perhaps from something ghastly like:

export PS1="$PS1"'$(just resize)'

or as part of PROMPT_COMMAND although ideally it would run before executing a command.

It now remains to learn why (although a serial terminal cannot be expected to send a SIGWINCH) bash was not responding to SIGWINCH.

Tuesday, December 9, 2014

On conspiracy theories...

The (perhaps hasty) assumption that conspiracy theory deniers are not a spook-stooge becomes evidence in favour of such theories.

Un-initiated, and without conscious inducement they defend such organisations as may exist.

By induction this may apply to various degrees at various levels, each person doing what they think is right and natural, a little self serving here and there - but selected because their resultant actions are convenient.

If you want a deed doing, find someone who wants to do the deed and let them do what they want. No direct commuication or coercion required. Their visible well meaning is all the better a disguise.

The strength of any conspiracy theory is it's veracity which trumps all so-called weaknesses.

Veracity is naturally hard to measure, particularly against a weakness of the defensibility of a theory when mistaken for a weakness of the theory.

A weakness of the theory would relate to difficulty for the parties of the theory to act according to the theory.

A weakness of the defensibility of the theory would be an implausibility of the theory which depends largely on the experiences of the listener and less on the theory itself.

What works wins. Does there have to be a grand design, or will natural selection work? or is that the grand design? I note that some players win many games of chess despite the implausibility of long term prediction of the opponents moves.

Let the hungriest power mongers fight it out, and nudge to authority those who can be led by their desires. And then lead them. That's the world we live in, and it has typically been so.

I happen to believe that the one with the most power is good, known as God, who does not enforce but rather invites, who can deliver the captives of the mighty and the prey of the terrible. This does not absolve me of responsibility but rather realises it.

And this is why I like Christmas. The birth of the one who was not deceived, who followed the will of the Father and provides the means to escape for all who will.

Hows that for a Christmas sermon? I can't apologise - for the ideas are inseparable; we can't save ourselves.

I think that a typical defence against truth is to find a reason not to listen. We cannot bear the horror of the world we live in, and pretend it is just about starving orphans in war torn droughts, attacks on the liberty of good manners and lazy rich people who've taken all the money.

Wednesday, December 3, 2014

Don't be illl (sic)

The danger of lapsing into sickness.

We think one small affliction will do us no harm, and maybe even some good - but then we find it becomes habit forming and we can hardly manage to get well again.

Consider the virtue of procrastination. Why get sick today if you can be sick tomorrow - or even next week!

Even better, make it a new years resolution and put it off for years. If you can procrastinate sickness, then maybe try to put off that final exit from life!

You know it makes sense!

Friday, September 26, 2014

shell-shocked: Updaing bash on Linux Mint Olivia

shell-shocked: Updaing bash on Linux Mint Olivia

I've go some old laptops running Linux Mint, Olivia release, based on Ubuntu Raring Ringtail which is not a long term support, and so no ready fix is available.

The bug is shown with this test
rm -f echo ; env X='() { (a)=>\' bash -c "echo date"; cat echo
bash: X: line 1: syntax error near unexpected token `='
bash: X: line 1: `'
bash: error importing function definition for `X'
Fri Sep 26 09:37:44 BST 2014
which wrongly creates a file called echo holding the output of the date command.

Here is what is required to fix the bash shell-shocked bug.

1. Update the no-longer supported repositories to the new location:

(You should already have done this).

You first need to make sure you have src-deb repositories enabled:
cd /etc/apt/sources.list.d/ 
test -r official-source-repositories.list ||
cp official-package-repositories.list official-source-repositories.list &&
sed -i -e 's/raring/trusty/g;s/^deb /^deb-src/' official-source-repositories.list
And then update the server to old-releases:
sed -i -e 's/\|' official-package-repositories.list
apt-get update

2. Install all updates for your system anyway

apt-get update
apt-get upgrade

3. Build bash from the trusty sources

apt-get build-essential bash
apt-get -b source bash

4. Install bash

dpkg -i bash*deb

5. Test the fix

# rm -f echo ; env X='() { (a)=>\' bash -c "echo date"; cat echo
bash: X: line 1: syntax error near unexpected token `='
bash: X: line 1: `'
bash: error importing function definition for `X'
cat: echo: No such file or directory