Makefile mash

The dumbest thing so far seen in a makefile:

ECHO      = `which echo`

Apart from the apparent fact that the writer assumes that the external command which is on the path but echo might not be (hint: they don't want to use the built-in echo for whatever reason), the main error is that everywhere $(ECHO) is used it will be replaced with `which echo`

The author probably intends

 ECHO      := $(shell which echo)

which will define ECHO to be something like /bin/echo which is what they intended.

Note the := otherwise which echo would be evaluated each time $(ECHO) is used.

TalkTalk moble data configuration

Today I learned that TalkTalk mobile users can txt 40127 with: TALK and get the configuration sent back.

Why are setjmp /volatile hacks STILL needed?

We all know the old gotcha of setjmp (but I'm going to recount it anyway) which is that when longjmp is executed (and setjmp appears to complete again, but this time with a non-zero result) some (or all) of the original register set will be restored.

Any local variables which were (possibly temporarily) stored in the local registers reserved for such use could appear to revert back to their previous values until they are re-freshed from the stack frame.

And so all pre-existing local variables that would be accessed after the longjmp should be declared volatile so that they will not be fetched from the registers.

Well... why is there no special #pragma or other option attached to the setjmp function indicating a register clobber-list to the compiler that all the local registers could have been modified and it should not depend on cached values any more?

Why do we need such a pragma? Why can't we #define setjmp to be something like:

#define setjmp(env) (setjmp(env) + __asm__ volatile ("mov %%eax,0" : : : "memory" "ebx" "ecx" "etc..." ));

I'm sure that example is insufficient, but I also think that some working method could be contrived, so I'll work on it...

and then...

OK, having tried; I realise that first problems are

• not stopping the extra return from setjmp from using stale values in restored registers
• that the correct values from the registers will not have been copied back into the stack frame, but will somewhere be pushed on the stack by a called function where we can't retrieve them

But the worst problem is that the optimiser can remove operations that ought to have been noticeable:

#include 
#include 

int getint() {
  static int i=3;

  i+=2;

  return i;
}

int main(void) {
  static jmp_buf j;
  volatile int x;
  int y;
  x = getint();
  y = getint();
  if (setjmp(j) == 1) {
    asm volatile("nop" : : : "memory", "ecx", "edx");
    printf("%d %d\n", x, y);
    return 0;
  }
  y++;
  x++;
//  printf("%d %d\n", x, y); // Second Printf
  longjmp(j, 1);
}

Unless the second printf is uncommented, the line for y++ is not emitted even at optimisation -O1
The reason is that y is not used after that point, even though the longjmp (acting somewhat like a goto) might have the effect of returning to a point where the y is used.

If I replace the longjmp with a goto then y++ is emitted in the code.

The first lesson: Use volatile in the way everyone says you should; all variables to be accessed in the second return of setjmp should be volatile.

The second lession, some contrivance of:

blah: if (setjmp(j) == 1) {
...
  if (never_true()) goto blah; else longjmp(j, 1);

generates the right code too by recognizing the effect of the goto rather than the longjmp however this cannot be sensibly managed as there can easily be multiple setjmp in a function.

So I still search for some means to cause a block of code to commit all temporary registers back to the stack frame before calling another function, and a means to cause all temporary registers to be flushed on second return from setjmp (although this may be covered by the first case).

further reading

I read http://stackoverflow.com/a/7734313 that:

The returns_twice attribute tells the compiler that a function may return more than one time. The compiler will ensure that all registers are dead before calling such a function and will emit a warning about the variables that may be clobbered after the second return from the function. Examples of such functions are setjmp and vfork. The longjmp-like counterpart of such function, if any, might need to be marked with the noreturn attribute.

So in fact gcc at least does evict any registers before calling setjmp but that doesn't prevent local variables being cached later on and not comitted to the stack frame before longjmp is called.

So I really just need a way to mark that functions might call longjmp and that temporary registers should be evicted before calling; but I know deep down that this is rubbish as longjmp might be called even from a signal handler.

Can all for-loops be transformed to while-loops?

I previously wrote on using for as a brace-less scope using a trick by Jens Gustedt but I wanted to be able to propagate any break clause that might be used within that scope so that it would take effect in an enclosing loop or case statement.

I found a method that worked for gcc but which made use of it's compound statements.

#include 

main() {
  int a;
  for (a=1; a<=2; a++) {
    printf("Main context a=%d\n", a);

    for (int o = 0; o >=0; ({ if (o == 1) { printf("Detected break\n"); break; } }) )
      for (int i=0; !o && (o=1), i==0; o=-1, i=-1 ) { printf("Inner context\n"); break; }
  }
}

produces:

Main context a=1
Inner context
Detected break

which shows that the break statement in the inner-context was propagated to take effect in the top level loop, by means of the break statement in the compound statement of the second loop.

Thats nice, and it is the intended effect, but a for-loop of this form

for ( expression-1 ; expression-2 ; expression-3 ) statement;

is meant to be equivalent to this while loop:

expression-1 ;
while ( expression-2) {
  statement
  expression-3 ;
}

In my case, expression-3 consisted of ({ if (o == 1) { printf("Detected break\n"); break; } }) and the break clause took affect in the containing scope - no doubt because it was not part of the statement of it's associated for-loop.

But it would transform into this while-loop:

expression-1 ;
while ( expression-2) {
  statement
  printf("Detected break\n"); break ;
}

Can there be any doubt that in this while-loop, the break statement would terminate the loop itself and not any containing scope? I don't think so.

Therefore gcc compound statements in for loops open the door to high-class trickery which cannot be achieved the normal way.

using sed to split a stream into 2 streams

An expensive file listing operation needs to invoke an action on the listed files.

xargs is normally the candidate for that, but what when there are multiple file types with varied actions?

Normally I would pipe into a bash scriptlet like this

... | while read "$file" ; do if [ $(expr "$file" :  "$pattern" ) = "0" ] ; then ... ; else ...

but it lacks the bulk appeal of xargs which can reduce the number of command invocations by thousands of times for a large file list.

So here I make use of sed, and bash's >( ... ) construct to open a subshell and substitute a magic filename that refers a file descriptor that writes to the input of the subshell. (The substituted filename is typically something like /dev/fd/63). The newline can be entered on a terminal session with ^V ^J. It is also essential that there are no spaces between the ' and >( and also between the ) and ', otherwise the sed script will be presented to sed as multiple arguments instead of one argument.

... | sed -e '/\.ko$/{w'>( xargs strip --strip-debug )'
;d}' | xargs strip

This allows kernel objects to be stripped of debug only but other objects to be stripped entirely.

An alternative would be to use tee and a separate grep

... | tee >( grep '\.ko$' | xargs strip --strip-debug ) | grep -v '\.ko$' | xargs strip

Installing ESXi Server with PXE and a serial console

I wanted to install ESXi Server 5.1 on a machine on the other side of the word. I have remote power control, and serial console and access to it's DHCP server and another server on that network, so I thought I could manage it.

I read many half-baked, typo-ridden and downright misleading sets of instructions so I thought I'd write some half-baked, typo-ridden and downright misleading instructions which worked for me.

Generally follow the instructions on the vSphere 5 Doumentation Center with a dash of this but here is what I did:

Unpack the iso into a tftp subdirectory (vmw for me), and so my pxelinux config file is here - and note that I am using the mboot.c32 from the vmware iso but the menu.c32 from my pxelinux installation, otherwise mboot tries to fetch a file called -c.

DEFAULT menu.c32
MENU TITLE ESXi-5.x.x-XXXXXX-full Boot Menu
NOHALT 1
PROMPT 0
TIMEOUT 80
LABEL install
  KERNEL /vmw/mboot.c32
  APPEND -c /vmw/boot.cfg gdbPort=none logPort=none tty2Port=com1
  ipappend 2
  MENU LABEL ESXi-5.x.x-XXXXXX-full ^Installer

LABEL hddboot
 LOCALBOOT 0x80
 MENU LABEL ^Boot from local disk

edit boot.cfg (unpacked from the iso), note that the leading / from all the filenames has been removed, a prefix has been added and some nonsense about comports added to kernelopt

bootstate=0

title=Loading ESXi installer

prefix=vmw

kernel=tboot.b00

kernelopt=runweasel com1_Port=0x3f8 tty2Port=com1 gdbPort=none logPort=none 

modules=b.b00 --- useropts.gz --- k.b00 --- chardevs.b00 --- a.b00 --- user.b00 --- s.v00 --- ata_pata.v00 --- ata_pata.v01 --- ata_pata.v02 --- ata_pata.v03 --- ata_pata.v04 --- ata_pata.v05 --- ata_pata.v06 --- ata_pata.v07 --- block_cc.v00 --- ehci_ehc.v00 --- weaselin.t00 --- esx_dvfi.v00 --- xlibs.v00 --- ima_qla4.v00 --- ipmi_ipm.v00 --- ipmi_ipm.v01 --- ipmi_ipm.v02 --- misc_cni.v00 --- misc_dri.v00 --- net_be2n.v00 --- net_bnx2.v00 --- net_bnx2.v01 --- net_cnic.v00 --- net_e100.v00 --- net_e100.v01 --- net_enic.v00 --- net_forc.v00 --- net_igb.v00 --- net_ixgb.v00 --- net_nx_n.v00 --- net_r816.v00 --- net_r816.v01 --- net_s2io.v00 --- net_sky2.v00 --- net_tg3.v00 --- net_vmxn.v00 --- ohci_usb.v00 --- sata_ahc.v00 --- sata_ata.v00 --- sata_sat.v00 --- sata_sat.v01 --- sata_sat.v02 --- sata_sat.v03 --- sata_sat.v04 --- scsi_aac.v00 --- scsi_adp.v00 --- scsi_aic.v00 --- scsi_bnx.v00 --- scsi_fni.v00 --- scsi_hps.v00 --- scsi_ips.v00 --- scsi_lpf.v00 --- scsi_meg.v00 --- scsi_meg.v01 --- scsi_meg.v02 --- scsi_mpt.v00 --- scsi_mpt.v01 --- scsi_mpt.v02 --- scsi_qla.v00 --- scsi_qla.v01 --- scsi_rst.v00 --- uhci_usb.v00 --- tools.t00 --- xorg.v00 --- imgdb.tgz --- imgpayld.tgz

build=

updated=0

It only remained to de-program F11 as a magic key for gnome-terminal as that key was needed to accept the license.

Bocelli - The Lords Prayer

This is another great song, good for headaches too.

My Song in the Night - More headache cures

More headache cures.

This is done a few times by different folks, but I like this the best because it shows all the players, after the race is done, together while the song is sung
remembering

https://www.youtube.com/watch?v=vKKDAQEk8HA

The nearby stars

Here is my answer to W.H. Auden's poem on stars; this time it is less a satire and more a re-expression of what the poem did to me.

The nearby stars

I care not for stars,
and they care not for me:
We see each-other distantly

But stars are also far apart
And well accustomed to their view
But you are near, and I love you.

The original is:

“Looking up at the stars, I know quite well
That, for all they care, I can go to hell,
But on earth indifference is the least
We have to dread from man or beast.

How should we like it were stars to burn
With a passion for us we could not return?
If equal affection cannot be,
Let the more loving one be me.

Admirer as I think I am
Of stars that do not give a damn,
I cannot, now I see them, say
I missed one terribly all day.

Were all stars to disappear or die,
I should learn to look at an empty sky
And feel its total dark sublime,
Though this might take me a little time.”


― W.H. Auden

gnuplot rolling average

Plot rolling average in gnuplot

After reading this example which provided a fixed 5-sample rolling average a put together this which will provide a variable size rolling average and which can be re-used for multiple traces.

The sprintf whose pattern is conditional on (int($0)==0) will reset avg_data when a new plot is started. I suppose this depends on the plot starting at the first row - perhaps a better technique could be found.

min(a,b) = a >= b ? b : a
samples(n) = min(int($0), n)
avg_data = ""

sum_n(data, n) = ( n <= 0 ? 0 : word(data, words(data) - n) + sum_n(data, n - 1))

avg(x, n) = ( avg_data = sprintf("%s %f", (int($0)==0)?"":avg_data, x), sum_n(avg_data, samples(n))/samples(n))

Then, a plot can be made like this which would plot a rolling average of the previous 5 values:

plot [0:*] "file.dat" using 0:(avg(column(3), 5))

Parking Charge Notice

A Parking Charge Notice arrived in the post last week for parking for more than two hours in an empty car park after the shops had closed, near Dewsbury Sports centre; charging £100, reduced to £60 if paid quickly enough that there is no time to think about it.

I thought about it.

First, there are a few flaws with the original notice:

1. It does not properly identify the car park; only a 1 line car park name with no mention of town or county.
2. It states that signage showing the terms and conditions is clearly displayed at the entrance - it was not, or it would have been noticed.
3. That by parking, the driver has agreed to be bound by the terms and conditions - not so! Not being aware of the sign, one could not know that parking there would bind one by the terms and conditions. (Even if one was aware it is still possible to park there without agreeing to the terms and conditions though this might constitute trespass instead of breach of contract).

And then the usual misleading characteristics of such threatening demands, the letter is so designed with the sole narrative of having the reader pay some money now, because the letter tells them to.

The letter states that as the keeper one should identify the driver, and comments here http://www.honestjohn.co.uk/faq/private-parking-penalties/ suggest that one must identify the driver.

The British Parking Association in 2011 lobbied Parliament for a clause in the Protection of Freedoms Bill (Chapter 2, Clause 56) to make vehicle keepers liable for private parking penalties in the same way they can be held liable for road traffic offences and street parking offences if they do not identify the driver of the car at the time of the offence

and

Two comments from Nev Metson who knows more about this issue than anyone else:

...

The Achilles heel in this legislation for the private parking company is that once the registered keeper has named the driver then the registered keeper can no longer be held liable, irrespective of whether or not the private parking company gets it's money or not from the driver.

but their letter does not state how to do this. of course not, they would rather that one just paid.

One might think that you could notify them of the driver by writing to the registered office address in small print at the bottom of page 2. The only large and obvious contact details are for paying. I did this by registered post, thus discharging the only legal liability as I understand it.

And in true cheapskate fashion it asks the reader to do their dirty work for them, stating

If you were not the driver at the time, you should tell us the name and current postal address of the driver and pass this notice to them. 

Well, actually, no, I don't think I have to do your delivery work for you, you can send a notice to the driver yourself. (Which they then did).

The appeals procedure is interesting, but to follow such a procedure would imply that one agreed with the original assertion that one had entered into a contract with them. They can have an appeals procedure if they like, but I don't need to use it.

£100

The size of the claim is interesting.  At http://www.honestjohn.co.uk/faq/private-parking-penalties/ we read:

Unfair Private Parking Penalties

(With thanks to Alan Salsbury)

Ticketed car parking in a car park or on private land is a contract between the car driver and the owner of the land or the company managing paring on the land. If an unfair penalty for parking in an off road car park is imposed by a parking enforcement company due to a breach, often due to circumstances beyond the car owner's control, there is a remedy. (Thinking in particular of the driver who, having purchased a train station parking ticket for two days, arrived back at the car 11 minutes late.)

The law covers this situation adequately with the Unfair Contract Terms Act 1977 which clearly provides under group 5 para 1(e)** that "Terms may be unfair if they have the object or effect requiring any consumer who fails to fulfil his obligation to pay a disproportionate high sum in compensation". In other words, the company owning or managing the parking space can only charge a penalty which accurately reflects the loss of income they suffered arising from that breach. Insofar as the driver arrived 11 minutes late the penalty, if any, should be the cost of an 11 minute ticket. If there were several other parking spaces available, then no loss was suffered.

"I have quoted this act several times to such organisations and in each case they have cancelled the ticket, albeit without prejudice to future decisions to avoid creating a precedent. It has worked every time. However, anybody thinking of using this law to avoid buying a ticket. beware. You have to purchase a ticket to establish a contract."

(** Amended by Schedule 2 under regulation 5 (5) of the Unfair Terms in Consumer Contracts Regulations 1999.) 

So what was the actual loss by my parking in an almost empty car park after the shops were closed? Nothing!

I also learn that although the car park is managed by ParkingEye, I do not need to deal with ParkingEye I can deal with the land owner (in the same way that I can deal with BBC instead of TV Licensing).

The driver will be identified (to ParkingEye) and after that I will have fun drawing the other points to them one at a time. I will leave the legal capacity of the actual driver in being able to enter into a contract till the very end. I will post updates on my blog.

A repeat demand addressed the actual driver was received today. The driver has no business relationship with ParkingEye and has elected to ignore the notice.

Using for as a brace-less scope

I commented previously on Jens Gustedt's trick of using a for construct as a brace-less scope in which additional variables may be introduced, but the disadvantage is that any enclosed break statement will apply to the abused for statements instead of a containing explicit for statement.

So I attempted to detect a the operation of a break statement in order to propagate it to an outer loop.

In the code below, highlighted lines would be inserted by a macro, the purposes is to have the inner break be propagated to the outer loop.

#include <stdio.h>

int main() {
  int a=0;
  for(a=0; a<2; a++) {
    for (int o=0; o>=0; (o==1) && break; )
      for (int i=0; !o && (o=1), i==0; o=-1, i=-1 ) {
        printf("In loop %d o=%d i=%d\n",a, o, i);
        break;
      }
  }
  printf("Finished with %d\n", a);
}

Sadly, (o==1) && break is not legal.

This works, ({ if (o==1) break; }) but sadly only for gcc.

Complete Computations in the Compiler

Interesting archived post here; A C++ program to test for the primality of the number 13

http://web.archive.org/web/20111113164723/http://homepage.mac.com/sigfpe/Computing/peano.html

The work is done in the compiler, so although that might take some time, the program will be very quick to complete!

Yet more try/finally/except

While using a for statement to declare a variable for the duration of a brace-less scope like this (requires gcc's c99 mode):

for(jmp_buf active_jmp_buf; ...; ...)

I needed a way to make sure that the loop only executed once, and really does execute once. This requires a variable that I can set after the first iteration. Where should such a variable come from? Naturally it would be declared in the same way.

for(jmp_buf active_jmp_buf, int cont=1; cont; cont=0)

However I can't make a for loop declare variables of different types, so I need 2 for loops (but they can use the variable of the outer loop).

for(int cont=1; cont; cont=0)
  for(jmp_buf active_jmp_buf; cont; cont=0) ...

And that will cause ... to be executed once, and give two scope local variables, cont which is used to terminate the loops after one iteration, and active_jump_buf which is the one we need.

So that gives:

for(int cont=1; cont; cont=0)
  for(jmp_buf active_jmp_buf; cont; cont=0)
    switch ((errno=setjmp(active_jmp_buf))) 
    if (0) {
      case 0: // first time around
           {
             ...
             ... throw(123);
             ...
           }
      break;
    } else {
      default: // break not needed for finally
           ...
    }

Yet another attempt at try/finally/except

Combined with setjmp/longjmp I previously tried to use a switch statement to hold the try/finally branches of code after noting the similarity between the C# except clauses and a switch/case statement.

I then rather too hastily abandoned this attempt under the impression that the curly braces are an essential part of the switch statement merely because that's how everyone shows them.

As I read further macro tricks I came across the blog of Jens Gusted, author of the unfortunately QT licensed P99 macro set which provides excellent capabilities, including try, catch and except in the syntax form I was seeking which does not involve trailing macros.

So I took a look at how Jens managed it and noticed a few tricks, including use of for (type var=...;...;...) as a way to start an indefinitely chainable single-statement scope that did not need a close-curly-brace but could declare a variable for that scope. But what most caught my eye was his use of the switch statement without curly braces.

It soon became apparent that the switch statement is just a statement, and that the case clauses are merely goto-label placeholders. Curly braces are normally used in switch to form multiple statements into a compound statement.

So if I can express my try and finally/except blocks without the need for me to provide curly braces, there will be no need for me to supply a close-brace.

An examples of brace-less switch statement:
http://stackoverflow.com/questions/8118009/is-there-a-useful-case-using-a-switch-statement-without-braces

switch (x)
    default:
    if (prime(x))
        case 2: case 3: case 5: case 7:
            process_prime(x);
    else
        case 4: case 6: case 8: case 9: case 10:
            process_composite(x);

and blessed day, as Johan Bezem shows further on that page, it allows you to start part-way through a loop, avoiding the continuation test on entry - that is useful!

uint8_t counter;
/* counter will get its value here somewhere */
switch (counter)
    default:
        while (0 < counter)
        {
            case 0:
                /* Perform action */
                counter--;
        }

Also, some people prefer switch(0) default: ... instead of do { ... } while(0) but Jens prefers if (1) { ... } else (void(0))

Try Again

So I repeat my switch based attempt, the parts in bold are provided by the macros:


for(jmp_buf active_jmp_buf; ...; ...)
  switch ((errno=setjmp(active_jmp_buf))) 
  if (0) {
  case 0: // first time around
       {
         ...
         ... throw(123);
         ...
       }
  break; } else default: // break not needed for finally
       {
         ...
       } 

Note that the if/else is bogus, merely a trick to hold two statements without using curly braces to denote a compound statement; although I took care to make it if (0) in order to get legitimate fall-through from the try branch into the finally branch.

I need to finish the for loop so that it only loops once, and perhaps include an extra single-case switch statement within each branch so that misplaced break directives will not affect our exception handling switch statement.

I also wonder if I can have multiple catch clauses like the C# except clauses or if I embed those within a single except clause with an explicit switch statement based on errno.

Another attempt at try/except/finally in C

Since previous work on C cleanup functions I've been trying various means to implement try/finally in C using macros that allow a natural in-code expression of a typical form, like:

try {
  ...
  ... throw(123);
  ...
} finally or except {
  ...
}

It was not possible to use cleanup functions to implement the finally block for many reasons including difficulty of syntax construction, but ultimately because a cleanup function must return normally and so cannot chain an exception:

 It is undefined what happens if cleanup_function does not return normally. http://gcc.gnu.org/onlinedocs/gcc/Variable-Attributes.html 

(Or can they? as I proceed with setjmp I think they can exit with longjmp but at the cost of missing out other cleanup handlers, so maybe I just contradicted myself there.)

I was reading about call-with-current-continuation for C and was re-reminded about setjmp/longjmp and wondered if these might work. (For GENSYM see my previous post).

jmp_buf GENSY(jmp_buf);
if((errno=setjmp(GENSYM(jmp_buf)))==0) {
  ...
  ... longjmp(????, 123);
  ...

} nothing or else { // nothing for finally, and else for except
...
}

the main advantage with setjmp/longjmp is the effective go-back-to consequence of longjmp which avoids the need to recall the names of any generated reference further on in the code. The main difficulty with throw is knowing the reference ???? to use in the longjmp. It should be the most recent active jump_buf --- but how to tell which it is?

We could have a local variable in the try scope which stores the address of the most recent jmp_buf, or in other words we don't need GENSYM macro after all so long as we start a new scope:

{ jmp_buf active_jmp_buf;
if ((errno=setjmp(active_jmp_buf))==0) {
  ...
  ... throw(123);
  ...
} } nothing or else {
  ...
}

which could be generated by these

#define try { jmp_buf active_jmp_buf; \
              if ((errno=setjmp(active_jmp_buf))==0)
#define finally }
#define except } else
#define throw(thrown) longjmp(active_jmp_buf, thrown)

Note how the exception number is smuggled into the exception handler through use of errno.

After looking at some exception handler examples for C# and seeing how much like a switch statement it appeared, I decided I wanted to be able to separate exception types with their own handler and realised that a case statement might do for this:

{ jmp_buf active_jmp_buf;
  switch (setjmp(active_jmp_buf)) {
  case 0:
       {
          ...
          ... throw(123);
          ...
       }
       ...
  case 123:
       {
         ...
       } 
  }
}

Which, though incomplete (consider the lack of break statements) might be generated by something like this:

#define try { jmp_buf active_jmp_buf; \
  switch (setjmp(active_jmp_buf)) { \
  case 0:
#define finally }
#define except(exception) case exception:
#define throw(thrown) longjmp(active_jmp_buf, thrown)

and so expressed like this:

try {
  ...
  ... throw(123);
  ...
} except(123) {
  ...
} except(default) {
  ...
}

Except that we don't have enough close-braces to close the switch statement or the scope containing the jmp_buf which now contains the entire switch statement. This is the sort of problem that plagued previous attempts based on cleanup functions.

With the last new keyword being the finally or the except, any special scopes must be constrained to the try keyword that finally or except can clean them up. Therefore the switch statement is off limits as it requires an extra close-brace, although Francesco Nidito was not too proud to introduce ETRY to solve this problem with his C exceptions (along with Duff's device).

Big heap o' stack frames

Why can't stack frames be allocated on the heap?

Of course I don't mean the heap, I want a stack frame to grow dynamically as much as anyone else, so a fresh memory map region might need allocating for each function cough; but of course dear reader you really want to know why.

Working with Samba multiple concurrent(ish) asynchronous network functions much work is done with continuations or callback functions which are invoked when forwarded network'd rpc responses are returned.

Co-routines

I had previously tried another way of working this by writing in synchronous style using libpcl, and spawning such functions as a private coroutine.

With every rpc-send function Samba has an rpc-receive function which will block in a nested event loop until the response packet is received (if the response has not already been received).

I modified the nested event loop code to switch back to the main co-routine. The packet-received callback routine then just has to switch back to my private co-routine and the rpc-receive function would then return with the now received packet.

It worked well enough but highlighted other bugs which came to light when concurrency became possible; such as use of a connection before it is properly established.

The reason co-routines were used here at all was to obtain a private stack as a simple way to preserve state when waiting for the response, as well as aid debugging - after all a linear stack trace is nicer than a state-machine state any day!

But if the main aim is to protect the stack from being torn down and destroyed, why not allocate the stack frame on a heap? This would allow a function to return to the caller (in some fashion) while preserving state so that mid-re-entry could be achieved at the callback. This is what C#'s yield return does and is perhaps along the lines of native C-ish support for call-with-current-continuation from Scheme (or Lisp). It would not be a full implementation of course unless a full copy of the application environment were made - but we don't want to re-invoke the return path anyway.

So can lightweight fibres be built into C, using such notation as:

do_something_slow(int a, int b) {

  int answer = callback(ask_question(a));

  printf("Answer %d\n", answer);

}

heap(do_something_slow(a, b));

printf("Task started\n");

where control will return to print "Task started" but sometime later the answer will be printed?

Of the two magic words heap and callback, the word heap signifies that a new non-returning stack frame would be allocated and the word callback signifies that the function ask_question will receive a callback address in some form which would cause execution to continue on the next line. This looks all very hand-wavy but of course the callback mechanism would have to be standardized, probably based on the event-loop mechanism by which the callbacks would be delivered.

This all corresponds very closely to my original case where a callback function was explicitly provided to the async function, and the sole role of the callback function was to switch execution to the saved stack. Having called the async function we then switch to the main stack and therefore the process which invoked heap.

I'll post my libpcl code soon, it has some custom varargs task launchers.

Come, friendly hell

I felt inspired to develop Betjeman's poem Slough after learning that over 55 millions babies had been killed since the end of the WWII. Did the Nazis just switch sides after all?

Come friendly hell and burn us now!
We're burning babies anyhow,
It's hardly worth you waiting now.
Swarm over, death

It's a work in progress, verse 2 addresses the institutions and their emblems of progress

Tricky

An answer to Dicky, by Robert Graves is Tricky, by Sam Liddicott

First the original poem, and then the answer.

DICKY by ROBERT GRAVES

Mother:
Oh, what a heavy sigh!
Dicky, are you ailing?

Dicky:
Even by this fireside, mother,
My heart is failing.

To-night, across the down,
   Whistling and jolly,
I sauntered out from town
   With my stick of holly.

Bounteous and cool from sea
   The wind was blowing,
Cloud shadows under the moon
   Coming and going.

I sang old roaring songs,
   Ran and leaped quick,
And turned home by St. Swithin's
   Twirling my stick.

And there as I was passing
   The churchyard gate,
An old man stopped me, 'Dicky,
   You're walking late.'

I did not know the man,
   I grew afeared
At his lean, lolling jaw,
   His spreading beard,

His garments old and musty,
   Of antique cut,
His body very lean and bony,
   His eyes tight shut.

Oh, even to tell it now
   My courage ebbs...
His face was clay, mother,
   His beard, cobwebs.

In that long horrid pause
   'Good-night,' he said,
Entered and clicked the gate,
   'Each to his bed.'

Mother:
Do not sigh or fear, Dicky.
   How is it right
To grudge the dead their ghostly dark
   And wan moonlight?

We have the glorious sun,
   Lamp and fireside.
Grudge not the dead their moon-beams
   When abroad they ride.

In my poem, I wished to tell the other side of a perhaps less fearful story.
Thinks aren't always what they seem, and sometimes, not even that.

Tricky, by Sam Liddicott

Uncle:
Oh, what a merry sound!
Ricky, are you chuckling?

Ricky:
This evening by this fire,
Uncle, My heart is laughing.

To-night, across the down,
   Whistling and jolly,
From town young Dicky sauntered
   With a stick of holly.

Bounteous and cool from sea
   The wind was blowing,
Cloud shadows under the moon
   Coming and going.

He sang old roaring songs,
   Ran, and leaping quick,
He turned home by St. Swithin's
   Twirling his stick.

And there as he was passing
   The churchyard gate,
Greeting him, I said: 'Dicky,
   You're walking late.'

I think he did not know me,
   He looked afeared
And my jaw, open fell,
   My theatre beard!

This outfit old and musty,
   Of antique cut!
I rocked in silent laugher,
   My eyes tight shut.

Oh, even to tell it now
   My humour rises...
His face was petrified,
   by costume trousers!

After an awkward pause
   'Good-night,' I said,
Passing through the churchyard gate,
   'Each to his bed.'

Uncle:
Do not laugh at fear, Ricky.
   How is it right
To terrify a little lad
   in the moonlight?

When by the glorious sun,
   Lamp and fireside.
 He knows you as a friend; so:
   Do not deride.

A long time ago I helped form the Open Rights Group.

Forgiveness

I post this to point out that forgiveness isn't something available on demand, and also the dangers of letting the perpetrator define what forgiveness does and does not mean.

It can take the victim longer to work out forgiveness than it took the culprit to commit the offences and finally achieve a change of heart. This is something a repentant perpetrator may or may not be able to comprehend.

For example I might finally forgive my accountant for ripping me off and dashing my dreams and losing my house, but I wouldn't give him the chance to do it again, and I don't expect to be able stop suffering the injury and seeking revenge and punishment at his demand and admission. It would take time and effort and more time and more effort.

In other words I can eventually stop suffering from the injury, and I can eventually stop seeking revenge and punishment. But I don't have to give him access to my bank account in order to prove to him that I have reached or am trying to reach that state.

If someone thinks I haven't forgiven them, then that is their problem whether or not they are right. They just might have to forgive me.

Only the innocent victim knows what forgiveness entails, and only the repentant perpetrator wants it badly enough to wait for it.

And, for accuracy, there is one truly innocent victim who provides the means of healing and forgiveness for the injured and the guilty; when they want it badly enough to receive it.

bash signal handling

A signal trap will not be executed while bash is implicitly waiting for an external command to complete.

I mean:

#! /bin/bash

trap "echo signal" 10

( sleep 1 ; kill -s 10 $$ ) &

sleep 5

We would expect the word signal to be emitted after 1 second, but the signal handler will not run until the sleep 5 is complete.

However, the built-in bash command wait does not have that problem, and so we can run the task in the background and immediately wait as in this example where the signal runs after 1 second.

#! /bin/bash

trap "echo signal" 10

( sleep 1 ; kill -s 10 $$ ) &

<&0 sleep 5 & wait

The <&0 is required to cover the case that the background command needs to read stdin which would otherwise not be connected for a background command.

The script also quits after 1 second even though the sleep is still running, so this variant detects if the background is still running and waits again:

#! /bin/bash

trap "echo signal" 10

( sleep 1 ; kill -s 10 $$ ) &

sleep 5 & wait

kill -s 0 $! &>- && wait

Of course this is insufficient for 2 reasons; the extra wait should be in a loop, but worse, the exit code is lost.

Poem: The smelly dog-pooh man

I wrote this in early 2008 for the Agbrigg & Belle Vue Community News

The smelly dog-pooh man

© 2008 Sam Liddicott

I'm the smelly dog-pooh man,
I walk around the town,
and leave my smelly dog pooh
where my doggie puts it down.

Not in gutters or in hedges
where no foot will hardly tread,
but the middle of the pavement
there to cake your shoes with dread.

Where the young ones and the old ones
and so many in between,
will spread about my pavement paste
from clumps they hadn't seen.

It's rather hard to miss something
so small and hard to see,
It's also hard to miss it
when its spread so wide and free

So the pushchairs and the children
do a dance around the mess,
that's been spread in pretty patterns –
now the cause of their distress

No-one sees me do it,
though you may observe my work,
or attempts by my apprentices
to spread a little dirt,

and I get a secret pleasure; when–
from behind a mother's door,
I hear the sound of pavement paste
discovered on the floor.

It's not that I'm too lazy
although that's also true,
but I like to punish others
with the things I hate to do.

So I'll leave it on the pavement
and pretend it isn't mine,
Till vengeful neighbours tell on me
and make me pay the fine.

gensym and unwind-protect for plain C

I wanted some simple C (I mean gcc's C, of course) version of unwind-protect, a clean-up block that was guaranteed to execute when scope for function finished. And a nice way to express it.

How does this look?

void do_test3() {
  int y=3;
  printf("hello\n");
  scope_cleanup {
    printf("cleaning up %d\n", y);
  }
  { int z=2;
    scope_cleanup { printf("inner cleanup %d\n", z); }
  }
  printf("bye\n");
}

main() {
  do_test3();
}

And when run it emits:

hello
inner cleanup 2
bye
cleaning up 3

Well darn tootin'! And you can have more than one in a function or scope too, without conflict!

How 'tis done

It's only half the story, but of course the heavy lifting is done with gcc's non-standard __attribute((cleanup(...))) which can be appended to any variable definition causing the function whose name is at ... to be called (with a pointer to the variable) when the variable goes out of scope.

Anyone who has used this feature will know how useful (though cumbersome) it is; e.g.

void free_ptr(void* ptr) {
  free(*(void**)ptr);
}
...
void* ptr=malloc(MAX_BUFFER) __attribute((cleanup(free_ptr)));

or

void close_ptr(void* ptr) {
  close(*(int*)ptr);
}
...
int fd =open(path, O_RDONY) __attribute((cleanup(close_ptr)));

But what I really want is to be able to specify is not a function but a clean-up block of code, and not tied to any particular variable, but which is executed when the scope end - like unwind-protect and such,

Something like that can be implemented using this method, but I was looking up nested function definitions in C today (I don't remember why) and I noticed that nested functions can be pre-declared using auto which made it possible to define a macro that would expand to this:

auto void cleanup (void* x);
int cleanup_var __attribute((cleanup(cleanup)));
void cleanup(void* x) ...

Of course the ... isn't part of the definition, it is the point at which a block can be expressed; which will be incorporated into the body of a nested function hastily called cleanup in this example.

Without pre-declaring the cleanup method, the cleanup method must be defined before the variable is declared, making it ugly having to pass the code block as a macro parameter.

gensym

Clearly it is easily possible to over-use the variable name cleanup_var and the function name cleanup simply by doing this more than once per function; what is needed is a way to generate unique symbol names in the C preprocessor.

It isn't possible, sadly (not really! got you there!) because the pre-processor cannot manage counters. But, (thinks I) we all know that a macro expands out to a single line and so one pre-processor symbol that always changes but remains the same for an expansion is __LINE__ (and "that be so" says you)
* see also __COUNTER__

So we have these definitions for gensym

#define GENSYM_CONCAT(name, salt) name ## salt
#define GENSYM2(name, salt) GENSYM_CONCAT(name, salt)
#define GENSYM(name) GENSYM2(name, __LINE__)

And this definition for cleanup (for those that couldn't work it out)

#define scope_cleanup auto void GENSYM(__cleanup__) (void* x); \
int GENSYM(__cleanup_var__) __attribute((cleanup(GENSYM(__cleanup__)))); \
void GENSYM(__cleanup__)(void* x)

of course, I am rather abusing the meaning of gensym by having it return the same name each time... the proper way would be to call gensym once and pass it's result to another macro but that would increase the risk of shadowing existing definitions, for which reason we are using gensym in the first place.

The original example expands out something like:

void do_test3() {
  int y=3;
  printf("hello\n");
  auto void __cleanup_29 (void* x); int __cleanup_var_33 __attribute((cleanup(__cleanup_29))); void __cleanup_29(void* x) {
    printf("cleaning up %d\n", y);
  }
  { int z=2;
    auto void __cleanup_33 (void* x); int __cleanup_var_33 __attribute((cleanup(__cleanup_33))); void __cleanup_33(void* x) { printf("inner cleanup %d\n", z); }
  }
  printf("bye\n");
}

A word on Technicalities

I read a comment from a chap who didn't care at all about technicalities but just thought people should pay their share of tax.

He didn't understand that it is the technicalities that make it possible to know what one's share of tax is so that one can pay it.

I think he felt like a benevolent dictator who could whip people into paying their share, but didn't realise the chaos that would ensue when the dictator next to him did the same thing but with a slightly different idea of what the share should be.

It was a sad case of the dangerous liberal dictators which goes like this: "If only everyone would do it my way it would all be so much better and we'd all get along so nicely."

It also reminds me of the more rabid US anti-gun folk who steam away thinking: If only we could MAKE them give up their guns, the country would be more safe and more peaceful; but without realising that they would have to MAKE them like it too, which is no more possible than making the anti-gun chaps like guns.

In other words: We only have these hard conversations because agreement is hard. Yet another attempt to sweep away disagreement isn't going to work.

Technicalities bring peace and order to confusion and disagreement without the need for full agreement.

It is sometimes easier to agree on technicalities than on principle; and that is what politics is, although in politics often the principle is not so much at stake as is who will get the bigger slice of pie.

Re-factoring bash

Sometimes one has to perform maintenance on someone else's bash scripts. At such times one keeps one's mouth shut and gets on with it.

Sometimes one gets to re-write them to add new features, and some re-factoring brings great relief.

See this function wrapper to run a command more than once in case it fails the first time. (Don't get distracted on thoughts of the validity of this approach).

do_cmd()
{
  retval=0
  for i in `seq 1 2`;
  do
    cmd 2>/dev/null $@
    retval=$?
    if [ $retval -eq 0 ];
    then
         break
    fi

    sleep 1
  done

  return ${retval}
}

Can we express that function any more concisely or correctly?

The first thing that comes to mind is local retval=0

...but that is closely followed by the wonder of `seq 1 2` which emits 1 2 and so that line could become:

for i in 1 2

At this point it becomes very obvious that the function will execute cmd up to twice, with a 1 second delay if it fails the first time. Any bash coder given that description could come up with a 1 liner; but let's approach it by degrees.

This chunk:

cmd 2>/dev/null $@
    retval=$?
    if [ $retval -eq 0 ];
    then
         break
    fi

becomes:

cmd 2>/dev/null $@
    retval=$?
    test $retval -eq 0 && break

but we don't need to remember retval if we return right away, so this might become:

cmd 2>/dev/null $@ && return
    retval=$?

(we still remember retval so that we can return the last failure code if the loop exits.

But who needs a loop for 2 iterations? How about this:

do_cmd()
{
  cmd 2>/dev/null $@ && return
  sleep 1
  cmd 2>/dev/null $@
} 

Which is pretty clear. But compare that to a fresh re-write based on the original description above:

do_cmd()
{
  cmd "$@" || sleep 1 && cmd "$@"
} 2>/dev/null

I also added in the missing quotes around $@ and added the stderr redirector to the function body definition.

The function will exit with either 0 because the first invocation succeeds, or it will return with the exit code of the second invocation 1 second later - presuming sleep doesn't fail.

If we are afraid that sleep might fail, we could make the second invocation not dependant on the success of the sleep command:

do_cmd()
{
  cmd "$@" || { sleep 1 ; cmd "$@" ; }
} 2>/dev/null

Of course this doesn't quite match the original function because any stderr resulting from a failure in executing the sleep command would also be hidden.

Fix Charging on Acer ZG5

My Acer One ZG5 was not charging for the last year or so. I took it for repair to my local friendly and trusted Wizard Electronics but the chap there was too wise. Armed with the knowledge that it failed to charge two different batteries but yet would boot up under charger power he could conclude quite well that it was not the charger, the connector or the batteries and some troubleshooting on the system board was required. I already knew this of course which was why I took it to him; but he suggested it would be cheaper to replace the main board and a quick look at ebay prices for smashed screen netbooks showed that he was right.

However, to my surprise, while googling to find what the motherboard fault might be I found a clue that it might be (and was!) fixed with a BIOS update!  http://www.acer.com/worldwide/support/download.htm and one version 3110 release later and I was charging both batteries again!

Acer Aspire SD slots on bootup

My jmicron SD card slots not being recognized by Linux Mint unless a card was in the slot when booted. And there were two slots.

early hacks

This post http://ubuntuforums.org/showpost.php?p=10842957&postcount=62 shows a way that works for me to re-scan and detect slots that now have a card in (although the card may need to be removed and re-inserted).

sudo echo 1 > /sys/bus/pci/rescan


Tricks here http://ubuntuforums.org/showpost.php?p=10842957&postcount=62 that use setpci don't work for me, failing to select the device if it isn't already detected, or the pciehp driver which doesn't seem to be present on my installation.

Success

I got auto-detection to work by following these tips https://bugs.launchpad.net/ubuntu/+source/linux/+bug/258446/comments/61 by adding the following files to /etc/modules

pciehp
acpiphp
jmb38x_ms
sdhci-pci
flash_bd
xd_card
jmb38x_xd


rebooting, and running: update-initramfs -u so that those modules would be built in to the ramdisk; and then rebooting again.

Nothing else was required; no use of pciehp, no edit of /etc/default/grub.conf and no creation of files in /etc/modprobe.d/