On the beach

The earthworks to the right go down to the water table, the quicksands and possibly the pre-cambrian.

A quiet Sunday afternoon.

I hear the sounds of a secret club, the sea, and endearingly cute lessons in solving word puzzles.
Surely sally is not a sudden violent excursion.
I played the Mickey Mouse game.
We had a nice walk down the beach - gentle wind and plenty sun.

Fast Bowls

Went crown green bowling today with Rebekah, Tim and some guys from church.
Rebekah won her game 10-4.
Toward the end we did fast-bowls. All the players stand in a line and bowl on a ready-steady-go. Do that twice and all the bowls are bowled.
It makes for a quick and exciting game (as if it wasn't quick or exciting enough already).

Exporting to facebook

Most of my not-chat blog style posts will be made to my blog instead of facebook, but I use facebooks RSS-blog tracking to import these posts into facebook notes.

As I already copied many of my facebook posts into my blog I don't want them importing back into facebook, so I consulted http://www.consumingexperience.com/2008/07/blogger-unofficial-feed-faq.html so that I could prevent facebook from importing blog posts from before yesterday.

I'll still be using facebook for chatting (maybe until I get get plus'd)

My Song in the Night

Slightly cheesy video in some parts, but I'll let them off.

Improved PNG and ZIP merging

Following on from my post on displaying a zip as a png, I wanted to do a more robust job, to have the zip file properly enclosed as a non-critical png ancillary stream and to maybe even to have the png included as non-compressed member of the zip archive. ouch!

My main reason for doing this is compatibility reasons as I hear that windows zip-explorer cannot easily open zip files appended to a png.

PNG

Reading the PNG file standard I find that I should probably use the chunk type: ziPS making it a ancillary, private and unsafe to copy if the PNG is changed.

As the zip file must be at the end of the PNG it must be the last data stream in the PNG, however ancillary chunks are not allowed to have ordering restrictions, and IEND should be the last chunk.

Making the chunk unsafe to copy reduces the chances that a png editor could place the chunk in another position, however 14.2 c states A PNG editor is always allowed to copy all unrecognized ancillary chunks if it has only added, deleted, modified, or reordered ancillary chunks. This implies that it is not permissible for ancillary chunks to depend on other ancillary chunks.

Of course we are not attempting to preserve the zip in any major way if the png is edited, only to stop any zip stream being preserved in a way that prevents it being used.

ZIP

The resource for the zip format was wikipedia ZIP_(file_format) where I learn the the final part of the zip central directory is a 2 byte comment length and then a comment.

This tells me that the IEND image trailer could be the last 4 bytes of the comment... but it would mean that the ziPS chunk of the png would not include the entire zip file, as the last 4 bytes of the comment would be external to the zip file.

It means that a zip file which was formally extracted from the png would be incomplete and not recognisable.

I can accept this as the ziPS chunk is not intended to be formally extracted by png-aware software, we only enclose it as a png chunk so that it may be preserved in... er... circumstances where anything after IEND might be removed.

Compatibility

I hear that some zip programs fail to work against zip archives that have been appended to a png. This is an error on the part of such zip programs, which seem to presume that the first item of the zip file is a zip File Entry.

Wikipedia ZIP_(file_format) states:

Often the first thing in a ZIP file is a ZIP entry, which can be identified easily by its signature. But it is not necessarily the case that a ZIP file begins with a ZIP entry, and is not required by the ZIP specification.

We cannot accommodate such demanding ZIP programs, as the ZIP entry signature is:

ZIP local file header

Offset	Bytes	Description[5]
0	4	Local file header signature = 0x04034b50 (read as a little-endian number)
4	2	Version needed to extract (minimum)
6	2	General purpose bit flag

...

and the first eight bytes of a PNG datastream always contain the following (decimal) values:  137 80 78 71 13 10 26 10

Method

The method then seems to be:

1. Modify the zip file, by adding 13 bytes to the zip comment
1. a NULL to sort-of terminate any existing comment to try and hide IEND which we add
2. 12 bytes of IEND
2. Parse the PNG to find IEND
   (Probably the last 12 bytes of the file)
3. replace with the crafted zip file

As this won't solve the compatibility problem, I don't feel inclined to write the necessary code unless it becomes important to stop streams after IEND from being removed by png uploaders, if you know what I mean. (An ancillary stream or critical stream in a png is part of the png. A stream after the IEND is not part of the png).

Fluid google template

I'm using the Picture Window template but wanted a fluid (stretchy) design so that folk with wide screens don't need to see my lines of computer program wrapped in a terribly hard to read fashion.

I was able to find notes on adapting this template with custom CSS.

body {
padding-left: 50px;
padding-right: 50px;
}

html body .content-outer {
max-width: 1600px;
}

which does the trick nicely

pdf and zip in google blogger

Google blogger allows only .png, .jpeg and .gif files to be uploaded as part of a blog post.

As my blog is somewhat technical, I want to be able to offer a zip file from time to time, containing sources I mention.

A first thought would be to rename the .zip to .png and tell the user to rename it after download, but even better is to have a chimera file which is both a .zip and a .png at the same time! The png could display the instruction to rename the file to .zip

I can then upload this .png file to google blogger which will display telling folk to download and rename to .zip

I've done this for a pdf in my blog post here where the thumbnail png brings up a large-png. If the large-png is saved to disk and renamed to .pdf, the pdf from which the thumbnail was taken is displayed.

This works because .png and .gif files have their magic at the beginning of the file, and .zip files and .pdf files have their magic at the end of the file. (The magic is the part of the file that is first read by the file viewer to work out how to treat the file).

Thus there is no deception. The chimera file is truly a .png file, and is truly a .zip file.

This example file here contains the instructions:

(right-click and save link/target as .zip not .png)

Be careful to save the link-target (the original and full sized image) and not the thumbnail.

Importing a threaded MBOX into google blogger

My first blog used wordpress. I worked out some script to export the wordpress database as a threaded-mbox and for a while ran my blog as a web interface to a citadel imap folder.

Circumstances moved me to google blogger and I didn't want to lose my old posts or comments.

So I wrote a sed script to convert a threaded mbox into google blogger format.

it's pretty complete. If you want to know how it words, just read ths source, man!

#! /usr/bin/sed -f

/^Content-[Tt]ype: /{
  # keep reading till new From
  :-body
  N
  /\nFrom - /!b-body
}

/^Content-[tT]ype: multipart\/alternative/{
  # Are we a comment (convert to text) or a post
  x;/\n *<category.*#comment.*>/!{x;b-get-html};x

  :-get-text
  /\nContent-Transfer-Encoding: quoted-printable/{
    s/=0D=0A/\n/g
    s/=20/ /g
    s/=\n//g
    s/=3D/=/g
  }

  s/^.*\nContent-type: text\/plain/Content-Type: text\/plain/i
  :-chop
  /^\n/b-chopped
  s/^[^\n]*\n//;
  b-chop
  :-chopped
  s/\n--[^\n]*\nContent-type.*//i
  s/^\n\n*//

  s/^/  <content type="html">\n/i;
  s/$/\n  <\/content>/i;

  H

  s/^.*/From - /

  :-get-html
  x
  s|$|\n  <category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/blogger/2008/kind#post'/>|
  x

  /\nContent-Transfer-Encoding: quoted-printable/{
    s/=0D=0A/\n/g
    s/=20/ /g
    s/=\n//g
    s/=3D/=/g
  }

  s/^.*\nContent-type: text\/html/Content-Type: text\/html/i
  s/\n--[^\n]*\nContent-type.*//i

  :-got
}

/^Content-[tT]ype: text\/html/{
  # try to save the <title> instead of the subject line
  /<title>/{
    x
    s/$/\n/
    G
    s/\n\n.*<title>\([^<]*\)<\/title>.*/\n  <title>\1<\/title>/i
    # rid of the old title
    s/ *<title>[^<]*<\/title>\n//
    x
  }

  s/^.*\?<body\b[^>]*>[\n ]*//i
  s/[\n ]*<\/body>.*//i

  # Are we a comment (convert to text) or a post
  x;/\n *<category.*#comment.*>/!{x;b-convert-html};x

  :-convert-text
  s/<br\b[^>]*>/\n/gi
  s/<p\b>/\n/gi
  s/<[^>]*>//g
  s/&\#8320;/…/g
  s/^\n\n*//
  s/\n\n*/\n/g
  b-quote

  :-convert-html
  x
  s|$|\n  <category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/blogger/2008/kind#post'/>|
  x

  :-quote
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g

  s/^/  <content type="html">\n/i;
  s/$/\n  <\/content>/i;

  H

  s/^.*/From - /
}

/^From - /{
  s/.*//
  1!{
    s/$/<\/entry>\n/
    H
    x
    p
    x
  }
  s/.*/<entry>/
  h
  d
}

/^Message-ID:/{
  s/^[^:]*:[[:space:]]*<\?//
  s/>\?$//
  s/#/_/g
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g
  s/^/  <id>/
  s/$/<\/id>/
  H
  d
}

/^References:/{
  s/^[^:]*:[[:space:]]*<\?//
  s/>.*$//
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g
  s/^/  <thr:in-reply-to ref="/
  s/$/" type="text\/html"\/>/
  s|$|\n  <category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/blogger/2008/kind#comment'/>|
  H
  d
}

/^Subject:/{
  s/^[^:]*:[[:space:]]*//
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g
  s/^/  <title>/
  s/$/<\/title>/
  H
  d
}

/^From:/{
  s/^[^:]*:[[:space:]]*//
  s/^"*//
  s/".*//
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g
  s/^/  <author><name>/
  s/$/<\/name><\/author>/
  H
  d
}

/^Date:/{
  s/^[^:]*:[[:space:]]*<\?//
  s/\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\([^ ]*\) *\(...\)\(..\).*/\4-\3-\2T\5.000\6:\7/
  s/Dec/12/i
  s/Nov/11/i
  s/Oct/10/i
  s/Sep/09/i
  s/Aug/08/i
  s/Jul/07/i
  s/Jun/06/i
  s/May/05/i
  s/Apr/04/i
  s/Mar/03/i
  s/Feb/02/i
  s/Jan/01/i
  s/&/\&amp;/g; s/"/\&quot;/g; s/</\&lt;/g; s/>/\&gt;/g
  s/^/  <published>/
  s/$/<\/published>/
  H
  s/published/updated/g
  H
  d
}

${
#  a</content></entry></feed>
}


The hard part is that google will sometimes drop posts or comments without explaining why (even after claiming to have imported 20 comments it may be that only 17 are available after import) so it took a lot of trail and error.

On comment seemed to be dropped because it contained this text:

13. Use with the GNU Affero General Public License.

And to prove it, I was able to post that a s a comment to a google blog, but if I exported and imported the blog the comment would be dropped.

Fast and Quiet - new heatsink

My computer is faster with a new heatsink.

3 of the fixing brackets on the old one had snapped and the computer would shutdown from overheating before it booted. It must have also been thermal-trottling before because now it is 2-3 times faster and SILENT instead of a noisy wind tunnel.

Anne bought this Zalman heat sink from CCL online after an online vendor failed to ship the official HP part after a week.

Zalman CNPS10X Performa Ultra-Quiet CPU Cooler - CNPS10X-Performa - CCL Computers

Socket LGA1366, LGA1156, 478, LGA775, AM3, AM2+, AM2, 754, 939, 940 compatible, Mfg Code: CNPS10X-Performa