LinuxMint 17.3 on dell XPS 8900

I'm having trouble installing Linux Mint 17.3 on my Dell XPS 8900

I re-sized the windows partition from within Windows, but the Ubiquity installer chokes on the partition table.

sda is my hard disk, and sdb is some intel SSD cache to go with the hard disk.

To avoid Ubiquity choking, I hide them, thus: (tips from http://unix.stackexchange.com/questions/102648/how-to-tell-linux-kernel-3-0-to-completely-ignore-a-failing-disk)

mint ~ # echo 1 > /sys/block/sda/device/delete
mint ~ # echo 1 > /sys/block/sdb/device/delete

and then install on a spare USB hard disk.

That worked, and I can boot off the USB hard disk by using F12 on power-up to get a boot menu.

I will later try to DD that installation to the spare partition, and somehow merge the UEFI partition.

I don't seem to need the tips here: http://askubuntu.com/questions/700617/problems-installing-ubuntu-14-04-3-on-dell-xps-8900

This may be useful: http://forums.linuxmint.com/viewtopic.php?f=90&t=211740&p=1104716

--

dmraid -ay

fixes the problem and supports Intel Matrix Raid which is the combination I have. http://www.linuxquestions.org/questions/ubuntu-63/intel-matrix-storage-manager-with-ubuntu-804108/

But when I came to install, having selected the new partition made under windows, it is confirmed as being /dev/mapper/blah_blah_5 (striped).

striped? So I daren't click yes. I would expect "sporadically mirrored" but not "striped".

One solution was to use "dmraid -ay" but install onto a separate internal SATA disk, and install the bootloader to the same partition I install onto, but the bootloader would not install on that disk, failing due to not being able to find the real mount point of /cdrom (!!) (and I don't want to install it onto SDA).

My working solution was to get a new internal SATA disk and connect it as an external USB disk and install on that, first using the "echo 1 > /sys/block/sda/device/delete" trick above on sda and sdb.

Then I move that disk to internal, and the UEFI boot entry still seems to work.

I will be repeating that trick later today when a spare SATA SSD arrives.

(I did try adjusting grub.cfg on /dev/sda1 or creating my own boot entries according to https://wiki.gentoo.org/wiki/Efibootmgr but could never get those to work).

--

Convert pdf to monochrome

From StackExchange

gs \
 -sOutputFile="BW-$1" \
 -sDEVICE=pdfwrite \
 -sColorConversionStrategy=Gray \
 -dProcessColorModel=/DeviceGray \
 -dCompatibilityLevel=1.4 \
 -dNOPAUSE \
 -dBATCH "$1"

Writes sample.pdf to BW-sample.pdf

Marriage of alloca and asprintf as asprintfa

It all comes down to not being able to return variable length arrays in C.

[Note: I realise that I am pushing obscenity to the limit, I don't really use this]

I think that asprintf is the way sprintf should work, but not everyone has it, and so folk either cart around an asprintf implementation, or enjoy getting the tedium of snprintf(NULL, 0, ...) or the worse error-prone tedium of adding up buffer sizes.

A single line  asprintf and corresponding  free() make a much simpler 1-2 lines of use.

Alloca

But while I'm not being portable it would be nice to be able to have asprintf use alloca as it's allocator...

alloca returns space from the current function stack frame (not scope) and is automatically freed when the function returns.

If alloca can't allocate enough memory, then who knows what might happen, but on the other hand If asprintf or malloc can't allocate enough memory, how does your program handle it?

So the plan is to alloca a buffer of the right size, and then sprintf into that.

This 1 line macro has the same syntax as the official asprintf.

#define asprintfa(PTR, ...) sprintf( (*(PTR)) = alloca(1 + snprintf(NULL, 0, __VA_ARGS__)), __VA_ARGS__)

and must be used with two lines:

char* x;
asprintfa(&x, "a %s b %d c\n", "<>", -1);

But I'd prefer a single line use:

char* x = asprintfa("a %s b %d c\n", "<>", -1);

So the plan is to alloca a buffer of the right size, and the right size is measured with 1 + nprintf(NULL, 0, ...). So we define a helpful macro:

#define measure_printf(...) snprintf(NULL, 0, __VA_ARGS__)

Once the size is measured, we can allocated with alloca and call sprintf(alloc, ...) to populate our alloc'd buffer. usual warning that macro arguments might get evaluated twice, but I don't see any other way around that without using a function call to stash the values, and require the heap or static buffers to hold them.

#define asprintfa(...) sprintf(alloca(1 + measure_printf(__VA_ARGS__)), __VA_ARGS__);

However, that will return whatever sprintf returns, which is the size, not the buffer. We need a temporary variable to hold the alloca pointer, and maybe return via a special GNU macro, like this:

#define asprintfa(...) ({ char* x=alloca(1 + measure_printf(__VA_ARGS__)); sprintf(x, __VA_ARGS__); x })

but that also requires us to come up with a variable name that will never be one of the __VA_ARGS__. And relying on alloca is bad enough without insisting on GNU extensions.

Maybe a tuple, like this:

#define asprintfa(...) ( char* x=alloca(measure_printf(__VA_ARGS__)), sprintf(x, __VA_ARGS__), x ) 

It's not a GNU extension but x still might clash with one of the __VA_ARGS__ and now we are declaring a variable in the middle of a scope.

It is simpler to define a wrapper to snprintf (or vsprintf ) that returns the buffer instead of the size. (Although perhaps a simple assembler push/pop would have saved the value nicely).

static char* a_sprintf(char* v, char* format, ...)
{
    if (! v) return NULL; // However alloca is undefined if it fails...
    va_list args;
    va_start(args, format);
    vsprintf(v, format, args);
    va_end(args);
    return v;
}

Which gives this winning combination which returns the string instead of the length (more useful):

#define measure_printf(...) snprintf(NULL, 0, __VA_ARGS__)
#define asprintfa(...) a_sprintf(alloca(measure_printf(__VA_ARGS__)), __VA_ARGS__);
/* sprintf wrapper that returns the buffer address. 
   Does not check size, intended to be used on a 
   properly allocated buffer as part of aasprintf */
char* a_sprintf(char* v, char* format, ...)
{
    if (! v) return NULL; // However alloca is undefined if it fails...
    va_list args;
    va_start(args, format);
    vsprintf(v, format, args);
    va_end(args);
    return v;
}

Here is another version that can be called like this:

char* x = asprintfa("a %s b %d c\n", "<>", -1);

it returns the pointer from the macro directly, but uses a stinky temp var along with GNU statement macro extensions again:

#define asprintfa(...) ({\
  char* __buf__;\
  sprintf(__buf__ = alloca(1 + snprintf(NULL, 0, __VA_ARGS__), __VAR_ARGS__);\
  __buf__; })

Variable Length Arrays

With this, you type:

char asprintfa(x, "a %s b %d c\n", "<>", -1);

and it expands to:

char x[1 + snprintf(NULL, 0, "a %s b %d c\n", "<>", -1)];
sprintf[x, sprintf("a %s b %d c\n", "<>", -1);

The code is:

#define asprintfa(name, ...) name[1 + snprintf(NULL, 0, __VA_ARGS__)]; \
sprintf(name, __VA_ARGS__);

Memory Allocation Failures

It doesn't warn you or fail sensibly if you try to print too much, but on the other hand it's 1 line to use and no need to worry about freeing.

Which is more likely?
With snprintf that you either:

• get the snprintf buffer arithmetic wrong
• or don't cope with allocating too much memory
• or forget to free the memory

or with asprintfa
that you

• allocates too much memory

There's certainly a lot less to get wrong, although admitedly you can't get it all right either - but there's asprintf for that.

sprintf was designed to get wrong, and snprint was designed to also get the arithmetic wrong.

And for completeness, asprintf (now tested):

int asprintf(char **v, const char* format, ...)
{ 
    int len = 0;
    if (! v) return -1;
 
    va_list args;
    va_start(args, format);
    len = vsnprintf(NULL, 0, format, args);
    va_end(args);
    va_start(args, format);
    *v = malloc(len + 1); 
    if (! *v) return -1;
       len = vsnprintf(*v, len + 1, format, args);
    va_end(args);
    return len;
}

[ Edit: Fixed loads of missing 1 + to the result of snprintf while measuring. snprintf returns just the number of characters written except the NULL character, but the length provided should also have space for the NULL.]

Packed binary in bash

Using bash, and other accessible commands, I wanted to output some based 64 encode packed binary string representations of 64 bit decimal numbers, in reverse byte order.

I used bc to convert the arbitrarily long decimal number into hex

bc <<< "obase=16; $N"

e.g.

$ N=123456789
$ bc <<< "obase=16; $N"
75BCD15

I then split this into hex digit pairs to represent one byte each, in reverse order, and prefixing with \x to produce a printf format string.  I do this by appending a space to the line, and then move two (or one, if two not available) digits that precede the space to instead append to the end of the string, (with \x prefix). And then finally, remove the space which is now a leading space.

sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'

e.g. the following pattern space is iterated

75BCD15

75BCD15␣

75BCD␣\15

75B␣\15\CD

7␣\15\CD\5B

␣\15\CD\5B\7

\15\CD\5B\7

Another method of separation might have been to pad with 0 to an even length, and then split off pairs of digits from the front. I then use all of that as the argument to printf which interpolates the characters, and then pipe to base64.

printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64

e.g.

$ N=123456789
$ printf $( bc <<< "obase=16; $N" |
  sed -re 's/$/ /;:start;s/([a-fA-F0-9]{1,2}) (.*)/ \2\\x\1/;Tdone;bstart;:done;s/^ *//;'
) | base64
Fc1bBw==

An alternate method for fixed with fields that don't need byte order fixings:
Convert to hex using printf:

printf -v hex '%08x' $dec
printf '%b' "\x${hex:0:2}" "\x${hex:2:2}" "\x${hex:4:2}" "\x${hex:6:2}"

To convert a string of space separated hex to characters:

input=" $input"
printf -v input '%b' ${input// /\\x}

.

Finding xterm Terminal Window Size for serial console resize

I use a networked serial console displaying in an xterm (TERM=xterm), and to my frustration it can't cope when I resize the xterm, instead either garbling the output or just using the original portion.

I don't know why this should be.
# shopt | grep checkwinsize
checkwinsize on

Various combinations of # kill -SIGWINCH $$ and # kill -s WINCH $$ had no effect at all, and # stty size proved to be 0 0, and thus not very useful.

Edit: It turns out that bash does not directly query the terminal size directly as a result of SIGWINCH or anything else. Changing the size of the tty with stty cols $COLUMNS rows $LINES is what causes the tty driver to send SIGWINCH to the foreground application which then queries the tty driver. Resizing an xterm is what causes xterm to modify the tty, thus sending the SIGWINCH and caused bash to take notice.

With a little help from the source to xterm's resize command and also this python interpretation I came up with this 1-liner for bash, to read the xterm width and height into LINES and COLUMNS and set these in the tty driver ready for vi or other programs to pick up:

IFS=$';\x1B[' read -p $'\x1B7\x1B[r\x1B[999;999H\x1B[6n\x1B8' -d R -rst 1 _ _ LINES COLUMNS _ </dev/tty \
&& stty cols "$COLUMNS" rows "$LINES"

It's worth looking at how this works.

$'...' is a bash quoted string that allows character entities to be expressed in hexadecimal, thus $'\x1B' is the character named ESC.

The string emitted as a prompt instructs the xterm to save the current cursor position, move the cursor to 999,999 which should be beyond the terminal bounds and so instead move to the bottom right corner; it then instructs the xterm to report the current cursor position, and then restore the previous cursor position.

The current cursor position is returned as a string (without newline) in this form: \x1B[lines;columnsR and so this can be read into two variables LINES and COLUMNS that bash uses. But how to do that?

The bash read function will emit the prompt, and then read the response. As the response is terminated in R instead of a newline, -d R is passed to read.

Other useless characters are \x1B [and ; so we put these into IFS causing read to use these to split the input. This gives us 2 empty strings, which we read into the bash underscore variable which gets overwritten every line anyway, so our read variable specification is _ _ LINES COLUMNS _ causing LINES and COLUMNS to take the 3rd and 4th values, with a final _ to take any potential junk that otherwise would have been appended to COLUMNS. Raw mode and silent mode are advised obviously, hence -r -s and a timeout of 1 second is set in case an xterm isn't in use and there will be no response.

So why not define this as a function...

resize() {
  IFS=$';\x1B[' read -p $'\x1B7\x1B[r\x1B[999;999H\x1B[6n\x1B8' \
                     -d R -rst 1 _ _ LINES COLUMNS _ 2>&1 &&
  stty cols "$COLUMNS" rows "$LINES"
} </dev/tty >/dev/tty 2>/dev/null

The stdio redirections are to ensure that stderr is thrown away (in case set -x debug is active, which could mess this up), to ensure that the tty is being accessed whatever stdin and stdout are, and to ensure that the read command has access to stderr (for the prompt) to the tty.

Depending when/how you invoke it you may wish to not loose the previous value of $?, saved here without the use of local variables (which through dynamic scoping might affect the called function).

just() {
  set -- $? "$@"
  "${@:2}" 
  return $1
}

so now you can invoke: just resize without affecting $?, perhaps from something ghastly like:

export PS1="$PS1"'$(just resize)'

or as part of PROMPT_COMMAND although ideally it would run before executing a command.

It now remains to learn why (although a serial terminal cannot be expected to send a SIGWINCH) bash was not responding to SIGWINCH. (Note: Explained here, the signal was not to signify to basg that it should query the terminal using the methods described here, but to tell bash that the STTY rows and columns had already been changed and that it should read those. As we set those using STTY in the resize function, the signal is then sent to bash).

update: a colleague points out the a malicious terminal (or spoofed user input) could emit a bad cursor position response, so $COLUMNS and $LINES must be quoted where used to avoid some stty argument injection, or perhaps worse to other programs which might make careless use of COLUMNS or LINES without validating them as sane integers