Thursday, May 26, 2016

Parsing paths & deleting empty items from a bash list

This spell will split a path into directories:

IFS=/ read -d "" -r -a path <<<"$path"

but because of the action of <<<, the final item will have a newline appended -- but that can be removed thus:

path[-1]="${path[-1]%$'\n'}"

If the path began with a /, then path[0] will be empty, so as the leading / is important, we'll recover that

test -z "${path[0]}" && path[0]="/"

This spell will remove empty items from the list (and also renumber the indexes):

IFS= eval 'path=(${path[@]})'

(We could avoid the eval by saving IFS, but eval is safe enough here not to bother).

Now it is simple enough to iterate over "${path[@]}" and perform a chdir on each stage.

Monday, May 16, 2016

Update on old DisplayLink drivers

An update on my earlier post of 2012.

Driver 1.1.62 (15 May 2016) is now available at http://blog.sam.liddicott.com/2012/02/hp-displayport-adaptor-on-mint-or.html and does not require any fixups for wrong USB vendor ID numbers, or patches for kernels > 4.5.0

For my HP displaylink hub I have:

/etc/udev/rules.d/50-displaylink.rules:

# DisplayLink devices always have the active configuration on configuration #1
ATTR{idVendor}=="17e9", ATTR{idProduct}=="01d4", RUN+="/usr/bin/dlconfig"


and /usr/bin/dlconfig:

#! /bin/bash

echo 1 > "/sys/$DEVPATH/bConfigurationValue"


However, I must insert the USB device AFTER logging into to X or I get two fb devices added, one owned by udl and one by evdi. I suspect this is a race condition problem.

In any case, none of it works for me -- I'm not able to view anything at all on the frame buffer. I guess my USB device is too old.

However, if I modprobe udlfb and ignore the evdi drivers, and udl then I at least get a working framebuffer that can display images with fbi. -- but the colour depth is stuck at 16.

I'm able to use it as a primary device for an X11 session with this /etc/X11/xorg.conf

Section "ServerLayout"
    Identifier     "X.org Configured"
    Screen      0  "Screen0" 0 0
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
    ModulePath   "/usr/lib/xorg/modules"
    FontPath     "/usr/share/fonts/X11/misc"
    FontPath     "/usr/share/fonts/X11/cyrillic"
    FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
    FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
    FontPath     "/usr/share/fonts/X11/Type1"
    FontPath     "/usr/share/fonts/X11/100dpi"
    FontPath     "/usr/share/fonts/X11/75dpi"
    FontPath     "built-ins"
EndSection

Section "Module"
    Load  "glx"
EndSection

Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "kbd"
EndSection

Section "InputDevice"
    Identifier  "Mouse0"
    Driver      "mouse"
    Option        "Protocol" "auto"
    Option        "Device" "/dev/input/mice"
    Option        "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
    Identifier   "Monitor0"
    VendorName   "Monitor Vendor"
    ModelName    "Monitor Model"
EndSection

Section "Device"
    Identifier  "Card1"
    Driver    "fbdev"
    BusID    "USB"
    Option    "fbdev"    "/dev/fb1"
    Option    "ReportDamage"    "true"
EndSection

Section "Screen"
    Identifier "Screen0"
    Device     "Card1"
    Monitor    "Monitor0"
    DefaultDepth    16
    SubSection "Display"
        Viewport   0 0
        Depth     1
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
    EndSubSection
EndSection

Monday, March 7, 2016

I want some bluetooth headphones.

I want some Bluetooth headphones

  • A2DP - audio
  • AVRCP  - remote control
  • HSP, HFP - telephony
  • CTP - cordless telephony (why not?)

That's the easy bit.
  • Built in mic
  • and voice-dialler.
  • and noise-cancelling mics AND playback
  • and volume-boost for quiet sources (I'm looking at you, DVD player)
  • and AVC/compressor for classical music on noisy trains, planes and auto-mobiles.

That's easy too.

Other inputs:
  • I want it to take a micro-SD card for a built-in mp3 player
  • And even have built-in storage, MTP or FAT32 block device.
  • And have a built-in FM/DAB+ radio.
  • It must charge from micro-USB, and be a USB audio-interface when plugged via USB.
  • I also want a 3.5mm stereo line-in socket so I can use them as normal headphones. But I want it to take a 4-connector 3.5mm jack so that it can function as headphones with mic.
  • It should detect when a 3-connector 3.5mm jack is inserted, and offer a 3.5mm microphone out socket.
  • I also want a 3.5mm line-out that feeds whatever audio source is selected, so that I can feed into an amplifier or to a friend who has the same headphones (so line-out should not necessarily cut off the built-in speakers).

Did I miss anything?
  • It should be able to record to micro-sd or built-in storage. In stereo. From the stereo noise-cancelling microphones any of the other audio sources, including USB, and the phone.
  • I should be able to play from any source to the phone. While recording the phone conversation.
  • And waterproof for jogging in the rain. (Not for me, for joggers).
But not Chrome-cast or Mira-cast. That requires WLAN, and that would be a step too far.


For well less than £100.

On a stick. From standard parts.

BT interface. USB interface. Cross-bar audio mixer. SD interface. GPIO controlled radios.

Wednesday, February 10, 2016

Android 6 semi-adopted storage

How to split your SD card between adopted internal storage and portable external storage.

(see how-to instructions below)

The pain


Fed up of waiting on Motorola for the Marshmallow upgrade on my XT1072, and being short of internal storage, and being very fed up of having to move-to-sd apps after upgrade (along with occasional tricks such as deleting all data for drive, google+, chrome to scavenge extra memory when clear-cache wasn't enough) I decided to upgrade to CyanogenMod 13 to get Marshmallow that way.

I was really looking forward to being able to use my SD card as internal memory and have no more problems about storage.

What really happens is that with Marshmallow you cannot move-to-sd without adopting your SD card as internal. And then the only apps that will move-to-sd (now called Change) are those that could already move-to-sd.

And one of those apps that won't move-to-sd (even after the adoption of the SD card as internal storage) is Google Music.

Only now there is no SD card for Google Music to store the music on. So it stores it on the *internal* memory.

Whaaaat! My music collection is way bigger than my app collection, how does storing my music internally instead of some apps (all of which could move to SD anyway) help anything?

After messing abuot with Links2SD, Apps2SD, root shells and mount points, reading about volume manager and changing API's I decided that even loopback file fat32 systems in the adopted storage probably wouldn't work.

So I looked at re-partitioning an adopted storage card to shrink the adopted partition to make room for a fat32 partition.

In searching how to access the encrypted partition outside of Android (so that I could resize the file system within it) I came across these notes https://nelenkov.blogspot.co.uk/2015/06/decrypting-android-m-adopted-storage.html?view=flipcard I came across the sm command which can create a mixed or public or private volume.

Mixed turned out to be exactly what I was looking for!



How to split your card

WARNING: Before you do this, be sure to eject the card from the Settings/Storage & USB menu.

I found how to partition my SD card to give 8G as internal storage to which all apps that can be moved will be moved and leave ~20GB as portable storage to hold music, etc.

First, you need adb working, and your SD card inserted and formatted as portable.

$ adb shell sm list-disks adoptable
disk:179,64

disk:179,64 is my SD card that can be made adoptable, I want 90% as external SD:

$ adb shell sm partition disk:179,64 mixed 90

Note: Your card may be listed with an underscore _ instead of an underscore, e.g. disk:179_64 in which case, that is what you type.

This erases the entire SD card, and then gives me 90% as portable storage and the rest as adopted internal storage.

The partition table looks like this:
Number  Start   End     Size    File system  Name            Flags
 1      1049kB  57.5GB  57.5GB  fat32        shared          msftdata
 2      57.5GB  57.5GB  16.8MB               android_meta
 3      57.5GB  63.9GB  6369MB               android_expand


The fat32 partition is not encrypted and can be mounted on a computer (provided it can handle the new GUID partition table format).

I advise a reboot after setting the new music storage location this as Google Music may get the wrong idea about much space is available.

Rename the adopted storage. If it had the same name as the portable storage partition then it may prevent one of the partitions from being available over USB MTP.

For reasons I don't understand, my disk label gets set as some junk similar to this: 82^GM-^KM-^?N-q^Xa^Oo and although I can change this by inserting it into a computer: mlabel -i /dev/sdf1 :: if I put it back into my phone, it looks right until I soft-eject and re-insert it (from the menu) - and then the weird label is back.

I wonder if this can be avoided by swipe-dismissing the notice that a new SD card is discovered (which shows after the mixed partition is complete) instead of selecting it.

If you want apps to be installed on the adopted SD partition by default, then you need to choose the Migrate Data option from the menu:



For CM13 this seems to work as an alternative to my original suggestion below which required the phone to be rooted, but for stock it seems ineffective.


# You won't need this if you chose migrate data above
$ adb root pm set-install-location 2

to have apps installed on the storage by default where possible. It is very effective. (Location 1 means internal, and 0 means auto-choose, but I don't know on what criteria).

However that command requires you to have rooted your phone. I wish this could be set another way.

(If you want to root your XT1072, follow the top-post instructions here: http://forum.xda-developers.com/moto-g-lte/general/root-marshmallow-stock-rom-t3317859)

Monday, February 1, 2016

Using flock in bash without invoking a subshell

flock -c can call external commands but not bash functions. Consequently users mess about with file descriptors, and often making a mess of it.

Inspired by Ivan's post http://blog.famzah.net/2013/07/31/using-flock-in-bash-without-invoking-a-subshell I've written a flock wrapper for bash, that uses flock underneath but allows flock -c to work for bash functions.

It can be called just like the regular flock command, with the benefit that the -c invocation is supported for bash functions; so you can use it like this:

flock -o /tmp/process my_thing "$@"

and I strongly recommend the -o option so that the file descriptor used for the lock is not passed to any sub-processes, which could be problematic if long live sub-processes (e.g. re-spawned daemons) keep it open.

It will pass through and invoke the regular flock if the command isn't a bash function, or if you aren't trying to execute a function.

Sadly, it doesn't recognize bash built-in's.

But the good news is, you can use flock -o ...file... on a shell function inside your shell script without having to worry about file descriptors.

# Helper function (in order to preserve $@ in the caller)
# If this isn't used to call a shell function then return 0
# otherwise return $? as the argumment number which represents
# the command/function to be called
_is_standard_flock() {
  local args=$#
  # find the first argument that doesn't begin with a -
  while test $# != 0
  do case "$1" in
     -*) shift ; continue ;;
     *) break ;;
     esac
  done

  # if it is numeric and there are not additional arguments 
  test $# = 1 -a -n "$1" -a -z "${1//[0-9]}" && return 0
  shift
  # (skipping -c if present)
  # or the following argument is also not a shell function then use the original flock
  if test "$1" = "-c"
  then declare -F "$2" >/dev/null || return 0
  else declare -F "$1" >/dev/null || return 0
  fi

  # we can't have shifted many args if this is a legitimate use of flock
  # so we will be in range of the exit code
  return $(( args - $# + 1))
}

# Help function to determine if -o or --close was given in the flock arguments
_wants_close() {
  test "${*/#--close/}" != "$*" && return # will also match bogus arguments like --closed
  # remove any -- options
  set -- "${@/#--*/}"
  # look for options with o
  test "${*/#-*o/}" != "$*" && return
  return 1
}

flock() {
  if _is_standard_flock "$@"
  then : # do outside the if-clause so bash can optimise exec where possible
  else # save the exit code (offset) as $1
       set -- $? "$@"
       # ${!$1} is the lock file 
       # ${@:$(($1 + 1))} might be -c
       test "${@:$(($1 + 1)):1}" = "-c" && set -- "${@:1:$1}" "${@:$(($1 + 2))}"
       if _wants_close "${*:2:$(( $1 - 1))}"
       then { set -- "$1" "$_" "${@:2}" ; command flock "${@:3:$(( $1 - 2))}" $2 && eval '"${@:$(( $1 + 2))}"' "$2>&-" ; set -- $? $2 ; command flock -u $2 ; return $1 ; } {_}<"${!1}"
       else { set -- "$1" "$_" "${@:2}" ; command flock "${@:3:$(( $1 - 2))}" $2 &&       "${@:$(( $1 + 2))}"          ; set -- $? $2 ; command flock -u $2 ; return $1 ; } {_}<"${!1}"
       fi
  fi
  command flock "$@"
}

x

Wednesday, January 27, 2016

Per-PC font sizes, etc

My home network has NFS homedirs, and so it doesn't matter which computer or laptop family members log into. They get all their files.

Now, one of the PC's has a 40 inch monitor, and users prefer a text-scaling-factor of 2 when using that PC.

Changing their personal settings means changing it back again; fortunately a system-wide default of 2 can be set on that PC, and apply to all users that don't override the system value.

Edit the text-scaling-factor in the schema:
sudo nano /usr/share/glib-2.0/schemas/org.gnome.desktop.interface.gschema.xml

and rebuild the schema:
sudo glib-compile-schemas /usr/share/glib-2.0/schemas

[from http://askubuntu.com/a/470253]

And then to reset the users custom value (if any):

gsettings reset org.gnome.desktop.interface text-scaling-factor
[from http://askubuntu.com/questions/60044/how-do-i-change-the-font-dpi-settings]

But doing the same for org.cinnamon.desktop as I do for org.gnome.desktop

text-scaling-factor
scaling-factor
cursor-size

Also default-zoom-level to larger and default-use-tighter-layouts to true, in /usr/share/glib-2.0/schemas/org.nemo.gschema.xml

Bash: setting and testing $?

You want to set $? in bash

?() {
  return ${1:-$?}
}

and then:

$ \? 2
$ echo $?
2 

You also want to test $?:

$ \? 2
$ \? && echo yes $? || echo no $?
no 2 

which is simpler than
$ test $? = 0 && echo yes $? || echo no $?
no 1 
which also replaces $? with the result of the test.

But how about this extended form that allows you to run a command while preserving (or forcing) the return code?

?() {
  set -- $? "$@"
  if test "$#" -le 2 -a -z "${2//[0-9]}"
  then return ${2:-$1}
  else "${@:2}"
       return $1
  fi
}

e.g.:

$ tar -xzf "$tar"
$ \? rm -fr "$tar"

which leaves $? set to the result of the tar extraction.

Of course a command which is purely numeric with no arguments is mistaken for an exit code.

Note the use of: set -- $? "$@" as function arguments are which is the only lexical scoped variables in bash.