Archive for the ‘Scripts’ Category

The Trouble Getting A 69

As someone who uses the bus a lot I’ve been using the Real Time Information system since it came out. I now use it pretty much every day to plan my trip.

Usually I find the data to be pretty accurate, with one constant exception – the 69 passing my house in the mornings which is constantly late.

A few weeks ago I decided to gather some data and find out just how late it was. There is still no API for accessing the real time information, so I wrote a python script to get the data from the website, and set up a cron job on my raspberry pi to run the script between 8:00 and 8:45.

That was three weeks ago, so I figured there was enough data gathered that it was worth putting it in a spreadsheet for further analysis. Here’s what I’ve learned:

  • At 8am the bus is due between 8:14 and 8:17
  • On most days it actually arrived between 8:30 and 8:35
  • The first three days seem to have had extra delays, arriving between 8:40 and 8:45
  • On one of the 15 days the bus didn’t run

The full data is available here

RPi Munin Temperatures

Just two weeks ago I upgraded my raspberry pi from the original squeeze image to the newer wheezy based raspbian. Something I tried to do was to get temperature information in munin, but it didn’t seem possible

Today’s update from Eben indicated this was now simples. So, I updated and after reboot the temperature was available from /sys/class/thermal/thermal_zone0/temp.

I’ve put together a quick munin plugin to record this. The temperatures end up getting rounded to the nearest degree, but I think that’s ok.


if [ "$1" == "autoconf" ]; then
     if [ -e /sys/class/thermal/thermal_zone0/temp ]; then
         echo "yes"
         echo "no"
         exit 1
elif [ "$1" == "config" ]; then
     echo "graph_title Raspberry Pi Temperature"
     echo "graph_args --base 1000 -l 0"
     echo "graph_vlabel Temperature in Celsius"
     echo "graph_category Sensors"
     echo "temp.label temp"
     temp=$(cat /sys/class/thermal/thermal_zone0/temp)
     echo "temp.value $((temp/1000))"

Fedora, LXDE and the Dell Mini 10

So, last night I won a Dell Mini 10, and decided to try out LXDE rather than XFCE for a change (using the LXCD spin), so it’s been a little more fun to get working than it otherwise would have been ūüėÄ


This seems pretty light and cool. The default applications are very light, but most are functional enough to use. The panel comes with just the right amount of gadgets and configurations to do everything I need in the smallest possible amount of screen space.



The default web browser Midori seems cool, but just doesn’t have enough functionality to use as a firefox/chrome replacement, so I installed chrome. This mostly worked fairly well, but, it couldn’t open downloads because of xdg-open not being at all LXDE friendly.

Hacking up the terrible default path to be sensible and putting a new xdg-open in /usr/local/bin got around this though.

[andrew@Callandra ~]$ cat /usr/local/bin/xdg-open

[ -z "$1" ] && exit 1

if echo "${1}" | egrep -q "^(https?://|www.)"; then
    exec google-chrome "${1}"
    exit 0

pcmanfm "${1}"


My biggest criticism of the LXDE terminal is that it was clearly designed by someone who didn’t love terminals like I do. It’s functional, but I can’t live without clickable links.

I tried installing the gnome-terminal, but, it too seemed to fail at opening links. Luckily the xfce terminal worked perfectly ūüôā

For some reason though, F11 is mapped to FullScreen by OpenBox, which is a real pain cause I use that for screen. Eventually I found to disable that in ~/.config/openbox/config


The hardware seems mostly good, the screen and keyboard size are most excellent (though, there’s enough room here that there could have been a Page Up/Page down button). The wireless (Broadcom 4312 *shudder*) works fine with kmod-wl from rpmfusion, and the network manager applet works great in LXDE.

Biggest problem with the hardware is the trackpad, which has the buttons integrated (like the ones on the new Mac). This is at best ‘tricky’ to use, it’s possible better drivers would help, I’m still looking.

WTF did I change?

So, it’s been a while since I posted, I thought I’d share my latest heap of hacky shell up here.

Last week we were having a discussion following on from Chris Siebenmann’s post on restoring files from a package, to see what changes had been made. Anyway, we decided that such a feature could be useful, so I promised to hack one together.

It took a week to finish this, because I spent most of my time confused about how apt-get couldn’t just download package without being root. I now know that it defiantly can’t do this. I also know that aptitude can.

# Check we have an argument
if [ -z "$1" ]; then
    echo "Usage $0 /path/to/file"
exit 1
# Check this is actually a text file, if
# not we drop out straight away
if [ `file $1|grep -c text ` -lt 1 ]; then
    echo "The file $1 is not a text file"
    exit 2
# Work out the package name
PACKAGE=`dpkg -S $1| awk -F: '{print $1}'`
# If there's no matching package then
# just exit with an error
if [ $? -gt 0 ]; then
    echo "No package provides this file"
    exit 3
# Download the package
echo "File $1 is provided by $PACKAGE. Downloading to /tmp/$$"
mkdir /tmp/$$
cd /tmp/$$
aptitude download $PACKAGE > /dev/null
# Extract the package
PKGNAME=`find /tmp/$$ -maxdepth 1 -type f -name "*.deb"`
dpkg -x $PKGNAME /tmp/$$
# Diff shit
diff -us /tmp/$$/$1 $1
# If there's a difference then offer to save the diff
if [ `diff -qs $1 /tmp/$$/$1|grep -c identical` -ne 1 ]; then
    echo -n "Save this diff? [yes] "
    read answer
    if [ -n "$answer" ]; then
    if [ "$ANSWER" = "yes" ]; then
        diff -us /tmp/$$/$1 $1 > /tmp/$PACKAGE.$$.diff
        File saved to /tmp/$PACKAGE.$$.diff
# We should be nice and offer to clean up
echo -n "Delete the temporary package files from /tmp/$$? [yes] "
read answer
if [ -n "$answer" ]; then
if [ "$ANSWER" = "yes" ]; then
    rm -fr /tmp/$$/

This script is also available from the RedBrick package repo

Dirvish, Part 2

In my earlier post on dirvish I said I’d have to beat it to do the kind of expire rules I wanted… well I’ve finally found time to do that.

It’s a pretty simple approach, I’ve used the cron script to touch a dotfile if it’s the start of the week/month, and if that file exists then it determines the expire time, which is just passed as an argument to dirvish

This way, the first backup of the month should always be kept for a year, and the first of the week for a month, no matter when they occur.

date=`date +%d`
day=`date +%a`
if [ "$date" -eq 1 ]; then
touch /home/backup/littlebrick/.monthly
if [ $day == "Sun" ]; then
touch /home/backup/littlebrick/.weekly
if [ -f /home/backup/littlebrick/.monthly ]; then
dirvish --vault littlebrick --expire 1year --image-time 00:00
elif [ -f /home/backup/littlebrick/.weekly ]; then
dirvish --vault littlebrick --expire 1month --image-time 00:00
dirvish --vault littlebrick --image-time 00:00

With that done, all that’s left to do is to remove the dotfiles following a successful backup. I used the post-server options for this.

post-server: ; /backup/dirvish/ $DIRVISH_DEST

And, this is the post_backup script

# This script runs after backups, and if the backup was
# successful clears out the month/week flags
#$2 is the DIRVISH_DEST
cd $2
cd ..
#we're now in /home/backup/littlebrick/YYYYMMDD/
success=`grep -c "Status: success" summary`
if [ "$success" -gt 0 ]; then
if [ -f /home/backup/littlebrick/.monthly ]; then
rm /home/backup/littlebrick/.monthly
if [ -f /home/backup/littlebrick/.weekly ]; then
rm /home/backup/littlebrick/.monthly

Laptop Backups With Dirvish

My laptop is something I should *really* backup, since most of my files and stuff I’m working on is there. Since I’m too lazy to do this regularly though I setup an automated system to do this after I re-installed it this week.

My objective was to setup a backup system that would always run automatically, without me having to press any buttons. Incremental backups were a must. Keeping multiple days of backups was also a relatively high priority.

Dirvish is a perl wrapper for rsync. It’s main feature is to automate running incremental rsync backups on a regular basis without too much hassle. Having worked with it alot before it seemed the obvious choice for running my laptop backups.

Before I continue, some terminology:

Server: the machine running dirvish, in this case my desktop
Client: the machine with the files, in this case my laptop (littlebrick)
Bank: A directory where dirvish stores backups (eg. /backup)
Vault: A particular backup source (eg, my laptop)
Branch: A particular run of a vault (usually, daily)


First, I needed to setup my laptop to have a static ip address on the wireless, which uses dhcp.

host littlebrick {
hardware ethernet       00:1b:77:37:af:3d;
fixed-address ;


and then I added this ip to the local dns server.

littlebrick     A

Now I can specify the laptop by hostname in the config files later. You could skip this step and just use the ip or /etc/hosts of course.


Next step was to generate an ssh key, so the server could connect without a password

laptop# ssh-keygen
laptop# cat > /root/.ssh/authorized_keys2
laptop# scp id_rsa root@desktop:~/.ssh/

I set the laptop to allow root ssh (I’ll tidy this up a bit later), and then tested the desktop root user could ssh to the laptop.

Actually, the default in most cases is to allow ssh access as root (either via password or key). I think this is fairly nuts as a default, and usually change it pretty quickly


Once that’s done I started on dirvish. On the server I installed dirvish, and the client already had rsync, so nothing to install there. With that done I started configuring dirvish. All the config for this needs to be done on the server.

Fedora didn’t provide any default master.conf for dirvish, so I copied one I’d written before, and made a few small changes.

littlebrick     00:00
# by default we keep backups for 7 days
expire-default: +7 days
*   *     *   *         1    +1 months
*   *     1   *         *    +1 year
# permissions for logs, etc.
meta-perm: 600

Most of this is fairly self explanatory – the bank is the directory where all the backups will be kept. The exclude rules specify directories that we don’t want to include in the backups.

The expire rules are setup to keep the first backup of the month for a year, and the first of the week for a month, and all other backups for 7 days. I’ve found this to work well in the past. Obviously, it’s flawed here, but I’ll get to that later.

server# mkdir -p /home/backup/littlebrick/dirvish

Then, setup the default.conf in this directory. (the format here is $bank/$vault/dirvish)

client: littlebrick
tree: /
xdev: 0
index: none
image-default: %Y%m%d
image-perm: 700

Most of the options used here are fairly obvious, I want to backup the root of the host littlebrick, without keeping any indexes, and with a permission of 700 on the backup directory, which should be named in a YYYYMMDD format (you can add times and stuff here, but i’ve never felt the need. Keeping it in this simple format makes it easier to script stuff).

The xdev option specifies whether rsync should move from the original file system. Since /home is seperate from /, and has most of my stuff xdev needs to be turned off here. A value of 0 here is false (don’t stick to one file system), all other values (including the word false) will be interpreted as true.

With that done it’s time to test it. Hopefully it’ll “just work” ™.

server# dirvish --vault littlebrick --init

Knowing this would take ages, I went to make some coffee. Before it finishes run get the exact rsync command ran on the client from ps.

With that command edit /root/.ssh/authorized_keys2 (on the client) to look like this:

command="rsync --server --sender -vlHogDtpre.iLs --numeric-ids . /",from="",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa UrCysuKiGZRQzAfypvZNUuOmcPHu
aXq+vBkguC8ausv+wnSXhR9TINA7jDlu6g4pkkl7eCvOelzxvJ6O62kimt6JNWHJTHUMwue/mwSog8HPmACwY/OSOtelAih76v0Pur6dlMZT5cw1oOnU8DBXjt1e3V2Mw5wQxMxorBtXHeYxw== root@littlebrick

Where the command is the rsync command is the one from ps earlier, and the ip address is the ip of the dirvish server. With that done, edit sshd_config and set: “PermitRootLogin forced-commands-only”

Then I just added a script to /etc/cron.daily on the server to run it daily


dirvish-runall will backup any vaults specified in the runall section of /etc/dirvish/master.conf. dirvish-expire deletes any images that have been lying around longer than the expire time.

Still to do….

I find the pattern of keeping the first monthly backup for a year, and the first weekly for a month works well on systems where I expect backups to run every day, but obviously this won’t run daily. I’d like a system whereby I can ensure that the first successful backup can be kept for the month, regardless of what date it is. Dirvish doesn’t support anything this fancy, so it’ll take some scripting.

At the moment backups are set to run about 4am, this means they should generally be finished before I want to leave in the morning, but it’s not that unusual for me to be still working on my laptop at 4am. I’d like a script to stall/cancel backups if I’m still working. I’m thinking I could do this by querying the state of gnome-screensaver…

Simple Word Document Viewer

I have an eeepc900. It’s great for reading emails and stuff, but not very powerful. I installed abiword to read the occasional word document but it seems to eat the cpu. What I really wanted was something that could just view the documents as quickly as possible, so I started asking around…

< andrew> has anyone come accross a gui word document viewer for linux?
< phaxx> andrew: antiword tbh.doc | xmessage
< andrew >phaxx: catdoc tbh.doc | less would have a similar affect

Actually, this wasn’t that bad an idea, but I wanted more formatting than catdoc would give, and without the bother of opening a terminal.

I’ve used wvHtml before for processing mutt attachments, it does a pretty good job of converting word documents to html.

$ cat /usr/local/bin/doc-viewer
wvHtml "$@" /tmp/$$.html
exec firefox /tmp/$$.html

This is what I came up with, works excellently. (I have /tmp mounted in ram on the eee, so I wasn’t too bothered about clearing from /tmp). I use doc-viewer as the default program to handle word documents in firefox, so word docs open within the browser ūüôā