Archive for the ‘Linux’ Category

Dirvish, Part 2

In my earlier post on dirvish I said I’d have to beat it to do the kind of expire rules I wanted… well I’ve finally found time to do that.

It’s a pretty simple approach, I’ve used the cron script to touch a dotfile if it’s the start of the week/month, and if that file exists then it determines the expire time, which is just passed as an argument to dirvish

This way, the first backup of the month should always be kept for a year, and the first of the week for a month, no matter when they occur.


#!/bin/bash
#
date=`date +%d`
day=`date +%a`
#
if [ "$date" -eq 1 ]; then
touch /home/backup/littlebrick/.monthly
fi
#
if [ $day == "Sun" ]; then
touch /home/backup/littlebrick/.weekly
fi
#
if [ -f /home/backup/littlebrick/.monthly ]; then
dirvish --vault littlebrick --expire 1year --image-time 00:00
elif [ -f /home/backup/littlebrick/.weekly ]; then
dirvish --vault littlebrick --expire 1month --image-time 00:00
else
dirvish --vault littlebrick --image-time 00:00
fi
#
dirvish-expire

With that done, all that’s left to do is to remove the dotfiles following a successful backup. I used the post-server options for this.

#/etc/dirvish/master.conf
...
post-server: ; /backup/dirvish/post_backup.sh $DIRVISH_DEST
...

And, this is the post_backup script


#!/bin/bash
#
# This script runs after backups, and if the backup was
# successful clears out the month/week flags
#
#$2 is the DIRVISH_DEST
cd $2
cd ..
#
#we're now in /home/backup/littlebrick/YYYYMMDD/
#
success=`grep -c "Status: success" summary`
#
if [ "$success" -gt 0 ]; then
if [ -f /home/backup/littlebrick/.monthly ]; then
rm /home/backup/littlebrick/.monthly
if [ -f /home/backup/littlebrick/.weekly ]; then
rm /home/backup/littlebrick/.monthly
fi
fi

Laptop Backups With Dirvish

My laptop is something I should *really* backup, since most of my files and stuff I’m working on is there. Since I’m too lazy to do this regularly though I setup an automated system to do this after I re-installed it this week.

My objective was to setup a backup system that would always run automatically, without me having to press any buttons. Incremental backups were a must. Keeping multiple days of backups was also a relatively high priority.

Dirvish is a perl wrapper for rsync. It’s main feature is to automate running incremental rsync backups on a regular basis without too much hassle. Having worked with it alot before it seemed the obvious choice for running my laptop backups.

Before I continue, some terminology:

Server: the machine running dirvish, in this case my desktop
Client: the machine with the files, in this case my laptop (littlebrick)
Bank: A directory where dirvish stores backups (eg. /backup)
Vault: A particular backup source (eg, my laptop)
Branch: A particular run of a vault (usually, daily)

DHCP

First, I needed to setup my laptop to have a static ip address on the wireless, which uses dhcp.

/etc/dhcpd.conf
...
host littlebrick {
hardware ethernet       00:1b:77:37:af:3d;
fixed-address           10.0.2.90;
}
...

DNS

and then I added this ip to the local dns server.

/var/named/master/db.internal
...
littlebrick     A       10.0.2.90
...

Now I can specify the laptop by hostname in the config files later. You could skip this step and just use the ip or /etc/hosts of course.

SSH

Next step was to generate an ssh key, so the server could connect without a password

laptop# ssh-keygen
laptop# cat id_rsa.pub > /root/.ssh/authorized_keys2
laptop# scp id_rsa root@desktop:~/.ssh/

I set the laptop to allow root ssh (I’ll tidy this up a bit later), and then tested the desktop root user could ssh to the laptop.

Actually, the default in most cases is to allow ssh access as root (either via password or key). I think this is fairly nuts as a default, and usually change it pretty quickly

Dirvish

Once that’s done I started on dirvish. On the server I installed dirvish, and the client already had rsync, so nothing to install there. With that done I started configuring dirvish. All the config for this needs to be done on the server.

Fedora didn’t provide any default master.conf for dirvish, so I copied one I’d written before, and made a few small changes.

/etc/dirvish/master.conf
bank:
/home/backup
exclude:
lost+found/
/var/log/
/var/tmp/
/var/run/
/var/lock/
/mnt/
/media/
/sys
/dev/
/proc/
/tmp/
/vmlinuz*
/initrd*
.gvfs
Runall:
littlebrick     00:00
# by default we keep backups for 7 days
expire-default: +7 days
expire-rule:
#    MIN HR    DOM MON    DOW  STRFTIME_FMT
*   *     *   *         1    +1 months
*   *     1   *         *    +1 year
#
# permissions for logs, etc.
meta-perm: 600

Most of this is fairly self explanatory – the bank is the directory where all the backups will be kept. The exclude rules specify directories that we don’t want to include in the backups.

The expire rules are setup to keep the first backup of the month for a year, and the first of the week for a month, and all other backups for 7 days. I’ve found this to work well in the past. Obviously, it’s flawed here, but I’ll get to that later.

server# mkdir -p /home/backup/littlebrick/dirvish

Then, setup the default.conf in this directory. (the format here is $bank/$vault/dirvish)

/home/backup/littlebrick/dirvish.conf
client: littlebrick
tree: /
xdev: 0
index: none
image-default: %Y%m%d
image-perm: 700

Most of the options used here are fairly obvious, I want to backup the root of the host littlebrick, without keeping any indexes, and with a permission of 700 on the backup directory, which should be named in a YYYYMMDD format (you can add times and stuff here, but i’ve never felt the need. Keeping it in this simple format makes it easier to script stuff).

The xdev option specifies whether rsync should move from the original file system. Since /home is seperate from /, and has most of my stuff xdev needs to be turned off here. A value of 0 here is false (don’t stick to one file system), all other values (including the word false) will be interpreted as true.

With that done it’s time to test it. Hopefully it’ll “just work” ™.

server# dirvish --vault littlebrick --init

Knowing this would take ages, I went to make some coffee. Before it finishes run get the exact rsync command ran on the client from ps.

With that command edit /root/.ssh/authorized_keys2 (on the client) to look like this:

command="rsync --server --sender -vlHogDtpre.iLs --numeric-ids . /",from="10.0.2.5",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa UrCysuKiGZRQzAfypvZNUuOmcPHu
lZy/B31Mje8UfooJCurM3JKL0t7U7mbor2Fs43OVZhUp6m1xburbDpFOtzKU0UDMO0MopRZ6q8si6fUP5oqAQAE2
U3Pieq3qJGLYzUB3zzzoDromtsYXbCwWDJsVdVVo6yvKNiSDWBZAo2UeyqRtsPuavVhuHqOxwu2/Fqu
aXq+vBkguC8ausv+wnSXhR9TINA7jDlu6g4pkkl7eCvOelzxvJ6O62kimt6JNWHJTHUMwue/mwSog8HPmACwY/OSOtelAih76v0Pur6dlMZT5cw1oOnU8DBXjt1e3V2Mw5wQxMxorBtXHeYxw== root@littlebrick

Where the command is the rsync command is the one from ps earlier, and the ip address is the ip of the dirvish server. With that done, edit sshd_config and set: “PermitRootLogin forced-commands-only”

Then I just added a script to /etc/cron.daily on the server to run it daily

/etc/cron.daily/00dirvish
dirvish-runall
dirvish-expire

dirvish-runall will backup any vaults specified in the runall section of /etc/dirvish/master.conf. dirvish-expire deletes any images that have been lying around longer than the expire time.

Still to do….

I find the pattern of keeping the first monthly backup for a year, and the first weekly for a month works well on systems where I expect backups to run every day, but obviously this won’t run daily. I’d like a system whereby I can ensure that the first successful backup can be kept for the month, regardless of what date it is. Dirvish doesn’t support anything this fancy, so it’ll take some scripting.

At the moment backups are set to run about 4am, this means they should generally be finished before I want to leave in the morning, but it’s not that unusual for me to be still working on my laptop at 4am. I’d like a script to stall/cancel backups if I’m still working. I’m thinking I could do this by querying the state of gnome-screensaver…

Fedora 10 vs. Dell D420

When I installed fedora 7 it involved beating thing about to get it to work. As expected, installing fedora 10 was almost flawless.

The sound, graphics, wireless all worked out of the box perfectly.

As with fedora 8 I had to fix the Font DPI, internal speaker, selinux, useless services. The mouse speed is still a problem, but since there’s now no xorg.conf I just changed this in the preferences menu. Like all my other fedora 10 boxes I just removed PackageKit so that it’d fuck off.

Plymouth worked fine once I added vga=0x318 to the kernel line in the grub.conf.

So, overall the re-install has been very painless. Next I need to get it working with my 3G modem, and set up dirvish to to automated backups. I’ll be posting about that over the next few days 🙂

Xmms in Fedora 8

First things first, xmms isn’t installed in Fedora by default since, well, a while now (FC4 i think). So the first thing to do is install it, in a terminal run
sudo yum install xmms.

At this point, when you get it running you’ll realise you can’t actually play any of your favorite mp3 files as the mp3 support has been removed. To enable it you install xmms-mp3 from the livna repository (if you haven’t set that up, clicky here to do that now. You should note that mixing multiple third party repositories can cause conflicts).

So, it can now play mp3 files, pity it looks like something that should have been abandoned with Windows 3.1. There’s a package in the fedora repo called bluecurve-xmms-skin that will help you with that point. Alternatively, as XMMS is a winamp 2.x clone you can grap any classic skin from winamp.com and put it in the ~/.xmms/Skins Directory (the ~ means your home directory. The files starting with . are hidden files – you’ll need to check that option in the View menu in nautlius, or use ls -A in a terminal)

So, it looks good, but you may have noticed after about 15 minutes that the volume controls don’t actually work. This isn’t actually xmms’ fault, but because Fedora now uses pulse audio. There’s a plugin available to fix this, but it’s not in repos at this time (as far as I know). Download it from here and save the file to your computer. Before you continue you’ll need to install the packages pulseaudio-libs-devel and xmms-devel.

Now that that file has downloaded it’s time to do something with it. Go back to your terminal and cd to the directory it downloaded to (it’s most likely just on the desktop)

cd ~/Desktop Change directory to your desktop.

tar -zxf xmms-pulse-0.9.4.tar.gz Untar the code (the version number of your plugin may differ).

cd xmms-pulse-0.9.4 Move into the directory.

./configure --disable-lynx configure the code to be compiled (if you get errors at this point, you most likely need to install gcc, or another complier tool).

make Compile the code.

sudo make install Install the plugin.

You can now enable the plugin in the xmms options (once you’ve restarted xmms).

You may notice that some of your song names come up as garbage. This is due to a lack of unicode support in ID3v2 tags. This should have been fixed in xmms 1.2.11 (which was not in the repos at the time of writing)

Disclaimer: This last section is only for nerds, or the criminally insane
If, like me, you like to sit on your laptop just out of reach of your desktop, but want to play music out of your nice desktop sound system however this is for you.

You can install a program called ncxmms which will give you a curses frontend for your xmms session that you can run in a terminal (or, more to the point, over SSH). Grab the soruce from here and compile it like the plugin. All the familiar xmms shortcut keys will work there too :)

Fedora 8 on Dell D420

I finally got around to installing the shiny new Fedora 8 on my latitude D420 today. Some of it was good, some of it was bad.

Things that worked first time

  • Wired Network
  • Display
  • Sound
  • Bluetooth
  • SD Card Reader
  • Suspend! (well, at least most of the time. either way, it doesn’t crash anymore)

Things that eh, don’t

  • Wireless

    Update 2: I think that I should be able to get this working with ifl3945, and there’s a kernel upgrade sitting there waiting for me, so after the exams i plan to blacklist the ipw3945 driver, upgrade the kernel and try get it working again.

    Update: I did lots of hacking about trying to get this working with wpa, (and failed miserably), but the wireless does appear to be working with the free driver on the 2.6.23.1-42.fc8 kernel.

    This appears to work with the free firmware just as it did in Fedora 7, but I couldn’t get it to connect to anything, so I went back to ipw3945. I decided not to go with atrpms this time as their mplayer conflicted with livna, which was annoying, so I used the packages from freshrpms (courtesy of lamouroux.net). Note: you should upgrade your kernel and restart the system before running this install.

    rpm -ihv http://ftp.freshrpms.net/pub/freshrpms/fedora/linux/7/freshrpms-release/freshrpms-release-1.1-1.fc.noarch.rpm
    yum -y install dkms-ipw3945 ipw3945d ipw3945-firmware kernel-devel

    I then edited the network settings so that both the wired and wireless devices won’t be started by default when the system boots. not doing this will make your boot time very very long. Then edit the system>services, adding NetworkManager and NetworkManagerDispacher to the default runlevel.

  • Fonts

    I think this is related to the video driver you use, but my fonts were way to large. To fix this go into System>Preferences>Look and Feel > Appearence. In the fonts tab click details and turn the DPI back to 96. While your there, turn the window title text down to size 9, they seem to look good like that.

  • Touchpad

    Just like in F7 you’ll need to change the speed settings, or end up in serious finger pain after not too long.
    Edit the /etc/X11/xorg.conf and change the minspeed option from 0.3 to 0.75 and it should be much better after you restart X.

  • Audio

    I haven’t had a single problem with my sound since the upgrade, but from reading the forums it seems other people have. If you’re having an audio problem it’s most likely related to the new pulse audio system in Fedora 8.

Things I’ve Installed

  • rpm.livna.org

    Some of these instructions will need the the livna repos. To set it up click here.

  • Media

    Xmms is a Winamp 2 clone, i like it because it’s lightweight and simple. The mp3 support is a package in livna called xmms-mp3. For more see my xmms page here.
    yum install mplayer mplayer-gui mplayerplug-in.. Codecs for the default media apps are in a package called gstreamer-plugins-ugly in livna.

  • flash

    Install this rpm to add the adboe repo to yum, then “yum install flash-plugin”.

  • Compiz Manager

    gnome-compiz-manager is in the repos, but not installed by default. Well worth installing if you run compiz.

I also did….

  • System Beep

    I use my laptop lots in lectures and meetings and can’t afford for the system speaker to beep whenever it wants. The best way around it that I found was just to disable it altogether. As root run:
    rmmod pcspkr and then add blacklist pcspkr to your /etc/modprobe.d/blacklist

  • Removed Graphical Boot

    The graphical boot slows down boot time, and isn’t that pretty anyway. To get rid of it edit /etc/grub.conf and remove “rhgb” from the kernel line.

  • Removed useless services.

    Avahi-daemon, cups (if you don’t have a printer), firstboot, isdn, kudzu, netfs, restorecond, sendmail & yum-updatesd are all running by default & if you don’t know why you need them you don’t need them

  • Turn Off SeLinux

    If you don’t know why you need it, then you don’t need it. I disagree with this being enabled by default, new users don’t need it, and it causes so much trouble.

  • Faster Boot Time

    An article by this guy would seem to indicate that if you’ve done all that you’ve halved your boot time by now. (I intend to look at his floppy probe suggestion after my exams).

Stuff I haven’t tested

  • Modem – I just have no use for it.
  • PCMCIA – Again, I have nothing to test with it.
  • Firewire – If anyone would like to send me a firewire cable for my hard drive….

New Stuff (that I noticed)

  • An applet I’d never seen before appeared in System tools for mobile phones, so I played with that. Since that I’ve been looking at other bluetooth/mobile stuff, which I’ll talk about soon…
    Update:I tried to play with some bluetooth stuff, but something seemed to be horribly crashing my system and I think it was bluetooth related, and leaving the bluetooth running forever on my phone seemed to make it run out of memory, so I’ll be leaving this for a while
  • The new audio stuff seems cool, haven’t really done anything to test it though.
  • Yum seems to have had more work done
  • New look & icons etc.
  • IcedTea – Clearly it’s not perfect, but it’s an excellent step in the right direction. This is what we want, free Java that actually works. I’m expecting to see improvements to this over the coming months
  • Lots more stuff, I just haven’t found it yet.

Stuff that kinda disappointed me

  • Yum Search – some moron somewhere decided that the yum output should be butchered to match the apt output (without mentioning that apt has a bazillion more options). Gone is the functionality to search and get a long description of the package, version and architecture information, repo information etc. in one quick and pretty search. I’ve started developing a plugin to fix this, I plan to get back to it the week after next.
  • Graphical boot – I don’t like rhgb, it just slows me down, but if it’s going to be there then it should appear just as neatly as in other distros.
  • Compiz – It’s great that it works with almost no effort, but it already did that. I’d have liked to seen the default configuration applet worked on to have more options and stuff.

Fedora 7 on a Dell Latitude D420

I have updated this page for Fedora 8 –click here.

Getting it to boot after install:

There is a bug in the kernel that ships with the install (2.6.21-1.3194.fc7), so to get it to start after the install you’ll need to add a kernel parameter at the grub menu to disable the acpi. The line should look like this:
kernel /boot/vmlinuz-2.6.21-1.3194.fc7 ro root=LABEL=/ rhgb quiet acpi=off
Once you upgrade to the latest kernel this won’t be an issue. Currently (12/08/07) I’m running 2.6.22.1-41.fc7 with no problems.

Things that work first time:

  • Wired Network
  • Display – runs at 1280×800 by default
  • Sound
  • Bluetooth
  • SD Card Reader

Things that, eh, don’t:

  • WirelessThe free firmware for the intel3945 wireless wouldn’t work for me, so i had to install ipw3945. There’s a pretty
    good set of instructions for how to do this at Fedora Mobile, but that wouldn’t work for me either. Eventually I got it working with the ipw3945 rpm from atrpms. To do this add the following lines to your /etc/yum.conf
    [atrpms]name=Fedora $releasever - $basearch - ATrpms
    baseurl=http://dl.atrpms.net/f$releasever-$basearch/atrpms/stable
    gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
    gpgcheck=1
    After that, its just a matter of “yum install ipw3945″, and a quick reboot. You’ll need to go into System > Adminstration > Services to add set the Network Manager Services to run when the system starts.I added an entry for the card in system-config-network because i wanted to manually set my dns servers, but it
    should work fine without this if you have network manager running.
  • Modem
    Well, i dont actually know if this works or not, because i have no need/way to test it. There’s some pretty good instructions for getting it working here.
  • Touchpad
    You’ll need to edit your /etc/X11/xorg.conf to speed it up a little bit. I changed the MinSpeed option to 0.75.

Stuff I havent tested yet:

  • Firewire
  • PCMCIA

Things I’ve Installed

  • rpm.livna.org
    Some of these instructions will need the the livna repos. To set it up click here.
  • xmms
    Xmms is a Winamp 2 clone, i like it because it’s lightweight and simple. The mp3 support is a package in livna called xmms-mp3. For more see my xmms page here.
  • mplayer
    My favourite media player – it has lots of options, and plays everything. You’ll want these three packages:
    yum install mplayer mplayer-gui mplayerplug-in
  • flash
    Install this rpm to add the adboe repo to yum, then “yum install flash-plugin”.

lspci output:

00:00.0 Host bridge: Intel Corporation Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile 945GM/GMS, 943/940GML Express Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.1 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 2 (rev 01)
00:1c.2 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 3 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
02:01.0 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev b4)
02:01.1 FireWire (IEEE 1394): Ricoh Co Ltd R5C552 IEEE 1394 Controller (rev 09)
02:01.2 Generic system peripheral [0805]: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 18)
09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5752 Gigabit Ethernet PCI Express (rev 02)
0c:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG Network Connection (rev 02)