Écrit par Nicolas Bernaerts

Samedi, 20 Juin 2009 00:00

You are hosting a server (or a workstation) with some important files that sould be backed-up regularly on some external storage ?

This article will explain how to use the fantastic possibilities of UDEV & RSync to automatise some files backup on an external USB hard disk.

The concept is as follow :

You insert an external USB disk on one of your server USB port
Your server recognise the inserted disk & mount it
According to some backup rules associated with that specific disk, your server starts a differential backup
When backup is over the disk is unmounted & the server beeps you the indicate that the backup is over

And, with this approach, you can use as many backup disks as you want, as the backup is done according to rules associated with a specific disk.

We will use :

UDEV to recognize the disk and start the associated backup script
RSync to do a very fast differential backup

This setup has been done on Debian Squeeze, but it should be compatible with any following version.

So, here are the installation steps to setup that type of backup.

Install the packages

First thing to do is to install Rsync & Beep packages :

# aptitude install rsync beep

Setup UDEV to recognise your backup disk

I suppose that your backup disk contains only one main partition and has already been formated.
As we will use RSync, the disk sould be formated using :

Ext2 or Ext3
This is the easiest option, the less troublesome.
But your backup disk won't be accessible on a Windows computer.
FAT32
Your backup disk to be usable on a Windows system.
Disk mounting and RSync command will need some specific options for the backup to be fully operational.

Before seting up UDEV, you need to find some specificities that will allow to recognise your backup disk.
To do so, just connect it to a USB port of your server, wait few seconds for it to be accepted and throw this command :

# dmesg

[...] USB Mass Storage support registered.
[...] usb-storage: device scan complete
[...] scsi 2:0:0:0: Direct-Access SAMSUNG HM400LI PQ: 0 ANSI: 2 CCS
[...] sd 2:0:0:0: [sdc] 781422768 512-byte hardware sectors (400088 MB)
[...] sd 2:0:0:0: [sdc] Write Protect is off
[...] sd 2:0:0:0: [sdc] Mode Sense: 34 00 00 00
[...] sd 2:0:0:0: [sdc] Assuming drive cache: write through
[...] sd 2:0:0:0: [sdc] 781422768 512-byte hardware sectors (400088 MB)
[...] sd 2:0:0:0: [sdc] Write Protect is off
[...] sd 2:0:0:0: [sdc] Mode Sense: 34 00 00 00
[...] sd 2:0:0:0: [sdc] Assuming drive cache: write through
[...] sdc: sdc1
[...] sd 2:0:0:0: [sdc] Attached SCSI disk

You see here that your disk is accessible as /dev/sdc.

To get some discriminant informations about that specific disk, just type :

# udevadm info -a -p $(udevadm info -q path -n /dev/sdc)

looking at device '/block/sdc':
    KERNEL=="sdc"
    SUBSYSTEM=="block"
    DRIVER==""
    ATTR{range}=="16"
    ATTR{removable}=="0"
    ATTR{size}=="781422768"
    ATTR{capability}=="12"
    ATTR{stat}=="    2149      205    17980    15232       92       92     1472     3988        0    15300    19224"

looking at parent device '/devices/pci0000:00/0000:00:1d.7/usb5/5-2/5-2:1.0/host2/target2:0:0/2:0:0:0':
    KERNELS=="2:0:0:0"
    SUBSYSTEMS=="scsi"
    DRIVERS=="sd"
    ATTRS{device_blocked}=="0"
    ATTRS{type}=="0"
    ATTRS{scsi_level}=="3"
    ATTRS{vendor}=="SAMSUNG "
    ATTRS{model}=="HM400LI         "
    ATTRS{rev}=="    "
    ATTRS{state}=="running"
    ATTRS{timeout}=="30"
    ATTRS{iocounterbits}=="32"
    ATTRS{iorequest_cnt}=="0x8c9"
    ATTRS{iodone_cnt}=="0x8c9"
    ATTRS{ioerr_cnt}=="0x0"
    ATTRS{modalias}=="scsi:t-0x00"
    ATTRS{evt_media_change}=="0"
    ATTRS{queue_depth}=="1"
    ATTRS{queue_type}=="none"
    ATTRS{max_sectors}=="240"

You can see in red some informations that can be used to recognise your specific disk.
We now need to setup a new UDEV rule to be applied any time your disk will be connected.
Such a rule should be declared in a file under /etc/udev/rules.d. This file should follow the naming convention xx-nameofrule.rules, where xx is used to order the rules execution. Usually, all user specific rules should be declared with xx = 50.

For this specific disk, where I want to backup all my video files, I will create this rule :

/etc/udev/rules.d/50-backup.rules

# UDEV rules to setup automatic backup upon disk insertion
# You can get the discriminant informations with the following command :
# udevinfo -a -p $(udevinfo -q path -n /dev/sda)

# Backup - Rule for the hard disk that will backup video files
KERNEL=="sd?1", ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="SAMSUNG", ATTRS{model}=="HM400LI", RUN+="/root/backup-video.sh %k"

# Backup - You can setup rules for other hard disks here
#KERNEL=="sd?1", ACTION=="add", SUBSYSTEMS=="scsi", ATTRS{vendor}=="xxxxxxxx", ATTRS{model}=="yyyyyyyy", RUN+="/root/backup-zzzzzz.sh %k"

Here the rule says something like this :

when the kernel finds some partition sd?1 (your only partition on the disk),
if subsystem is SCSI (which is the case for USB drives),
if vendor is SAMSUNG,
if model is HM400LI,

then run the script /root/backup-video.sh.

In our example, the parameter %k which will be passed to our script will contain /dev/sdc1.

You then need to restart UDEV for the rule to be operationnal :

# /etc/init.d/udev restart

Create the backup script

We now need to write the differential backup script which will be called by UDEV.

This script will use RSync to compare the files on the server and the ones already backed-up.

The beginning and end of backup will be logged in /var/log/syslog.

/root/backup-video.sh

#!/bin/bash
#
#-----------------------------------------------------------------------
#
# Script to backup all needed data upon USB hard disk insertion
# It is called thru UDEV with :
# - the device name (sda1, ...) given as the first parameter
#
#-----------------------------------------------------------------------

# Log beggining of backup
/usr/bin/logger Backup - Beginning at `date`
# if needed, create the mount directory
if [ ! -d /mnt/backup ] ; then mkdir /mnt/backup ; fi

# if your backup disk is formatted in ext2 or ext3, use following line to mount the backup disk
/bin/mount -t auto /dev/$1 /mnt/backup
# or if your backup disk is formatted in FAT32, use following line to mount the backup disk
#/bin/mount -t vfat -o shortname=mixed,iocharset=utf8 /dev/$1 /mnt/backup

# Backup command using RSync
/usr/bin/logger Backup - Video
/usr/bin/rsync -rtv --del --modify-window=2 /path/to/your/files/video /mnt/backup

# You can add here some other backups ...
#/usr/bin/logger Backup - other files
#/usr/bin/rsync -rtv --del --modify-window=2 /path/to/your/files/other /mnt/backup

# force sync of files to disk before unmounting
/bin/sync

# unmount the backup disk
/bin/umount /mnt/backup

# Log end of backup
/usr/bin/logger Backup - End at `date`

Here, the files I want to backup are under /path/to/your/files/video. They will be backed-up under /video on your USB disk.
The option --modify-window=2 indicates to RSync to consider a 2 seconds difference between timestamp of one file and its backup as acceptable. It is necessary in case you use a FAT32 filesystem on your backup disk, as FAT32 time management is not as accurate as Ext2 ro Ext3.

Note that the backuup disk mount command line is different depending on the filesystem type.
In case of a FAT32 backup disk, the mount options are very important for the file comparison to be accurate.

Be sure that your script is executable.

Everything is setup.

You just need to connect your USB disk to see the light blinking and to automatically start your backup !
After the 2 seconds beep, just disconnect it and keep it safe :-)

Hope it helps.

< Préc		Suivant >