Recently at The Station we’ve decided to enter into digital data backup and archiving. What, you may ask, don’t we do this already? Well, typically in the past at the end of an edit job, the client has been happy to take away, for example, a Digital Betcam or HDCAMSR video tape. However, over the last twelve months, more and more clients are shooting digitally (tapeless) and also asking for digital delivery of final master assets. Therefore, we would like to offer as a service long-term data archiving. Spinning disk is not the best platform for this. Drives can fail, and to set up up a totally secure SAN is an expensive proposition.
Enter the world of data tape. The most recent advance, although at least twelve months old now, is LTO5. This format can store up to 1.5TB of uncompressed data, or 3TB of compressed data on one tape. Single tape, desktop drives are in the region of £2,500 and tapes retail around £60 (I’m sure better prices can be found if one tries). LTO data tape is an excellent, cost-effective medium for long-term archival purposes.
One of the great advantages of LTO5 is that drives can be mounted with LTFS and they simply appear as any other attached storage device. Files can be dragged and dropped to and from the tapes. Transfer rates are allegedly in the region of 140Mbps. We have secured a loan Quantum LTO5 drive from the excellent people at Era UK, via an introduction by our friends at JCA. Of course I was intending to connect this drive to a Linux machine, there was never any question about it. Unfortunately, Quantum only officially supports Red Hat and SuSe (I’m guessing Fedora and CentOS would probably work too), for which they provide compiled binaries. Now, I’m a Debian person and the best available machine had Squeeze installed. You know what’s coming next, and I love a challenge.
Start by downloading the source from Quantum. Unfortunately compiling this source wasn’t as simple as configure, make, make install. After installing the dependencies:
> sudo apt-get install libicu-dev libfuse-dev uuid uuid-dev libxml++2.6-2 libxml++2.6-dev
./configure worked without a problem. However, make was failing with the following error:
make: Entering directory `/home/station/Downloads/qtmltfs-1.2.0′
Making all in messages
make: Entering directory `/home/station/Downloads/qtmltfs-1.2.0/messages’
genrb number of files: 3
sh: ./icu-config: not found
pkgdata: icu-config: Could not read from icu-config. (fix PATH or use -O option)
required parameter is missing: -O is required for static and shared builds.
Run ‘pkgdata –help’ for help.
make: *** [bin_mkltfs_dat.o] Error 1
make: Leaving directory `/home/station/Downloads/qtmltfs-1.2.0/messages’
make: *** [all-recursive] Error 1
make: Leaving directory `/home/station/Downloads/qtmltfs-1.2.0′
make: *** [all] Error 2
Not really knowing how to solve this initially, I decided to see what would happen if I compiled these sources on my little Ubuntu 10.04 NBR netbook. Of course, the sources compiled without error, but a netbook is not really the device I wish to connect an LTO5 drive to. So, it seemed like there was some sort of Debian Squeeze related package error.
I’m not ashamed to say that I posted to LinuxQuestions.org and a user by the name of knudfl came up with the solution. Behold, the power of the Internet. After removing the development files for libicu44, and installing the relevant counterparts from libicu42, following the suggested directions, I was able to finish compiling the driver. Great, here we go…..
Not so fast. The next step is to format a tape. The supplied documentation from Quantum suggests the following:
> mkltfs -d /dev/st0
This returned the following error:
> Error remapping st device /dev/st0 to sg : Unable to open /proc/scsi/sg/devices
More investigation finds that /proc/scsi is a legacy way of dealing with scsi devices, and in the Debian kernel this has been disabled. The good news is that, there is an option within menuconfig to turn this on, but it does mean compiling your own kernel. A new challenge for me as I’d never needed to do this before.
More help from the Interwebs, and using a very helpful tutorial page, I was able to build my own custom Linux kernel (2.6.39 if you’re interested). It wasn’t hard at all.
Booting into this kernel and everything seems to work fine. The final problem is:
LTFS20062E Unsupported cartridge type (LTO3RW)
LTFS11299E Cannot format: unsupported medium
I guess LTO3 tapes don’t work in LTO5 drives.
In summary, to setup a Quantum LTO5 tape drive on Debian Squeeze, the following steps are necessary:
1. Download the sources from Quantum.
2. Install the necessary dependencies, including libicu42-dev files (follow the steps here).
3. Configure, make and make install the source
4. Compile your own kernel, with legacy scsi support by selecting that option in menuconfig (follow the tutorial here).
5. Buy LTO5 tapes and format them.
6. Mount the LTO5 drive.