Recently at The Station we’ve decided to enter into digital data backup and archiving. What, you may ask, don’t we do this already? Well, typically in the past at the end of an edit job, the client has been happy to take away, for example, a Digital Betcam or HDCAMSR video tape. However, over the last twelve months, more and more clients are shooting digitally (tapeless) and also asking for digital delivery of final master assets. Therefore, we would like to offer as a service long-term data archiving. Spinning disk is not the best platform for this. Drives can fail, and to set up up a totally secure SAN is an expensive proposition.
Enter the world of data tape. The most recent advance, although at least twelve months old now, is LTO5. This format can store up to 1.5TB of uncompressed data, or 3TB of compressed data on one tape. Single tape, desktop drives are in the region of £2,500 and tapes retail around £60 (I’m sure better prices can be found if one tries). LTO data tape is an excellent, cost-effective medium for long-term archival purposes.
One of the great advantages of LTO5 is that drives can be mounted with LTFS and they simply appear as any other attached storage device. Files can be dragged and dropped to and from the tapes. Transfer rates are allegedly in the region of 140Mbps. We have secured a loan Quantum LTO5 drive from the excellent people at Era UK, via an introduction by our friends at JCA. Of course I was intending to connect this drive to a Linux machine, there was never any question about it. Unfortunately, Quantum only officially supports Red Hat and SuSe (I’m guessing Fedora and CentOS would probably work too), for which they provide compiled binaries. Now, I’m a Debian person and the best available machine had Squeeze installed. You know what’s coming next, and I love a challenge.
Start by downloading the source from Quantum. Unfortunately compiling this source wasn’t as simple as configure, make, make install. After installing the dependencies:
> sudo apt-get install libicu-dev libfuse-dev uuid uuid-dev libxml++2.6-2 libxml++2.6-dev
./configure worked without a problem. However, make was failing with the following error:
make: Entering directory `/home/station/Downloads/qtmltfs-1.2.0′
Making all in messages
make: Entering directory `/home/station/Downloads/qtmltfs-1.2.0/messages’
genrb number of files: 3
sh: ./icu-config: not found
pkgdata: icu-config: Could not read from icu-config. (fix PATH or use -O option)
required parameter is missing: -O is required for static and shared builds.
Run ‘pkgdata –help’ for help.
make: *** [bin_mkltfs_dat.o] Error 1
make: Leaving directory `/home/station/Downloads/qtmltfs-1.2.0/messages’
make: *** [all-recursive] Error 1
make: Leaving directory `/home/station/Downloads/qtmltfs-1.2.0′
make: *** [all] Error 2
Not really knowing how to solve this initially, I decided to see what would happen if I compiled these sources on my little Ubuntu 10.04 NBR netbook. Of course, the sources compiled without error, but a netbook is not really the device I wish to connect an LTO5 drive to. So, it seemed like there was some sort of Debian Squeeze related package error.
I’m not ashamed to say that I posted to LinuxQuestions.org and a user by the name of knudfl came up with the solution. Behold, the power of the Internet. After removing the development files for libicu44, and installing the relevant counterparts from libicu42, following the suggested directions, I was able to finish compiling the driver. Great, here we go…..
Not so fast. The next step is to format a tape. The supplied documentation from Quantum suggests the following:
> mkltfs -d /dev/st0
This returned the following error:
> Error remapping st device /dev/st0 to sg : Unable to open /proc/scsi/sg/devices
More investigation finds that /proc/scsi is a legacy way of dealing with scsi devices, and in the Debian kernel this has been disabled. The good news is that, there is an option within menuconfig to turn this on, but it does mean compiling your own kernel. A new challenge for me as I’d never needed to do this before.
More help from the Interwebs, and using a very helpful tutorial page, I was able to build my own custom Linux kernel (2.6.39 if you’re interested). It wasn’t hard at all.
Booting into this kernel and everything seems to work fine. The final problem is:
LTFS20062E Unsupported cartridge type (LTO3RW)
LTFS11299E Cannot format: unsupported medium
I guess LTO3 tapes don’t work in LTO5 drives.
In summary, to setup a Quantum LTO5 tape drive on Debian Squeeze, the following steps are necessary:
1. Download the sources from Quantum.
2. Install the necessary dependencies, including libicu42-dev files (follow the steps here).
3. Configure, make and make install the source
4. Compile your own kernel, with legacy scsi support by selecting that option in menuconfig (follow the tutorial here).
5. Buy LTO5 tapes and format them.
6. Mount the LTO5 drive.
I travel a great deal in Europe. Specifically Germany and Austria. I also use the Internet excessively. UK mobile data roaming rates are absurdly expensive. T-Mobile UK sends me a friendly text whenever I try to use data roaming informing me that it will cost £1 for 3MB, £5 for 20MB or £10 for 50MB. Crazy, and crazily expensive.
I finally decided to bight the bullet and buy a 3G data dongle in Austria. Originally I had thought to purchase something from 3 (Drei), however it seemed that everything advertised was on a two year contract. That’s not for me. Ultimately I went for a bob breitband package. This includes a ZTE MF180 3G dongle/stick and a SIM card. Buying the package for €50 was the easy bit. The package itself was €50 for the dongle, with 1GB data gratis, and then €9 for 9GB per month, on a 30 day rolling contract. €9 for 9GB isn’t too bad, although if you exceed this limit the cost is €4 per GB, which is bad. The other gotcha is that you need an Austrian bank account to setup a direct debit for the monthly payments. I won’t bore you with the details of how I have access to one of those.
After opening the package, inserting the SIM card and then plugging the dongle into my Mint Debian laptop, nothing happened. That’s right, nothing. The dongle showed a red light and Network Manager didn’t recognise any new modem. Cutting long stories short, I unplugged, plugged in, unplugged, plugged in, fiddled and swore for a good hour with no luck. At the point of the customer support help line answering, magically I obtained a connection. The light on the 3G dongle had turned blue. I had no idea how. All is good for the remainder or the day, except that the connection is slower than anticipated. Using speedtest.net, initially I was seeing a download speed of only 1.6Mbps and an upload of 0.10Mbps. Pretty poor. Later in the evening this increased to 2.6Mbps download and 0.3Mbps upload. Still not great.
This morning, I once again found it impossible to obtain a connection? Why? Why? WHY? I extracted the SIM card from the dongle and inserted it into my Nokia N900 phone. With the marvelous help of bluetooth, I easily obtained a data connection. The connection dropped occasionally, but in general all was good. The problem was not with the SIM.
Now that I had a connection again, I used our Googlian friends to search for “ZTE MF180 Debian” for answers. The answer was easy to find. Apparently, the 3G USB dongle is initially recognised as a CD-ROM drive. Eject the drive and everything works. Network Manager asks for a pin number to unlock the SIM, then one is able to connect. I don’t know why the dongle is recognised as a CD-ROM drive, and I don’t know why ejecting it fixes the problem, but it does. Thanks to Christian Amsuess for the information. If everything doesn’t work as expected after ejecting the CD drive, there is more help on Christian’s page.
The network speed is no better today, but at least I’m back online. I wouldn’t recommend bob breitband, even with the cool name, and I certainly would not recommend ZTE MF180 dongle. I’m sure ejecting the CD drive every time I need a connection will become annoying, but at least it works.