Why aren't tape drives a thing in consumer space? - 45 TB cartridge for the price of a 2 TB SSD

[SandAthlete]

When planning out a new way to store my old files and make backups of existing things, I stumbled upon an article on tape storage.

Main selling point of said article:

Today, a modern tape cartridge can hold 15 terabytes. And a single robotic tape library can contain up to 278 petabytes of data.
It's true that tape doesn't offer the fast access speeds of hard disks or semiconductor memories. Still, the medium's advantages are many. To begin with, tape storage is more energy efficient: Once all the data has been recorded, a tape cartridge simply sits quietly in a slot in a robotic library and doesn't consume any power at all. Tape is also exceedingly reliable, with error rates that are four to five orders of magnitude lower than those of hard drives. And tape is very secure, with built-in, on-the-fly encryption and additional security provided by the nature of the medium itself. After all, if a cartridge isn't mounted in a drive, the data cannot be accessed or modified. This "air gap" is particularly attractive in light of the growing rate of data theft through cyberattacks.

Obviously, a random hobbyist won't have an entire robotic archive. But I still got curious and went down the rabbit hole (read: did few google searches). Here's what I found:

• The most common supertape standard is Linear Tape-Open.
• The latest generation (LTO-9) standard is capable of holding either 18TB natively or 45TB compressed, with speeds up to 400 MB/s uncompressed or 1,000 MB/s compressed.
• The lifespan of data stored on LTO tape is usually quoted as 30 years vs HDDs which generally last 5 years.
• A random google search for one gave me this HP tape, which goes for around 200 dollars. In realm of drives, they go for few grands.

While yes, the drives are pricy as hell, they aren't pointed at your average consumer either. Which begs the question: if tapes are a superior format in terms of data density & security, why isn't there any consumer-grade adaption for it?
Data hoarders won't need instant access anyways and your mother can fit all family memories on a single tape (as backup).

 

[The Great Milenko]

The answer is, and always will be, there is no money in it. ROI is king.

If by 'consumer grade' you simply mean niche then I get you. Otherwise the average consumer has little interest. Price, instant data access, and the cloud are enough.

 

[Slobber Mutt]

Price, instant data access, and the cloud are enough.

This. Hosts buy tape drives and normies buy "cloud access".

 

[Zyklon Ben's Poison Pen]

Cost. Even in the days of VHS there was always concerns about economies of scale but not sure what the main issue is.

 

[Flaming Dumpster]

why isn't there any consumer-grade adaption for it?

You also need specialized software to write to a tape and it's not at all suited for random access.

External HDDs are basically better for normal people in every conceivable way and that 5 year longevity is not at all correct. I've pulled data off of HDDs that have sat cold for decades and generally the hardest part is the fact their interface (IDE, SCSI, whatever) is obsolete. In 20 years time, it'll be a lot easier to pull data off of a random HDD sitting around than a current gen LTO tape.

 

[mr.moon1488]

I hope VHS tapes become a consoomer thing because I'd absolutely be happy with getting rid of the junk cluttering my guest room closet in exchange for some soy shekels.

 

[Manul Otocolobus]

You also need specialized software to write to a tape and it's not at all suited for random access.

External HDDs are basically better for normal people in every conceivable way and that 5 year longevity is not at all correct. I've pulled data off of HDDs that have sat cold for decades and generally the hardest part is the fact their interface (IDE, SCSI, whatever) is obsolete. In 20 years time, it'll be a lot easier to pull data off of a random HDD sitting around than a current gen LTO tape.

This is accurate. In the last few years I've pulled data successfully off of rather poorly made IDE interface drives in laptops from literally more than 30 years ago. I've ever booted and run systems off of internal drives that are 25 years old without a problem after they haven't been powered up in more than 20 years.

 

[Kosher Dill]

In 20 years time, it'll be a lot easier to pull data off of a random HDD sitting around than a current gen LTO tape.

Yeah, although things are better than they used to be, tape is locked into a tight cycle of planned obsolescence. That's fine (perhaps even a good thing) if you're an enterprise with a constantly expanding database that you should be rewriting and re-backing-up anyway, but if you're just a guy who wants to put a few terabytes into deep cold storage, tape isn't the way to go.

If you want to recover an LTO tape in 20 years, the tape will be fine but you'd better have Jurassic Parked an entire workstation hardware and software setup in amber alongside it.

 

[Besachf Jhakut]

Biggest problem I see with tapes for consumers is that current LTO media needs to be kept in certain environmental conditions, and you potentially lose all of it in a particular place if it exceeds heat and/or humidity limits; hard disks aren't hardly so reliable or shock resistant, but especially offline this is the one place they really shine. On the other hand if you're not putting at least one copy offsite you're doing it wrong.

Other issues include some extra hardware to buffer enough for a single tape file that should be at least 10GB, and a SAS card and cables to talk to the drive. But you can find everything you need especially in earlier generations used, although I'd buy fresh tapes, and ideally look for a deal on a new drive.

You also need specialized software to write to a tape and it's not at all suited for random access.

Normies are using Windows or Macs, right? So they can buy the "specialized software," I just use GNU tar OOPS: and a very simple C program I wrote that copies some of dd and deals with the end of the tape to continue on to the next. Before that I used Bacula which is full featured, includes a GUI etc.

You're of course right about random access, and unlike original computer tape systems where it was often primary storage, they squeeze more out of tapes by shingled style writing where each track is partly overwritten by the next, so no going back and changing previously written blocks or files. There is a standard file system way of using LTO tape just based on software, but I use it strictly for backups.

In 20 years time, it'll be a lot easier to pull data off of a random HDD sitting around than a current gen LTO tape.

But much less certain if the tape has been properly stored. Just because you and others have had luck with old drives doesn't mean you can depend on them working in the very next second.

Yeah, although things are better than they used to be, tape is locked into a tight cycle of planned obsolescence. That's fine (perhaps even a good thing) if you're an enterprise with a constantly expanding database that you should be rewriting and re-backing-up anyway, but if you're just a guy who wants to put a few terabytes into deep cold storage, tape isn't the way to go.

If you want to recover an LTO tape in 20 years, the tape will be fine but you'd better have Jurassic Parked an entire workstation hardware and software setup in amber alongside it.

There's always going to be firms that'll read old media. I wouldn't say it's planned obsolescence, just demands for storing data continue to soar past what the technology can support.

LTO in particular, with one exception when media changed, and I'm not sure about LTO-9, had a strict policy of being able to read the current and two past generations of tapes, and write current and one generation previously. If you're a consumer, you're going to buy everything you need and potentially keep using it until your tapes start giving out, at which time you'll move wholesale to a newer but still far from state of the art generation. You'll be using your drive at a duty cycle so much lower than enterprises it'll probably last for a decade or more. My current setup is in its second decade.

 

[Maggots on a Train v2]

Most people have bad memories of prying mangled tapes out of their VCR or car stereo. Marketers probably know that the average person associates "tape" with a shitty, unreliable, disposable medium.

 

[SandAthlete]

You all make very good points.
I just want to try it out, maybe get some older system for cheap and play around with that.

 

[Smaug's Smokey Hole]

You all make very good points.
I just want to try it out, maybe get some older system for cheap and play around with that.

There are cheap (used) drives that uses DAT or other familiar types of cassettes. If you just want to play around with it, that is. No SAS necessary, they use SATA.

 

[Densetsu]

My recent forays into buying computer parts have led me to believe that even enthusiasts care a lot less about data storage than before. Finding a case that has space for lots of hard drives or even an optical drive is fucking impossible these days. It's mostly due to SSDs becoming cheaper and the NVME/M.2 format becoming more commonplace, that and chipset designers refusing to add more SATA slot support.

Then you get down to the normalfag level and you find increasing amounts of tech retardation and easier/cheaper/faster internet access. There's no need for a 150GB iPod anymore when you can stick a 128GB SD card into your phone. There's no need to store music files on an SD card when you can just get shitty 96kbps versions streaming from Spotify or Amazon Music or Sirius or whatever the fuck. Hell, there's no need to store photos anymore because you can just put them on Someone Else's Computer (the Cloud) for free or almost free. Nobody needs storage anymore unless they're running a business, an actual computer enthusiast, or some gamer who needs to have the latest 200GB AAA game installed.

And then you seriously think there's a case for high-density magnetic tape storage in the consumer/normalfag market?

 

[XYZpdq]

I recall consumer tape backup in the 90s before CD-R became common, like around the age of Zip Disks and a bit before
but yeah between burning optical and xgig thumbdrives it was a niche that never really found an audience
sorta like how MiniDisc didn't take off in the USA so much

 

[1996 Toyota Camry]

Consoomers aren't buying in to tape backups because the hardware required to facilitate them and the tapes themselves cost way too much compared to the cost and convenience of cloud storage. I do use tape backups myself but I can see why the mainstream doesn't see the advantage.

 

[MisoSupper]

What's the average start-up costs for this? Just from a cursory glance these LTO tapes themselves are very affordable but the decks used to read them and the software seem to be astronomically expensive (were talking 2 grand minimum). But maybe I'm looking in the wrong places.

 

[1996 Toyota Camry]

What's the average start-up costs for this? Just from a cursory glance these LTO tapes themselves are very affordable but the decks used to read them and the software seem to be astronomically expensive (were talking 2 grand minimum). But maybe I'm looking in the wrong places.

I sold a HP LTO-5 SAS tape drive recently for about 150 AUD. The other problem is that Windows won't see tape drives natively, not so sure about linux but they're designed to be used specially with backup software such as Veeam and shit like that, another reason why the mainstream isnt buying in to tape, using Windows as the primary reasoning here because that is the OS of choice for consooomers.

 

[Besachf Jhakut]

The other problem is that Windows won't see tape drives natively, not so sure about linux....

I'd guess the former is because the tape drive is behind a SAS host interface? Or you're not using a server version of Windows?

Linux will natively recognize my HP LTO-4 SAS tape drive, the "mt" command usually with a -f for which special file it is can be used to directly manipulate a tape. Like "mt -f /dev/nst0 eject" or marking a used one as unwritten by "mt -f /dev/nst0 weof" as in write end of file. The leading 'n' for the special file means the tape won't be automatically rewound after the command is finished.

A bit of searching just now indicates Bacula which has both free and paid versions natively supports Windows. Here's a general discussion from them as of early 2020 including possible provided with Windows backup tools. Note there's two types here, system restore and individual files backup.

For the former you're usually imaging all or part of a disk, ideally you've put your system into its own partition for just that and to limit damage if any one filesystem gets into trouble (but make Windows system partitions big! Microsoft isn't economical shall we say). You can use a Linux live cd like SystemRescue to boot into and run on DRAM to make a compressed copy of one or more partitions or a whole disk, then use dd to copy it to your tape drive, assuming it'll fit on one tape. Thus the recommendation to set up your system so that critical bit of data is on not very many partitions.

BE CAREFUL USING THESE TOOLS, if you use the wrong special files or make a typo you can destroy the contents of your disks! But they make loss of your system drive much less obnoxious, write the newest copy and then use individual file based backups to bring it up to the current state.

A good reference for this sort of "bare metal" backup and restore is the otherwise somewhat dated Backup & Recovery: Inexpensive Backup Solutions for Open Systems by W. Curtis Preston. Also see if he's still running forums on these and more "enterprise" types of backups.

I would also note some cloud backup systems have iffy reputations, and you don't have a real solution until you've done a full restore as a test. Actual users only care about restores, not backups.

 

[TracdacianTortoise]

You also need specialized software to write to a tape and it's not at all suited for random access.

External HDDs are basically better for normal people in every conceivable way and that 5 year longevity is not at all correct. I've pulled data off of HDDs that have sat cold for decades and generally the hardest part is the fact their interface (IDE, SCSI, whatever) is obsolete. In 20 years time, it'll be a lot easier to pull data off of a random HDD sitting around than a current gen LTO tape.

This isn’t entirely correct nowadays, LTO drives past LTO4 can be formatted as LTFS, which allows for interfacing with the tape drive like a normal hard drive by storing metadata for files separately from the actual files, so it doesn’t have to read through the entire tape to list out the files.

What's the average start-up costs for this? Just from a cursory glance these LTO tapes themselves are very affordable but the decks used to read them and the software seem to be astronomically expensive (were talking 2 grand minimum). But maybe I'm looking in the wrong places.

They’re expensive cause of how complex the internals of the drives are and the fact only datacentres or archives tend to have them.

You all make very good points.
I just want to try it out, maybe get some older system for cheap and play around with that.

I did this myself. If you want to save yourself hassle, try to get an LTO5 or newer drive, they’re more expensive but they support LTFS which makes things a lot easier to deal with than manually writing tar files to the tape and keeping track of where each file is manually. Make sure you buy an SAS drive, and not a fibre channel or SCSI drive unless you can also get hold of the supporting hardware. If you go with an LTO4 or earlier drive, you’ll mostly be using it with Linux, using those older drives on Windows is a pain and requires dedicated backup software, where as on Linux you can just dump whatever on it.

As for why they aren’t common in consumer circles, its mostly that the drives are too expensive, and most people aren’t used to non-random access storage. There are some Lightning Cable based desktop varieties of the newest drives that seem to be oriented towards photographers and video producers, like this one: Unitex USB LTO 8 drive

Companies that use tape drives tend to use automated tape libraries with multiple drives and a robotic loading mechanism to automate backups and retrieval