[bhattisatish]

Are there any tape based solution which can be used at home? I don't care about time retrieval. It's more for home archival purpose.

I have two NAS servers (both based on Synalogy). But I need something where I can back it up and forgot about it till I want to restore the stuff. I am looking at a workflow of say, weekly backup to tape. Update the index. Whenever I want to restore a directory or file, I search the index, find the tape and load the same for retrieval.

NAS can be used for continuous backup (aka timemachine and timeshift). And archival at a weekly level.

[mm0lqf]

Tape drives are generally SAS so you will need a controller card

I've got a HP StorageWorks Ultrium 3000 drive (It's LTO-5 format) connected to one (LSI SAS SAS9300-4i), in my NAS/file server (HP Z420 workstation chassis). Don't go lower than LTO-5 as you will want LTFS support.

About £150 all in for the card and drive (including SFF-8643 to SFF-8482 cables etc..) on EBay

Tapes are 1.5TB uncompressed, and about £10/each on Ebay, you'll also want to pick up a cleaning cartridge.

I use this and RDX (1TB cartridges are 2-4 times the price, but drives are a lot cheaper, and SATA/USB3, and you can use them like a disk) for offline backup of stuff at home.

[embedding-shape]

Not OP, but similar situation, trying to figure out tape archiving, already using SAS.

However, is there no open formats? The whole LTO ecosystem of course reeks of enterprise, and I'd expect by now at least one hardware hacker had picked together some off-the-shelf components to build something that is magnitude cheaper to acquire, maintain and upgrade.

[uniqueuid]

Short answer: no

Tape is really complicated and physically challenging, and there are no incentives for people investing insane amounts of time for something that has almost no fan base. See the blog post about why you don’t want tape from some time ago.

Edit: https://blog.benjojo.co.uk/post/lto-tape-backups-for-linux-n...

[embedding-shape]

> there are no incentives for people investing insane amounts of time for something that has almost no fan base

Like that has stopped anyone before? :p Probably explain why we haven't seen anything FOSS in that ecosystem yet though.

[adrian_b]

The LTO cartridges are cheap and the programs that you need for using LTO tape drives are open source.

The only problem is that the LTO tape drives are very expensive. If you want to use 18 TB LTO-9 tapes, the cost per TB is much lower than for HDDs, but you need to store at least a few hundred TB in order to recover the cost of the tape drive.

There is no chance to see less expensive tape drives because there is no competition and it would be extremely difficult for anyone to become a competitor as it is difficult to become able to design and manufacture the mechanical parts of the drive and the reading and writing magnetic heads.

[progbits]

If you "back up and forget" there is a good chance you will not be able to restore the tapes when the time comes.

At least with drives you can run regular health checks a corruption scans. Tape is good for large scale but you must have automation that keeps checking the tapes.

[adrian_b]

A tape can be checked much faster than a HDD, because its sequential read/write speed is several times higher than that of a HDD.

However, there is little need to check the tapes, because the likelihood of them developing defects during storage is far less than for HDDs.

Much more important than checking the tapes from time to time is to make multiple copies, i.e. to use at least duplicate tapes that are stored in different places.

Periodic reading is strictly necessary only for SSDs, and it is useful for HDDs, because in both cases their controllers will relocate any corrupted blocks. For tapes it is much less useful. There is more risk to damage the tape during an unnecessary reading, e.g. if the mechanism of the tape drive happens to become defective at exactly that moment, than for the tape to become defective during storage.

The LTO cartridges are quite robust and they are guaranteed for 30 years of storage after you write some data on them.

In the past there have existed badly designed tape cartridges, e.g. the quarter-inch cartridges, where the tape itself did not become defective during storage, but certain parts of the cartridge, i.e. a rubber belt, which was necessary to move the tape, disintegrated after several years of storage. Those have disappeared many years ago.

[adrian_b]

You can buy a tabletop LTO tape drive, a SAS HBA card and an appropriate cable and you can use them with any desktop computer with a free big enough PCIe slot.

The problem is that while the tapes are at least 3 times cheaper than HDDs, and you have other additional advantages, e.g. much higher sequential reading/writing speed and much longer storage lifetime of the tape, the tape drives are extremely expensive, at a few thousand $, usually above $3k.

You can find tape drives for obsolete standards at a lower price, but that is not recommended, because in the future you may have a big tape collection and after your drive dies you will no longer find any other compatible drive.

Because the tapes are cheap, there will be a threshold in the amount of data that you store where the huge initial cost of the tape drive will be covered by the savings from buying cheap tapes.

That threshold is currently at a few hundred TB of stored data.

I use an LTO tape drive and I have recovered its cost a long time ago, but I have more than 500 TB of data.

However, only a third of that is actual useful data, because I make 2 copies of each tape, which are stored in different locations. I am so paranoid because it is data that I intend to keep forever and I have destroyed all the other supports on which it was stored, e.g. the books that I have scanned, for lack of storage space. An important purpose of the digitization has been to reduce the need for storage space, besides reducing the access time.

I keep on my PC a database with the content of all tapes, i.e. with all the relevant metadata of all the files that are contained inside the archive files stored on the tapes.

When I need something, I search the database which will give me the location of the desired files as something like "tape 47 file 89" (where "file 89" is a big archive file, typically with a size of many tens of GB). I insert the appropriate tape in the drive and I have a script that will retrieve and expand the corresponding archive file. The access time to a file averages around 1 minute, but then the sequential copying speed is many times higher than with a HDD. Therefore, for something like retrieving a big movie, the tape may be faster overall than a HDD, despite its slow access time.

There are programs that simulate a file system over the tape, allowing you to use your standard file manager to copy or move files between a tape and your SSD. However I do not use such applications, because they reduce a lot the performance that can be achieved by the tape drive. I handle frequently large amounts of data, i.e. the archive files in which I store data on the tapes are typically around 50 GB, so the reduced performance would not be acceptable.

[doe88]

It is a shame drives are so pricy. Just out of curiosity (knowing nothing to this area), what would you consider the minimum 'viable' LTO tape drive?

[adrian_b]

I have a Quantum LTO-7 drive (6-TB tapes) bought many years ago for $3000.

Today I would strongly recommend against buying a LTO-7 drive, as it is obsolete and you risk to have a tape collection that will become unreadable in the future for the lack of compatible drives. A LTO drive can read 2 previous generations of tapes, e.g. a LTO-9 drive can read LTO-7 and LTO-8 tapes. LTO-10 drives, when they will appear in a few years, will no longer be able to read LTO-7 tapes.

The current standard is LTO-9 (18-TB tapes). If you write today LTO-9 tapes, they will remain readable by LTO-11 drives, whenever those will appear.

Unfortunately, LTO-9 is a rather new standard and the tape drives, at least for now, are even more expensive.

For instance, looking right now on Newegg, I see a HPE LTO-9 tape drive for $4750.

Perhaps it could be found somewhat cheaper elsewhere, but I doubt that it is possible to find a LTO-9 tape drive anywhere for less than $4500.

If you need to store at least 200 TB of data, you may recover the cost of the tape drive from the difference in price between LTO-9 cartridges and HDDs.

Otherwise, you may choose to use a tape drive for improved peace of mind, because the chances for your data that is in cold storage on tapes to become corrupt are far less than if it were stored on HDDs.

I have stored data for many years on HDDs, but the only thing that has kept me from losing that data was that I have always duplicated the HDDs (and I had content hashes for all files, for corruption detection, as the HDD controller not always reported errors for the corrupted blocks). After many years, almost all HDDs had some corrupted blocks, but the corrupted blocks were not in the same positions on the duplicated HDDs, allowing the recovery of the data.