In these days of cheaper disk storage, is it really worth to have data on tape? Disk acquisition costs might be more (but hey..there are cheaper ones out there) and so is a tape library. Of course, if you already have any of these, the acquisition costs might not be a question.
Lower RTOs are achieved by employing disk-to-disk (D2D) technologies - even if data is remote / offsite. A better understanding of the backup-environment, backup-application, and needs can help in deciding which one is a better route. If the requirements are only to have 1-month (or 3-month) retention, and to have that data offsite, the most cost-efficient way to achieve it is D2D backups. Of course, you can do D2D2T if there is a permanent archival requirement.
Now, let us take a use-case of 50TB of data that need to be backed up with a 10% daily incremental growth and retention of 90-days.
Assuming, backup times need to be faster, you will need LTO-4 drives which has native speed of 120MB/s and can accommodate 800GB un-compressed (1.6TB compressed). If all the 50TB is from the same client and backup-policy, you will not need more than 1 or 2 drives. But in real life, that data is coming from several systems at the same time and hence you will need at least 4-6 drives (of course you can do multiplexing, but that will slow down the speeds).
With a 2:1 hardware compression, you would need approx 32 tapes to do 1-full backup + 10% incremental for 6 days = 16 tapes. For one week, it is approx 48 tapes consumed. Now, if they need to go offsite and if you need data available readily for restores, you will have duplicate them and send them offisite (maybe IronMountain). That will make it 96-100 tapes per week. With a 90-day retention (let's say 12-weeks) you would need at least 1200 tapes to start with. Each media costs approx $100 which makes it $120,000. Plus the tape library cost ~300,000. This and labor costs to change tapes daily, offsite costs make it approx 350K for 50TB backup to hold for 90-days
So, an LTO-4 tape library (STK/Quantun/IBM) with 6-drives and 90-day retention for 50TB backups, it is an investment of approx $400K.
In this case, time to backup 50TB is 115hrs and assuming data is sent in equal streams to all drives, it would be approx 20hrs. And then for duplicating tapes would be another 20-hrs. So your backups are running 40-hrs everyday (yes, that is correct!). You will now need to get more drives!!! Increased drives makes the cost go by at least 100K (backup licensing and hardware) - total of 500K
Now, let's do the disk-way.
You would need a VTL that can do site-site optimized replication (for offsite) and has de-duplication capabilities, can compress and integrate with Backup application for the replication component. Now, since the weekly backup size is 80TB (full+10%incr), you will need backend storage for the de-duplicated repository (SATA is good enough) of at least 80TB (to be on safe side) i.e 80 drives of 1TB or approx 6-disk shelves with 14-disks each. VTL space itself can be approx 10TB of FC/SAS. As backups keep coming in, the data can be de-duplicated and the VTL space will be freed up. Also, optimized replication can start at the same time.
With 4Gbps FC SAN, this can be done in 3.5hrs (with 10-virtual drives - add as many as you want - its free!!). Also, de-dup (using one of the VTL vendor's information), can be completed in 27hrs (this is for the entire 50TB), and minus backup-time= 23hrs. Also, at the end of 27hrs from start of backups, your entire 50Tb is also offsite (you don't need 40hrs per day). And with new backup application features, after the first full backup, you can follow the incremental-forever model and still have your full backups created on-the-fly or near-instantaneously. This will save tons of hours as well as the congestion/traffic on the LAN/SAN.
Now comes the good part, you dont need anymore storage space for further backups. All data that gets backed up daily, weekly is a duplicated backup. I have seen upto 9.5:1 ratios in such cases, and that means a 95% reduction in space requirements. So for your 90-day backup retention you just need approx 30-40TB of de-dupe repository ( with 2:1 compression as well) and not 80TB as I mentioned above.
So 2 sites with VTL appliances and 40TB SATA and 15TB FC/SAS costs approx 400-500K and backup licensing (if you do not have it already for VTL) approx 50-100K. Plus bandwidth costs approx 50-100K. So total solution costs is at the high-end 700K as opposed to 500K.
It may seem high cost initially but you are eliminating any manual intervention (or mucking) with your tapes. Also, tapes might not be readable when you need them!! And data is offsite within few hours of your backups. Finally, if the retention is a lot more than 90-days and if you do not have local hands-and-feet support, the cost benefits of having tape will significantly drop down.
Lower RTOs are achieved by employing disk-to-disk (D2D) technologies - even if data is remote / offsite. A better understanding of the backup-environment, backup-application, and needs can help in deciding which one is a better route. If the requirements are only to have 1-month (or 3-month) retention, and to have that data offsite, the most cost-efficient way to achieve it is D2D backups. Of course, you can do D2D2T if there is a permanent archival requirement.
Now, let us take a use-case of 50TB of data that need to be backed up with a 10% daily incremental growth and retention of 90-days.
Assuming, backup times need to be faster, you will need LTO-4 drives which has native speed of 120MB/s and can accommodate 800GB un-compressed (1.6TB compressed). If all the 50TB is from the same client and backup-policy, you will not need more than 1 or 2 drives. But in real life, that data is coming from several systems at the same time and hence you will need at least 4-6 drives (of course you can do multiplexing, but that will slow down the speeds).
With a 2:1 hardware compression, you would need approx 32 tapes to do 1-full backup + 10% incremental for 6 days = 16 tapes. For one week, it is approx 48 tapes consumed. Now, if they need to go offsite and if you need data available readily for restores, you will have duplicate them and send them offisite (maybe IronMountain). That will make it 96-100 tapes per week. With a 90-day retention (let's say 12-weeks) you would need at least 1200 tapes to start with. Each media costs approx $100 which makes it $120,000. Plus the tape library cost ~300,000. This and labor costs to change tapes daily, offsite costs make it approx 350K for 50TB backup to hold for 90-days
So, an LTO-4 tape library (STK/Quantun/IBM) with 6-drives and 90-day retention for 50TB backups, it is an investment of approx $400K.
In this case, time to backup 50TB is 115hrs and assuming data is sent in equal streams to all drives, it would be approx 20hrs. And then for duplicating tapes would be another 20-hrs. So your backups are running 40-hrs everyday (yes, that is correct!). You will now need to get more drives!!! Increased drives makes the cost go by at least 100K (backup licensing and hardware) - total of 500K
Now, let's do the disk-way.
You would need a VTL that can do site-site optimized replication (for offsite) and has de-duplication capabilities, can compress and integrate with Backup application for the replication component. Now, since the weekly backup size is 80TB (full+10%incr), you will need backend storage for the de-duplicated repository (SATA is good enough) of at least 80TB (to be on safe side) i.e 80 drives of 1TB or approx 6-disk shelves with 14-disks each. VTL space itself can be approx 10TB of FC/SAS. As backups keep coming in, the data can be de-duplicated and the VTL space will be freed up. Also, optimized replication can start at the same time.
With 4Gbps FC SAN, this can be done in 3.5hrs (with 10-virtual drives - add as many as you want - its free!!). Also, de-dup (using one of the VTL vendor's information), can be completed in 27hrs (this is for the entire 50TB), and minus backup-time= 23hrs. Also, at the end of 27hrs from start of backups, your entire 50Tb is also offsite (you don't need 40hrs per day). And with new backup application features, after the first full backup, you can follow the incremental-forever model and still have your full backups created on-the-fly or near-instantaneously. This will save tons of hours as well as the congestion/traffic on the LAN/SAN.
Now comes the good part, you dont need anymore storage space for further backups. All data that gets backed up daily, weekly is a duplicated backup. I have seen upto 9.5:1 ratios in such cases, and that means a 95% reduction in space requirements. So for your 90-day backup retention you just need approx 30-40TB of de-dupe repository ( with 2:1 compression as well) and not 80TB as I mentioned above.
So 2 sites with VTL appliances and 40TB SATA and 15TB FC/SAS costs approx 400-500K and backup licensing (if you do not have it already for VTL) approx 50-100K. Plus bandwidth costs approx 50-100K. So total solution costs is at the high-end 700K as opposed to 500K.
It may seem high cost initially but you are eliminating any manual intervention (or mucking) with your tapes. Also, tapes might not be readable when you need them!! And data is offsite within few hours of your backups. Finally, if the retention is a lot more than 90-days and if you do not have local hands-and-feet support, the cost benefits of having tape will significantly drop down.
Hi
ReplyDeleteFew comments if I may:
I feel the pricing is off - "Each media costs approx $100" well a quick google search says $55 for one tape. 1200 would be a lot cheaper than that. Ditto for the $300k quoted for a library, having sold 1200 slot i2000 for a lot less than that recently I feel thats over egged too.
1200 slots for 50TB? By my crude maths you are saying that on average every file will have 19 version on the system? (10 with 50% tape wastage) Now I may come from a media background where our files change very little, but even so this sounds like its a scenario designed to play to dedupe. Dedupe also has major limitations with some files, for example it really doesn't work for quicktime files or dpx based projects.
A proper installation using something like TSM, would always go to disk cache 1st then migrate onto a tape pool (and copy pool simultaneously), to avoid shoe shining the tape. Also TSM would allow you to have a disk array with the 'active' copy on, to increase restore times.
"you would need approx 32 tapes to do 1-full backup + 10% incremental for 6 days = 16 tapes. For one week, it is approx 48 tapes consumed." A monthly/weekly full backup is unnecessary, you would do a full backup on day 1 and incremental there after. 2 drives could keep up with a change rate of 3 tapes per day, in duplicate.
Expiration & Reclamation will consolidate and rewrite the data to new tapes once a volume has fallen below a set utilisation level (eg 50%). For a crude example, once 50% of the data on 10 tapes is 'expired' the system would write 5 tapes and reclaim the first 10. This would happen on another pair of drives.
The VTL was afforded twin sites with a data link between them, why not have two LTO libs with a server at each side, one managing the backup the other the copy pool target. I would leave the tapes in the library that are performing backups, only removing archival tapes. Keeping human interaction to a minimum. The admin overheads appear to be the same, as the VTL solution also is reliant on backup knowledge. Lets please also consider the TCO in power and cooling.
I am a big fan of disk based systems, but lets have a fair fight. Does anyone have realistic Copan pricing they would care to share? Also lets talk recovery times, something VTL would wipe the floor with tape on.
Hi Tim
ReplyDeleteThe pricing might be really off for media, but it is off for everything I mentioned. The numbers mentioned (other than media) were almost near to standard prices being offered. You mentioned tape library costing less than 300k, but that might be the case of larger discounts for large customers. You can consider these as standard discount list prices.
Apart from the pricing, yes, the post is pitching for Disk-to-Disk backups having an edge over tape-setup. That does not mean, I am pro-eliminate-tape. Tape has (and will) have its place in any kind of backup/recovery setup. The idea here is to look at new technologies with which you can do more with less i.e storing more backup data on less amount of disk storage. And yes, de-duplication is not for all data types (see my other posts on de-duplication).
This post is to give an idea of what can be achieved using new features of backup products and their integration with VTL. Of course, if you are using TSM this does not apply - and it can always be argued that TSM is version based and does an incremental forever. If you have short retentions, then it should not matter. But in case of longer retentions (1Yr), just imagine the number of tapes you would need for a restore (with TSM) - also, if proper policies were not created, whatever the backup application be, there will always be excessive media usage.
As you mentioned in your comment not everything can be linked to cost. The most ignored fact is recovery - how fast and how soon can it be done. With data readily available on disk, restores can be done faster with better throughputs than tape. Also, having 2 Libs in both locations does not make sense anymore if you can replicate between sites, have one-lib at a site and do physical tape creation there.
Let me know your thoughts.
Hi
ReplyDeleteYes please do add media brand salt to my posts, I primarily deal with video data so have a bias.
Regarding pricing, I still think that for even an unheard of company there is more discount to be had.
TSM can be set to consolidate media during reclamation, it will always try to not interleave different nodes data to stop you needing 100+ tapes to restore one server/project.
Maybe I am being naive with my retention policies, but surely most companies would do something along the lines of, 8 versions active in the last month, 3 active in the last year and 1 version held for 3 months if the file is deleted.
I assume in this model the 50TB is churning data? 10% change per month.