Amazon migrates offline tape data to Glacier with Snowball Edge VTL-in-a-box

Amazon Web Services suitcase-sized Snowball Edge box can now collect ingest data from tape cartridges and be shipped to an AWS centre for file import into Glacier object storage.

The idea is to offer a tape data migration service via a shipped disk drive container rather than by sending the tape cartridges direct to Amazon or networking the data. 

AWS chief evangelist Jeff Barr has written a blog outlining the process. ”We are taking another step forward by making it easier for you to migrate data stored offline on physical tapes. You can get rid of your large and expensive storage facility, send your tape robots out to pasture, and eliminate all of the time and effort involved in moving archived data to new formats and mediums every few years, all while retaining your existing tape-centric backup and recovery utilities and workflows.”

AWS Snowball Edge device.

Sending tape data across a network link can take much, much longer than the time needed for AWS to despatch a Snowball Edge device to a customer, have it filled with data and then returned to AWS. 

For example, transmitting 80TB of data — the Snowball Edge tape import limit — across a WAN link can take about 41 days if sent across a 200Mbit/sec link, and around 82 days if the link speed is halved. Then there are the network costs to take into account, plus any latency issues.

By having the customer copy the files from tape and import them into the Edge box, AWS can be sure that the tape can be read, the data is in the right format, and the customer deals with any data ingest problems.

Storage gateway software is set up on the Snowball Edge box and the tape files read in as if coming into virtual tape files in a virtual tape library (VTL). A example in Barr’s blog shows tape-held backups from Veeam Backup being moved across into the Edge box. We don’t know what other backup file formats are supported. The data is tagged to be stored in S3 Glacier Flexible Retrieval or Glacier Deep Archive storage classes.

The net result is that an online tape gateway and VTL is set up in AWS for customers. Access can be from backup and recovery software running either on-premises or in the public cloud. The on-premises route requires the VTL to run in a virtual machine or hardware appliance.

Snowball Edge can be used for offline tape migration in the US East (N. Virginia), US East (Ohio), US West (Oregon), US West (N. California), Europe (Ireland), Europe (Frankfurt), Europe (London), Asia Pacific (Sydney) AWS regions. 

Comment

The big issue with AWS Glacier is egress costs. But, if restores are done infrequently, that cost will be manageable. If you get hit by a ransomware attack or disaster and have to do a bulk restore back on premises then your level of cost and time-taken unhappiness could be quite high. But you could use AWS’s reverse Snowball service to have the Glacier-held data returned to you.