Amazon held a Storage Day on September 2 and announced a whole raft of new features for files, objects, blocks, file/object transfer, and backup.
They are aimed at lowering costs through tiering data to cheaper storage classes, simplifying access, automating data movements and verifying backup status. There is a list here, with — for us — a NetApp deal and file/object transfer facility being the highlights.
An AWS blog by senior developer advocate Marcia Villalba lays out the list of announcements.
File
The first and main one is FSx for NetApp ONTAP which we covered here and which provides ONTAP as a native managed service on AWS.
The second file announcement adds intelligent tiering to Amazon’s Elastic File System (EFS). This is similar to S3 tiering, with tiers cost- and performance-optimised on the basis of file access patterns. AWS Customer CapitalOne is using this to get lower cost options for its analytics workloads.
If an AWS user has a file that is not used for a period of time, EFS Intelligent Tiering will move it to the Infrequent Access (IA) storage class. If the file is accessed again, Intelligent Tiering will automatically move it back to the Standard storage class.
File Transfer Family
A third file announcement is a completely new service. The AWS Transfer family of Managed Workflows is an onramp/offramp into AWS that automates file and object transfers via SFTP (Secure Shell (SSH) File Transfer Protocol), FTPS (File Transfer Protocol over SSL) and FTP, into and out of S3 or EFS. The use of SCP, HTTPS and AS2 transfer protocols is not supported.
Villalba writes: “Without using Transfer Family, you have to host and manage your own file transfer service which requires you to invest in operating and managing infrastructure, patching servers, monitoring for uptime and availability, and building one-off mechanisms to provision users and audit their activity.” The Transfer Family is a fully managed service to accomplish this.
It can scan files for malware, personal identifying information and anomalies with customised and auto-triggered file upload workflows. Errors in processing files can be automatically handled with failsafe modes as well. AWS says all this can be done with low-code automation.
AWS Transfer Family Managed Workflows lets users configure all the necessary tasks at once so that tasks can automatically run in the background. Read a Transfer Family FAQ to find out more.
Object
Amazon S3 Intelligent Tiering itself gets enhanced cost-optimisations with:
- No monitoring and automation charges for small objects;
- No need to analyse object sizes;
- No minimum storage duration for objects;
- No need to analyse an object’s expected life.
According to Villlalba, “Now that there is no monitoring and automation charge for small objects and no minimum storage duration, you can use the S3 Intelligent-Tiering storage class by default for all your workloads with unknown or changing access patterns.”
S3 Multi-Region Access Points provide a global endpoint in front of buckets in multiple AWS regions. They work across multiple AWS Regions to provide better performance and resiliency. This feature dynamically routes requests over AWS’s network, to the lowest latency copy of your data, increasing read and write performance by up to a claimed 60 per cent, and providing operational resiliency.
We understand that S3 Multi-Region Access Points rely on S3 Cross Region Replication to replicate the data between the buckets in the Regions chosen by a customer. The customer selects which data is replicated to which bucket. There are replication templates available to help simplify applying replication rules to buckets.
Villalba blogs: “You can now build multi-region applications without adding complexity to your applications, with the same system architecture as if you were using a single AWS Region.”
Block
EBS direct API snapshots now support any volume up to 64TB, increased from 16TB and equal to the size of the largest EBS io2 Block Express volume. Snapshots can be recovered to EBS io2 Block Express volumes for protection and for test and dev.
AWS says it has built the first SAN for the cloud with io2 Block Express instances providing hundreds of thousands of IOPS at sub-millisecond latency and fine ‘nines’ durability. It’s claimed to be good for SAP and Oracle ERP, SharePoint, MySQL, SQL Server, SAP HANA, Oracle DB, and NoSQL databases such as Cassandra, MongoDB and CouchDB.
Backup
AWS Backup is a fully managed service to initiate policy-driven backups and restores of AWS applications. AWS Backup Audit Manager provides customisable controls and parameters, like backup frequency or retention period for AWS backups. It provides evidence of backup compliance for data governance, continuously tracking AWS backup activities, audits backup practices and generates audit reports.
There are more details here.
Comment
The FSx for ONTAP facility will enable NetApp customers to use AWS much more easily. The File Transfer Family will enable other file-access and also object-access customers to do so as well. File tiering will lower the cost of longer-term file storage and the S3 tiering restriction removals will help with storing lots off smaller objects.
The upping of EBS snapshot capacity to 64TB is welcome as is the Backup Audit Manager. Altogether this set of announcements should help AWS to make progress in storing more of the world’s data.
