VMware has created a scalable cloud file service for its VMware Cloud on AWS, taking on NetApp’s FSx for ONTAP in provisioning storage for its VMs and Tanzu containers.
The Cloud Flex Storage service is based on VMware’s July 2020 acquisition of Datrium and its disaggregated hyperconverged storage software tech. The news was announced in a VMware blog by Oliver Romero, the senior product marketing manager for the software.
He writes: “Cloud Flex Storage is built on a mature, enterprise-class file system that has been developed and production-hardened over many years, dating back to Datrium’s DHCI storage product.”
This file system, already used in VMware’s Cloud Disaster Recovery service, has a two-tier design that allows for independent scaling of storage performance and capacity.
The Cloud Flex Storage service is delivered as a service, natively integrated into the VMware Cloud. NFS data store capacity is provisioned with clicks from the VMware Cloud Services Console.
Users can provision up to ~400TiB (439.805 TB) of storage capacity with ~300K IOPS per file system. Cloud Flex Storage “provides always-on data services such as at-rest encryption, deduplication, compression, and data integrity checks.”
VMware says users get a storage SLA of 99.9 percent availability within an AWS availability zone and regional S3 replication for data durability.
It has, Romero says, a predictable pay-as-you-go consumption model or a term subscription billed at a predictable $/GiB rate.
A blog by Sazzala Reddy, VMware’s Cloud Storage CTO and Datrium co-founder and ex-CTO, provides more information about Cloud Flex Storage’s internal design. Reddy says it has a Log Structured File System (LFS) design with app-centric snapshots, instant cloning, and immutability, making it a multipurpose file system.
For example, “OLTP databases need very high random IO performance, OLAP databases need high sequential IO performance, and file shares have a mix of 20%/80% hot/cold data.” Ironically “The original idea for LFS was first proposed in 1992 by Mendel Rosenblum, who also happens to be the founder of VMware.”
Cloud Flex Storage has “a 2-tier design: a capacity tier, and a performance tier. All data is stored in S3 (capacity-tier), and we use EC2 with NVMe for IO performance (cache-tier). This 2-tier design helps us decouple storage capacity from storage performance and be able to dynamically scale them independently.”
“All incoming data is converted to large ~10MB sequential segments, and these large segments are stored as S3 objects, and S3 is excellent at large sequential IOs. This allows data to be stored in S3 at high speed.”
Reddy says: “Cloud Flex Storage will be offered as supplemental storage to vSAN for capacity-heavy workloads.”
Romero adds: “Longer-term, our vision for VMware Cloud Flex Storage is to deliver an enterprise-class storage and data management as-a-service for the multi-cloud.”
That’s quite clear in its implications. We should see the Cloud Flex Storage product available for Azure and the Google Cloud Platform and on-premises.
It will have added data management features, with Romero saying: “We plan to support a broad range of workloads by enabling multi-dimensional scaling of compute, storage performance, and storage capacity – while delivering a seamless and consistent data management experience across clouds.”
VMware Cloud Flex Storage is now generally available.