StorageOS reinvents VSAN for containers

StorageOS, a UK startup, has released version 2 of its eponymous software, which effectively reinvents virtual SANs for containers.

Alex Chircop, CEO, said StorageOS 2 delivers production grade storage in increasingly complex large clustered Kubernetes deployments.

According to StorageOS, traditional storage arrays cannot handle the complexity of clustered deployments at scale. “Kubernetes users working with increasingly complex deployments require storage that delivers predictability for replication and failover,” Chirchop said in a statement. “Users are also deploying more mature Kubernetes environments resulting in a need for production-grade storage.”

StorageOS was founded by three storage-skilled execs at major financial institutions and a fourth exec who was involved enterprise storage. Such institutions are customers for mainstream enterprise storage arrays, virtually all of which have CSI plug-ins for Kubernetes – meaning they can provide storage for containers. Such institutions will also have much experience of VMware and knowledge of VSAN.

The company said paying customers include financial services and life sciences firms and service providers, which have made more than 3000 cluster installs since releasing the first version of its product.

StorageOS architecture

StorageOS aggregates the local disk storage in a cluster of servers (nodes) into a one or more virtual block storage pools. The cluster can be a traditional cluster or a hyperconverged system or a set of several clusters. The multi-cluster scenarios we can envisage include topologies such as a centralised storage cluster with satellites consuming the storage, and data replicated between the clusters for high-availability and disaster recovery

There must be a minimum of three nodes in the cluster. Storage in the pool is carved out into thinly provisioned virtual volumes. Each node has a storage container in which StorageOS is deployed. Application containers in the nodes mount and access these virtual volumes via the storage container.

Blocks & Files diagram of StorageOS’ architecture. Disk drives can also be SSDs ad it’s optimised for NVMe SSDs

Any container can mount a StorageOS virtual volume on any node, regardless of whether the container and volume are colocated on the same node or if the volume is remote. Applications may be started or restarted on any node and access volumes transparently, meaning on another nodes physical storage. Volumes are cached in DRAM to improve read performance and compressed to reduce network traffic.

This is, in effect, a virtual SAN set up expressly for containers. It has a control plane to set it up and maintain operation. The containers perform their storage IO, using a data plane.

StorageOS runs on physical servers, virtualized servers or in the public cloud. It interoperates with Kubernetes, OpenShift, Amazon EKS, Azure EKS, Rancher and Docker. A StorageOS cluster will typically map one-to-one to a Kubernetes or similar orchestrator cluster.

The StorageOS container runs on all nodes in the cluster where app containers need to consume storage. The etcd distributed key:value store is used to to maintain state and manage distributed consensus between nodes. This helps in recovery from node failure.

StorageOS V2.0

Storage OS V2.0 improves four aspects of the software. General performance has been accelerated and recovery time from a failure has been reduced. Synchronous replication has been added for high-availability, supporting up to five replicas of a primary volume.

Resiliency has been strengthened in large clustered environments that may experience more transient failures. A Delta Sync feature reduces the time to recovery on transient failures, allowing faster cluster convergence by only replicating the missed data to the node. It is designed to cope with unpredictable failure scenarios.

 Also the StorageOS container runs like any other application, with no dependencies on proprietary kernels, storage protocols or other layered services meaning customers don’t suffer from these kinds of lock-in.

V2.0 is designed to enable security at every layer of the stack, improving security with encryption in transit. Traffic between nodes is encrypted and authenticated.

A free developer edition of StorageOS V2.0 with 5TB is available and users can upgrade to Project and Platform editions for enterprise capabilities and comprehensive product support.


Blocks and Files asked StorageOS about competing products from Portworx, Kasten, Diamanti and mainstream storage array providers.

B&F: How is StorageOS better than/different from Portworx?

StorageOS: StorageOS differentiates itself in the market through a number of features:

  • Disaggregated Consensus for volume management – each StorageOS volume has a “mini-brain” capable of managing recovery and placement independently, effectively reducing the blast radius of any component failure.  This enables the highest levels of reliability for clusters at scale by distributing redundancy whilst improving the bandwidth and lowering latency for deterministic performance.
  • Delta Sync – reduces the time to recovery on transient failures, allowing rapid cluster convergence by only replicating the missed data to the node
  • Runs Anywhere – deployed as a container, all data services are optimized and integrated inline to the StorageOS data plane ensuring the lowest resource overhead and low latency performance.  The StorageOS container runs like any other application, with no dependencies on proprietary kernels, storage protocols or other layered services.
  • Secure by default – StorageOS enables security at every layer of the stack with  automated certificate management, secure endpoints and encryption of data between nodes.

B&F: How is StorageOS better than/different from Kasten?

StorageOS: StorageOS and Kasten are complementary and provide different services.

StorageOS is a software defined, cloud native storage platform, providing automation and self-service via a control plane and data services (such as replication, compression, encryption) through a data plane. Kasten is a data management product, providing, for example, backup and restore services that layer on top of other storage solutions.

B&F: How is StorageOS better than/different from Diamanti

StorageOS: StorageOS is a software-only solution and has no dependencies on hardware. Diamanti has a special focus on providing Kubernetes solutions in a hardware appliance form factor.

B&F: If I have a storage array with a CSI plugin (e.g. Infinidat) or strong Kubernetes support (NetApp) where is the benefit in using StorageOS?

StorageOS: StorageOS provides a number of benefits compared to a traditional storage system:

  • As a software defined solution, deployed as a container, StorageOS runs anywhere a container can – on-prem, in the cloud or any hybrid.
  • StorageOS is Application Centric – storage volumes are provisioned to an application not a server/node, allowing storage to be able to follow an application as it scales, grows and moves between platforms. 
  • Cloud Native Scalability – Cloud Native Workloads are very demanding, supporting thousands of containers, advanced workflows and dynamic scaling. StorageOS performs provisioning and cluster operations in milliseconds.


StorageOS performs provisioning and cluster operations in milliseconds, which does not seem that fast when compared with the sub-millisecond latencies that all-flash NVMe arrays deliver.

Wouldn’t any all-flash array with a CSI plug-in be as fast?

Dell EMC VP for Technology (PowerStore, XtremIO, VxFlexOS) Itzik Reich says: “Of course [but] it has nothing to do with the SSD drives, it’s all about the control path. What matters is how fast the mapping of a volume (and in some cases, the time it takes to format the FS).”

We asked storage consultant Chris Evans for the reasons an enterprise storage customer would use StorageOS software when they can get all-flash arrays with CSI plugins or VMware/Tanzu.

He said: “I guess the question is why use any SDS solution.  Flexibility, cost, lock-in, these are the most obvious, however, I think with container integrated solutions the benefits are a bit more nuanced.  

“I can integrate provisioning and other functionality into my application workflow in a way that operates way better than using a fixed array.  I can make that infrastructure truly portable – move it to the cloud, another platform, all abstracted with nothing more than Kubernetes and some local disks. 

“It means I can build on-demand test environments and tear them down five minutes later etc., etc.  I doubt any storage array could cope with creating and destroying hundreds of volumes per hour (or more), whereas on StorageOS, those constructs are mainly in software on the same node as the application, so can be created/destroyed in milliseconds.”