Object storage has a well-deserved reputation for slowness but two high performance startups are on a mission to change this.
They are California startup MinIO, founded in 2014, and French rival OpenIO, founded in 2015. The companies have tuned their IO stacks and embraced parallelism to smoke the object storage competition. And they are playing performance benchmark leapfrog with each other.
This month it is the turn of MinIO, which announced its object storage has passed the terabit per second IO rate. Last month OpenIO achieved 1.372 Tbit/s throughput (171.5GB/sec), using an object storage grid running on 350 commodity servers.
Now MinIO is in the same ballpark with a faster write speed and equal read speed than OpenIO. In its latest benchmark reveal, the company went past 1.4Tbits/sec for reads, using 32 nodes of AWS i3en.24xlarge instances each with 8 NVMe drives, making a total of 256 NVMe drives. According to AWS these instances offer the lowest price per GB of SSD instance storage on Amazon EC2.
The i3en.24xlarge instances supply up to 100 Gbit/s networking bandwidth, 96 vCPUs from Xeon Skylake processors with a sustained all-core Turbo CPU clock speed up to 3.1 GHz and 768 GiB of memory. An Elastic Fabric Adapter provides low and consistent network latency. The instances deliver up to two million random IOPS at 4 KB block sizes and up to 16 GB/sec sequential drive throughput.
Test run
The MinIO test used the S3-benchmark by wasabi-tech which involved a single client accessing a single endpoint.
MinIO delivered 183.2 GB/sec (1.46 Tbit/s) on reads and 171.3 GB/sec (1.37 Tbit/s) on writes. Each node contributed 45.8 Gbit/s and 43 Gbit/s on average for reads and writes, respectively. The write speed more or less equalled OpenIO’s throughput of 1.372Tbit/s while the read speed exceeded it.
In a blog detailing the test run MinIO engineer Sidhartha Mani wrote: “The network was almost entirely choked during these tests. In this setup, MinIO shared the same network for server-client and server-server communication. The throughput can nearly be doubled if a dedicated network was available for internode traffic.”
That means MinIO thinks it could achieve 2.92Tbit/s read bandwidth – 366GB/sec.
MinIO has trumpeted its speed credentials recently, showing it was faster than Hadoop in August and beating AWS S3 with Spark and Presto in July.
MinIO NAS gateway
Gregory Touretsky, Infinidat senior director for product management, this week revealed MinIO’s software has a NAS gateway. It can accept S3 object access requests and send them as NFS requests to back-end storage, such as an Infinidat InfiniBox array which supports block and file access.
This MinIO NAS gateway effectively adds object storage access to Infinidat arrays. Touretsky said all InfiniBox functionality “including compression, replication, and space-efficient snapshots with WORM capabilities is available behind MinIO”. Several MinIO gateways can be used to increase throughput and provide high availability.
Blocks & Files thinks we will hear more about MinIO NAS gateways in coming months.
