In a high-performance computing (HPC) world that’s historically focussed on parallel file systems for feeding data as fast as possible to a supercomputer’s multiple processors what does a supplier do when AI rewrites the demand environment and unstructured data feeds become as important as files?
Quobyte, which supplies its eponymous parallel filesystem software, says its software is “engineered for AI.” That’s because its software has been created under the direction of the by ex-Google programmers who founded Quobyte. They designed the software to scale simply and easily, and also cope with any sort of failure and keep running,
The two, CEO Björn Kolbeck and CTO Felix Hupfeld, say classical enterprise storage is, compared to fast, scalable and complex HPC software, simple, reliable, and slow. AI file/object storage needs to combine the advantages of both ends of this spectrum, by being simple – like operating a smartphone, bullet-proof reliable, fast, exabyte level scalable, and have hyperscaler efficiency.
Kolbeck says that Quobyte’s software runs on-premises both in core data centers and at edge sites and on x86, Arm or Nvidia Grace CPUs, in colocation sites, the public cloud or hybrid combinations of these locations, and it supports the CPU-attached storage on GPU servers; the tier zero storage.
The software is hardware-agnostic and views all hardware as potentially unreliable. Quobyte claims 100 percent, always-on availability, with software fault-tolerance at data center scale, and no need for maintenance windows. It is designed for multi-tenancy with strong isolation between tenant environments and policy-based data management.
Quobyte supports dynamic, on-demand and seamless scaling from four to multiple tens of thousands of storage servers. Data performance scales linearly to TBps levels and Quobyte says it has the fastest the and most efficient file system in the MLPerf 3D-Unet test, supporting the largest number of GPUs per client machine, and the lowest cost and energy consumption per performance unit.
Indeed, Quobyte claims it’s 33 percent faster than the closest competitive alternative and orders of magnitude better than the remaining ones.
The software, which has a single layer architecture, runs on standard Linux, with no kernel modules, no Data Plane Development Kit (DPDK) requirement, and standard, commodity, compute, storage and networking hardware; nothing exotic. The admin burden is slight and small teams, Quobyte says, can scale up to exabytes of capacity with no need for careful, lengthy planning. The file system is almost like a database, and has end-to-end observability, with metrics from all components and support for Prometheus’ time-series metrics and Grafana data visualization.
Quobyte was started up in 2013. We understand there was a 2014 A-round (7-figure euros raised) with High-Tech Gründerfonds (HTGF) and Target Partners contributing. Other investors include Alstin Capital, the Samsung Catalyst Fund and Yttrium. There was a B-round in 2016, raising an undisclosed amount, and nothing since then. Quobyte has not taken in VC funding since then, and is growing organically.
Customers include Airbus, Booking.com, Yahoo!Japan, Siemens and Zoox, which has a 70 PB system storing autonomous driving data usable by >40,000 clients.
Quobyte provides performant file/object storage system software for petabyte-to-exabyte scale workloads with no muss, no fuss and virtually no marketing budgets. Its competitors include Nvidia partners DDN, Pure Storage, WEKA, VAST Data, IBM Storage Scale, and also Dell, HPE, and NetApp. The market presence of these players in the storage market is huge, their engineering and go-to-market organizations substantial, and Quobyte has to prosper and grow pretty much on the basis of its software performance, scale, simplicity and reliability, and customer’s word-of mouth.
We hope see substantial AI-related, software functionality news coming later this quarter from Quobyte. Watch this space.
