Quobyte storage software takes a bite at HPC and machine learning

Profile Quobyte produces unified block, file and object storage for high-performance computing (HPC) and machine learning.

Its software runs on-premises and in the Google cloud and is used for data analytics, machine-learning, high-frequency trading, life sciences and similar petabyte-scale, fast-response applications. The company also has a TensorFlow plug-in.

Quobyte was established by CEO Björn Kolbeck and CTO Felix Hupfeld in Berlin in 2013. The pair studied computer science and did distributed storage systems research at Berlin’s Zuse Institute, where they developed the open source XtreemFS distributed file system as part of their PhD. They then worked for Google and this gave them the idea that they could build a Google-like scalable, distributed, parallel access and multi-protocol storage system for HPC customers.

Kolbeck and Hupfel thought that existing HPC storage was placed in restrictive silos such as file or object, was hardware-dependent, and needed a lot of administration. Their response was to develop high-performance and multi-protocol storage software that was independent of its hardware supplier and manageable by a small team.



Björn Kolbeck (left) and Felix Hupfeld

The software use commodity server hardware and is intended to replace several separate storage vaults while delivering good random and small file IOPS and large file throughput.

Competition

Quobyte is a small company. To give an indication of size, the company lists a dozen customers on its website and has 19 employees, according to LinkedIn profiles. Alstin Capital, an investor, valued Quobyte at $35m in 2018 and describes it as an early growth company. The funding history is a little opaque but the last capital raise was in 2016. According to one source, the company is exploring a fresh round.

As an HPC storage software supplier, Quobyte competes with Ceph, DDN, Intel DAOS, Lustre, Panasas, Qumulo, IBM’s Spectrum Scale and WekaIO, amongst others. Compared to most of these, Quobyte is a minnow without substantial backing.

Open source Ceph, which also supplies file, block and object access access storage, has an underlying object store, with file and block access layered above that. Kolbeck and Hupfeld thought they would get better performance from a file system base with block and object access layered on it.

IBM’s Spectrum Scale has no block storage and needs fairly complex tuning and administration. Panasas PanFS software has recently become independent of its hardware, and has high-performance but this is limited to file access. DDN HPC storage requires DDN hardware.

Quobyte technology

Quobyte has developed a Data Centre File System or Unified Storage Plane – two names for the same thing – which supports POSIX, NFS, S3, SMB and Hadoop file and block access methods.

The software has a distributed parallel, fault-tolerant, highly scalable and POSIX-compliant file system at its heart. This links Quobyte nodes (servers) with Quobyte clients in accessing servers, which make remote procedure calls (RPCs) to the Quobyte servers. 

The servers are nodes in a Quobyte storage cluster. Each node’s local file system handles block mapping duties using SSDs and HDDs. The system uses a key:value store designed for file system use and metadata operations are not part of the IO path.

Quobyte software can host any number of file system volumes which can be assigned to tenants. Various system policies control data placement, tiering, workload isolation, and use of partitions. The software has built-in accounting, analytics and monitoring tools plus built-in hardware management. It also supports cloud-native workloads with Docker, Mesos and Kubernetes and supports OpenStack and a Google TensorFlow plug-in.

Quobyte’s software operates in Linux user mode and not privileged kernel mode. This avoids context switches into kernel mode and means the software is safer as it can’t execute potentially damaging kernel mode instructions.

Three basic services

The software has three basic services; Registry Service to store cluster configuration details, Metadata Data Service for file access data and Data Service for actual file IO. One Data Service operates per node and can handle hundreds of drives. There can be hundreds or more data service nodes.

The metadata service is deployed on at least four nodes and can scale to hundreds of nodes. Placing metadata on NVMe SSDs provides the lowest latency accesses.

A client can issue parallel reads and writes to the servers. All clients can access the same file simultaneously. Byte-range or file-level locking is supported across all protocols that support locks to prevent corruption from multipole writes to the same data.

Get more information from a Quobyte white paper.