Startup QiStor claims it can accelerate storage engine key-value processing so much that two servers can do the work of 20, saving time, money, and reducing carbon emissions.
We were introduced to QiStor in February and have now learned more about its technology from CEO and co-founder Andy Tomlin, providing a clearer picture of its value proposition. It starts from the recognition that much database processing involves a software stack with a storage engine component.
This piece of software talks across a network link, using a protocol like NVMe KV, to the storage drive hardware. But having applications talk through file systems to SSDs, which have Flash Translation Layers (FTLs) to map file storage IO requests to the drive’s block structure, is intrinsically inefficient. At heart, these layered constructs perform the allocation of space and the tracking of data locations on the drive. It’s simpler and more efficient to do this in one place through a key-value (KV) interface. Tomlin said: ”We’re doing it in hardware with full knowledge of how it’s actually working in the device.”
QiStor is, in effect, replacing an SSD’s controller, or building an abstraction of a controller. Tomlin said: “I originally started the company to build a controller. But I abandoned that because just getting funding requires raising, ultimately, $100 million to build a controller, and no one was interested. So I figured out a cheap way to do this [and] we’re building an FPGA now.”
Tomlin says that key-value stores (KV stores) underlie NoSQL databases such as Aerospike, Amazon’s DynamoDB, Redis, Couchbase, LevelDB, and RocksDB. KV storage engine processing is computationally intensive and needs offloading from host server CPUs as it’s a waste of their resources and best handled by dedicated hardware.
Vector databases such as Milvus, Pinecone, and Vespa are used in large language model (LLM) processing for training and inferencing. Their operation can be accelerated by QiStor’s technology, reducing the necessary server infrastructure and speeding LLM responsiveness.
Tomlin says most large scale web applications today are built on key-value storage. He is firmly in the hardware acceleration and offload camp, strongly established by GPUs offloading graphics processing from x86 CPUs, and encompassing data processing units (DPUs) such as Intel’s IPU and Nvidia’s BlueField devices, smart NICs, and dedicated AI hardware. This kind of hardware acceleration is becoming mainstream.
He believes that hardware acceleration is needed for KV store software engines as well, and that relatively simple FPGAs can be used for this; Xilinx FPGAs, for example. Such FPGAs could be housed in database servers fitted in service provider datacenters, public clouds, or on-premises datacenters, and consumed through a Platform-as-a-Service (PaaS) facility.
Amazon EC2 F1 instances use FPGAs to enable delivery of custom hardware accelerations, and QiStor storage engine operations run on them as well as on Xilinx FPGAs.
QiStor is developing FPGA programs to provide functionality needed by the storage engine. The databases higher up the stack “talk” to the engine through existing access methods, Object-Relational-Mappers (ORMs) and libraries. It talks to the drive through a standards-based NVMe-KV interface. Tomlin says of the databases: “Most of the databases today already have plug-in storage engines … The idea is that the customer, the service, effectively, will run their databases, and they’ll just think they’re running on RocksDB as their back end.”
Tomlin says QiStor does not want to sell hardware. Other parties can do that, with QiStore producing the code that runs on or in it.
An initial task was to develop coding tools that can be used to write FPGA code, the hardware description language (HDL) describing the logic implemented on the FPGA. HDL code is converted by a synthesis tool into a netlist detailing the logic gates and interconnections needed on the FPGA, and this is converted into a bitstream, which is transferred to the FPGA. Now QiStor has developed its FPGA coding toolset, it can produce its code faster.
All this seems complex but it is more practicable and simpler than developing a storage engine-specific ASIC or other hardware, such as the Pliops XDP card.
Tomlin and QiStore are doing more than this. They are replacing an SSD’s block-based low-level structure with a KV storage system so that no traditional FTL is needed to map the KV data constructs to a target SSD’s block structure. He reckons this is inherently more efficient, with payoffs in eliminating FTL processing, simplifying garbage collection (recovery of deleted cells), and reducing memory requirements for SSD metadata. There is no need to use complex zoned infrastructure and the drives, with less write amplification, last longer.
Couple this with hardware acceleration and multiplicative performance advantages result.
A QiStor prototype demonstration ran with a 1 PB Aerospike database. When instantiated on x86 servers, this needed 20 x 60 TB servers, with 1,200 TB total capacity, delivering 753,000 write IOPS, and 3,017,000 read IOPS. The latency was 1 ms and the modeled three-year total cost of ownership (TCO) was $4.17 million.
This was set against just two servers with QiStor FPGAs and 1,000 TB of total capacity. They delivered 1.5 million write IOPS and 6 million read IOPS with a 250 μs latency. The modeled three-year TCO was less than $1 million.
Because fewer servers and SSDs were needed, the 58.2 kW of power needed for the pure x86 server setup was reduced to 5.8 kW for the pair of QiStor servers – a saving of 325 metric tons of CO2 per year.
It is becoming a truism that datacenters using GPU servers need lots of power, and the datacenter electricity budget left over for x86 servers, storage, networking, and cooling is much lower than in pre-GPU server days. This situation is likely to get worse. By lowering the amount of electricity needed to run petabyte-scale NoSQL databases ten times more efficiently, QiStor can make better use of that power budget.
Using QiStor technology can make NoSQL databases, including vector databases, run much faster on far fewer servers, saving cash, electricity, and carbon emissions. It’s the sort of technology customers, once they have adopted it, will see no going back. Many can’t wait to check out a beta test product.
