Redis Labs bypasses the NoSQL storage bottleneck

The NoSQL database ocean has some 300 different databases sailing across it. Redis is a super-tanker in the key-value sector, with one million public cloud instances and 8,000 customers, including Uber and Twitter. (Other NoSQL sectors include document, columnar and graph).

Redis Labs supports and sponsors the open source NoSQL Redis (Remote Dictionary Server) key-value database. In a briefing with Blocks & Files, Redis Labs co-founder and CTO Yiftach Shoolman said the software is a database cache and store at the same time. 

Redis Labs was founded in 2011 in Israel, has some 350 employees and sells enterprise grade Redis software. It has received $145.6m in funding, including a $60m slug in February this year.

Use cases include ranking in the gaming world, message broking and managing queues and streams. As Red Hat is to Linux so, to an extent, is Redis Labs to Redis, Shoolman said. “They [Red Hat] sell mostly support. We sell mostly software… We don’t believe in headcount as a business model.” 

Speed

Redis runs on a cluster of nodes and provides fast cache-based data retrieval. This is because the database loads into main memory from disk at system start-up time, effectively using memory as a cache. Fresh data is written back to disk by snapshotting at regular intervals or by writing a background journal file. Data is sent to the persistent backing store every two seconds or faster. 

Redis is written in C and optimised for speed. It is five to 100 times faster than other databases, Shoolman claimed. When run in server DRAM Redis can produce five million database operations/sec from servers, he said. If it runs in Optane persistent  memory Redis can output one million database ops/sec and 100K to 400K if running off SSDs. All three ops/sec numbers are with less than 1msec latency.

Redis Labs CTO and co-founder Yiftach Shoolman talking on a video about Redis Labs and Intel Optane.

Redis Labs started working on Optane with Intel three years ago, as he explains in a video.

Raw structures exposed

A second attribute of Redis is that users operate on raw data structures. They issue commands, not queries, to the database engine. These specify operations to be applied to particular data types; the database exposes raw data and does not use secondary indices or other aggregation methods. This design also increases database access speed.

Redis supports various data types: strings, lists, maps, sets and sorted sets, bitmaps and spatial indices. V5.0 was released in October 2018 and added a stream data type. This allows storage of multiple fields and string values with an automatic, time-based sequence at a single key.

Comment

Redis Labs’s success is in part due to its memory caching speed. This remind me of another Israeli startup – Moshe Yanai’s Infinidat storage array business, which also uses memory caching for its speed advantage over all-flash arrays. For Redis Labs, avoiding storage IO except for background operations is the only way to keep server CPUs busy. Storage is a bottleneck and needs bypassing – which applies in a way to Infinidat as well.