Startup DeltaStream has bagged $10 million seed money for its idea of real-time stream data processing as-a-service.
The concept is to develop a serverless abstraction layer over real-time streaming data ingest services such as Apache Kafka and AWS Kinesis, with SQL query support and RDBMS-type management facilities for the data.
CEO and founder Hojjat Jafarpour told us: “Accessing fresh low-latency data provides a substantial competitive advantage to enterprises; however, the complexity of building and operating real-time applications prevents many companies from benefiting from this competitive advantage. We remove such complexity and make building and running real-time streaming applications simple and fast by providing a serverless database to manage and process all streams via SQL.”
He was previously a software engineer at Confluent and creator of the KSQL project there. His LinkedIn profile states: “KSQL is an open source streaming SQL engine for Apache Kafka. It provides a simple and completely interactive SQL interface for stream processing on Kafka; no need to write code in a programming language such as Java or Python. KSQL is open-source (Apache 2.0 licensed), distributed, scalable, reliable, and real-time. It supports a wide range of powerful stream processing operations including aggregations, joins, windowing, sessionization, and much more.”
There is a steadily increasing need to analyze event data, such as a credit/debit card transaction or online product query, in real time so as to detect and prevent fraud or make a product recommendation. There are a variety of data ingest engines and services such as Kafka, Confluent Cloud, AWS Kinesis, Azure Event hub, and GCP Pub/Sub. The task of developing any particular streaming data ingest, analysis, and management application is made more difficult by this wealth of underlying storage services and lack of any standard query-building and management frameworks.
Another problem area is provisioning compute services. DeltaStream is bringing RDBMS management ideas to real-time data and developing an SQL query building and running facility. It provides an abstraction layer over the ingest and storage services including data at rest in data lakes. Because it is serverless, it will provision and manage elastic compute services in the cloud.
Other companies such as Imply, Ocient, SingleStore, and Sqream provide stream data storage and analysis services and we asked Jafarpour how DeltaStream differed from them. His reply was: “DeltaStream provides stream processing as-a-service. The products you mentioned are ‘store and query’ systems where the data is stored and queried. On the other hand, DeltaStream queries are long-running continuous queries and keep processing data as data arrives. So these are two different processing models.”
DeltaStream is available on AWS in a private beta and the company is looking for testers (no charge). It plans to offer it on GCP and Azure soon. A blog provides a lot of background information.
