Freshly-minted startup ChaosSearch has devised a way to analyse, index and search log data in the S3 part of Amazon’s cloud.
It calls itself CHAOSSEARCH but this all-caps shrieking is painful to read so we’ll be restrained and use ChaosSearch. The company was founded in 2017 and has raised $10m to develop its idea – index S3 log datasets with built-in schema detection, normalisation, and compression algorithms.
It says its Data Edge technology turns an S3 data set into a searchable ElasticSearch cluster. It operates as a fully managed, secure service on AWS backed by S3 as a data store. This gives customers the ability to store, search and query the data within their own S3 environment.
Thomas Hazel, ChaosSearch CTO and co-founder, says: “Historically, companies could not scale their log and event analytics at an effective cost, or even consider AWS S3 as the only backing store.
“ChaosSearch has [built] a cloud first, high-performing database that enables text search and relational analytics directly on the customer’s S3 account.”
Features include:
- Turns AWS S3 into a easy-to-use, searchable ELK-compatible logging solution with no changes required to existing ELK-compatible applications
- Automates the discovery, organization and indexing of log and event data types and sources
- Visualises hot and warm log clusters from a single Kibana instance
- Offers unlimited data retention, within customer’s own AWS S3 infrastructure
- Enables integrated text search and relational queries from a single platform
- Enable Machine Learning-based insights and predictive analysis with access to massive data sets over time
- Multi-tenancy and account management
- Built-in data cataloguing and organising services for S3 management
- Elasticsearch-compatible API with fully integrated Kibana visualisation
- Delivers relational and text-based analysis through a single console
- Enables integrated text search and relational queries
- Provides high performance at low cost with pricing starting as low as $20 per GB daily volume
The company claims customers can be up and running in minutes at a fraction of the cost of running their own Elasticsearch cluster or ELK Stack.
The acronym ELK stands for a trio of open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash ingests data simultaneously from multiple sources simultaneously, transforms it, and then sends it to Elasticsearch. Kibana is a data visualisation tool producing charts and graphs for Elasticsearch.
CEO and co-founder Les Yetton blogs: “We’ve extended the S3 REST API and exposed the Elasticsearch API — and that’s just for starters (MongoDB, JDBC, SQL coming soon, and other clouds as well). Primary use cases today are log management, threat hunting, and application and security analytics. Best of all, the pricing is disruptive, starting at 50 per cent [of] the cost of traditional solutions.”
An online document tells you how to use the ChaosSearch ELK service.
