On-prem and public cloud storage vendors can add information lifecycle management (ILM) features to their products – but that only confirms the need for a global, supplier-agnostic ILM capability. So says Komprise CEO Kumar Goswami.
Goswami has written one blog about whether customers should use public clouds, and another about optimizing cloud usage. We took these as starting points for a set of questions about Komprise’s place in the ILM area.
Blocks & Files: Komprise analyzes your file and object data regardless of where it lives. It can transparently tier and migrate data based on your policies to maximize cost savings without affecting user access. Can Komprise’s offering provide a way to optimize cloud storage costs between the public clouds; arbitraging so to speak between AWS, Azure and GCP?
Kumar Goswami: Komprise enables customers to test different scenarios by simply changing the policies for data movement and visualizing the cost impacts of these choices. By running such what-if analysis, customers can choose between different destinations and then make their decision.
And yes, we support AWS, Azure and GCP as destinations. Komprise also supports cloud-to-cloud data migrations. Komprise does not arbitrage across different options. Customers make these choices and Komprise gives them the analytics to make informed decisions and data mobility to implement their choices.
Blocks & Files: How would you view AWS’s S3 Intelligent Tiering and the AWS Storage Gateway with integration between the cloud and on-premise, offline AWS-compatible storage. What does Komprise offer that is better than this?
Kumar Goswami: AWS has a rich variety of file and object storage tiers to meet the various demands of data, with significant cost and performance differences across them. Most customers have a lot of cold data that is infrequently accessed and can leverage highly cost-efficient tiers like Glacier Instant Retrieval that are 40 percent to 60 percent cheaper.
Yet customers still want to be able to see the data in Glacier IR transparently and access it without rehydrating it up to a more expensive tier, and this requires a data management solution like Komprise that preserves native access at every tier with transparency from the original location.
Komprise manages the data lifecycle across AWS FSX, FSXN, EFS, S3 tiers and Glacier tiers and preserves file-object duality so you can transparently access the data as a file from the original source and as an object from the destination tiers – without rehydration – using our patented Transparent Move Technology (TMT). For customers with hundreds of terabytes to petabytes of data, they want the flexibility, native access and analytics-driven automation that Komprise delivers.
AWS S3 Intelligent Tiering will move data not used for 30 to 90 days to its own internal lower cost tier, but if you access the object, it will bring it back up to the expensive tier and keep it there for another 30 to 90 days before tiering it again. S3 Intelligent Tiering has its own internal tiers so you cannot use it to put data in the regular S3 tiers like Glacier Instant Retrieval for instance.
You also cannot directly access the tiered data from the lower tier without rehydration. Nor can you set different policies for different data, it automatically manages data within itself and is like a black-box. AWS S3 Intelligent Tiering is useful if you have small amounts of S3 data, you don’t have file data, you don’t have analytics-based data management and don’t require policy-based automation and are worried about irregular access patterns.
With Komprise you can migrate, tier and manage data at scale across the hybrid cloud to AWS. When tiering to S3, Komprise leverages AWS ILM to automatically tier data from higher-cost object tiers to lower-cost tiers including Glacier and Glacier IR using policies set within Komprise. This gives customers transparent access from the source while getting the cost benefits of ILM.
We also support options like AWS Snowball for customers who need to move data offline but still want transparent access. Most customers want to continue using their enterprise NAS environments like NetApp and Dell EMC as they tier data to AWS, and Komprise supports this. Komprise also supports tiering to Amazon S3 Storage Gateway which replaces a customer’s existing NAS with a gateway.
Blocks & Files: Azure’s File Sync Tiering stores only frequently accessed (hot) files on your local server. Infrequently accessed (cool) files are split into namespace (file and folder structure) and file content. The namespace is stored locally and the file content stored in an Azure file share in the cloud. How does this compare to Komprise’s technology?
Kumar Goswami: Hybrid cloud storage gateway solutions like Azure File Sync Tiering are useful if you want to replace your existing NAS with a hybrid cloud storage appliance. Komprise is complementary to these solutions and transparently tiers data from any NAS including NetApp, Dell, and Windows Servers to Azure. This means you can still see and use the tiered data as if they were local files while also being able to access the data as native objects in the cloud without requiring a move to a new storage system.
Blocks & Files: Google Cloud has lifecycle management rules in its storage buckets. What does Komprise offer that is better than this?
Kumar Goswami: Komprise leverages cloud lifecycle management policies and provides a systematic way to use these ILM policies on both file and object data with transparent access to the data from the original location as well as direct access to the data from the new location. So, Komprise adds transparency, file to object tiering and native access to any cloud ILM solution including that provided by Google.
Blocks & Files: NetApp’s BlueXP provides a Cloud Tiering Service that can automatically detect infrequently used data and move it seamlessly to AWS S3, Azure Blob or Google Cloud Storage – it is a multi-cloud capability. When data is needed for performant use, it is automatically shifted back to the performance tier on-prem. How does Komprise position its offering compared to BlueXP?
Kumar Goswami: NetApp BlueXP provides an integrated console to manage NetApp arrays across on-premises, hybrid cloud and cloud environments. So it’s a good integration of NetApp consoles to manage NetApp environments, but it does not tier other NAS data.
Also, NetApp’s tiering is proprietary to ONTAP and is block-based, not file-based. Block-based tiering is good for storage-intrinsic elements like snapshots because they are internal to the system – but cause expensive egress, limit data access and create rehydration costs plus lock-in for file data. To see the differences between NetApp block-based tiering and Komprise file tiering, please see here.
Blocks & Files: Komprise is layering a set of file-based services (cloud cost-optimization, global file index, smart data workflows) on top of existing on-prem filers and cloud-based file services. The on-prem filer hardware and software suppliers are making their software operate in the hybrid multi-cloud environment, such as NetApp and Qumulo and also Dell with PowerScale. As part of this they are adding facilities that overlap with Komprise features. NetApp’s BlueXP is probably the most advanced such service. How will Komprise build on and extend its relevancy as hybrid-multi-cloud file software and service suppliers encroach on its market?
Kumar Goswami: Customers want data storage cost savings and ongoing cost optimization. Customers want flexibility of where their data lives and customers want to run workflows and monetize their data. This demand is driving the market, and it is excellent validation of the Komprise vision and value proposition.
Storage vendors who stand to gain by having customers in their most expensive tiers are recognizing the demand and starting to offer some data management. But the storage vendor business model is still derived from driving more revenues from their storage operating system – they offer features for their own storage stack and tie the customer’s data into their proprietary operating system.
Customers want choice, flexibility and control of their own data, which is what we provide. And data management is much more than just data placement for efficiency. This is why customers want a global index of all their data no matter which vendor’s system the data lives on, why customers want native format of data without proprietary block-based lock-in, and why customers want data workflow automation across environments.
We are barely scratching the surface of unstructured data management and its potential. Think about the edge. Think about AI and ML. Think about all the different possibilities that no single storage vendor will be able to deliver. We are focused on creating an unstructured data management solution that solves our customers’ pain-points around cost and value today while bridging them seamlessly into the future.
Blocks & Files: There are many suppliers offering products and services in the file and object data management space. Do you think there will be a consolidation phase as the cloud file services suppliers (CTERA, Lucid Link, Nasuni, Panzura), data migrators (Datadobi, Data Dynamics), file services metadata-based suppliers (Hammerspace) ILMs such as yourself, and filesystem and services suppliers (Dell, NetApp, Qumulo, WekaIO) reach a stage where there is overlapping functionality?
Kumar Goswami: As data growth continues, customers will always need places to store the data and they will need ways to manage the data. Both of these needs are distinct and getting more urgent as the scale of data gets larger and more complex with the edge, AI and data analytics. The market across these is large, and data management is already estimated at about a $18B market.
Typically for such big markets, you will see multiple solutions targeting different parts of the puzzle, as the market supports multiple independent players. Our focus is storage-agnostic analytics-driven data management to help customers cut costs and realize value from their file and object data no matter where it lives. We see our focus as broader than ILM. It is unstructured data management, which will broaden even further to data services.
Blocks & Files: If you think that such consolidation is possible, how will Komprise’s strategy develop to ensure its future?
Kumar Goswami: Komprise is focused on being the fastest, easiest and most flexible unstructured data management solution to serve our customers’ needs. To do this effectively, we continue to innovate our solution and we partner with the storage and cloud ecosystem. We have and will continue to build the necessary relationships to offer our customers the best value proposition. Having built two prior businesses, we have learned that focusing on the customer value proposition and continually improving what you can deliver is the best way to build a business.
Blocks & Files: If consolidation is not coming do you think an era of more overlapping functionality is coming? And why or why not?
Kumar Goswami: In our experience, consolidation is tough to predict because it can be influenced by many conditions such as competitive positioning, market conditions and innovation pipelines. Given the large size and growing demand for unstructured data and given the fact that we are still early in the unstructured data explosion, there is enough market demand to support multiple independent players. It is hard for us to predict beyond that.
