HCI is a fairly mature market and we think it should be simple enough for tech analysts to agree on what vendors to classify as HCI vendors. Not so fast.
Forrester has published a Wave report looking at hyperconverged infrastructure (HCI) systems, with Nutanix top of the HCI pops, VMware second and Cisco in third place. However, it disagrees with Gartner and Omdia on who the HCI suppliers are.
Forrester defines HCI vendors as Challengers, Contenders, Strong Performers, and Leaders. Here is its HCI Wave chart, in which the size of vendor circle represents market presence.
Nutanix, VMware and Cisco occupy the Leader’s quadrant. HPE leads the Strong Performer groups, with Huawei next, followed by Microsoft (Azure Stack), Scale Computing, and Pivot 3.
DataCore is a Contender, close to the boundary with Strong Performers, followed by Red Hat.
Analyst HCI confusion
There are big differences between the Forrester HCI and the recent Omdia Decision Matrix for HCI. Omdia left out Nutanix and Huawei, which were asked to – but didn’t – contribute data, and Microsoft. Omdia includes Fujitsu, Hitachi Vantara, and Lenovo.
Gartner’s HCI Magic Quadrant is at odds with both Forester and Omdia. The analyst firm omits NetApp, while including Huayun Data Corp, Sangfor Technologies, StarWind and StorMagic as ‘Niche Players’ (which is the Forrester equivalent of ‘Challengers’).
A blast from the recent past
We have unearthed a 2016 Forrester HCI Wave for IT history buffs.
It shows EMC and VMware separately, and also SimpliVity and HPE as separate vendors. Atlantis Computing, GridStore and Stratoscale have exited the market since then and NetApp had no HCI presence in 2016.
SimpliVity was almost up with Nutanix but now, in the arms of HPE, has fallen back. The EMC-VMware-Dell combo has progressed to second place, with Cisco rising to third. Buying Springpath was a good move for the networking giant.
In addition, we emailed the company some questions about its storage strategy and here are the answers provided by Tom Black’s staff. The Q&A is lightly edited for brevity and is also available as a PDF. (The document also contain’s HPE’s summary of its storage product line-up, which comrpises the Intelligent Data Platform, InfoSight, Primera, Nimble Storage, XP, MSA SAN Storage, Nimble Storage hHCI, SimpliVity, Cloud Volumes and Apollo 4000.)
Blocks & Files: Does Tom Black look after HPC storage? I’m thinking of the ClusterStor arrays and DataWarp Accelerator that were acquired when HPE bought Cray. If not will there be separate development or some sort of cross-pollination?
HPE: No, HPC and HPC storage are part of an integrated group in a separate business unit. HPC is run as a segment with integrated R&D and GTM.
Blocks & Files:IBM and Dell are converging their mid-range storage products. Does HPE think such convergence is needed by its customers and why or why not? Will we see, for example, a converged Primera/3PAR offering and even a converged Primera/3PAR/ Nimble offering?
HPE: We believe that businesses need workload-optimized systems to support mid-range and high-end requirements. Systems that are designed to deliver the right performance, resiliency, efficiency, and scale for the SLA and use case.
HPE Primera.
With the introduction of HPE Primera last year, HPE consolidated its primary storage portfolio with one lead-with mid-range storage platform – HPE Nimble Storage, and one lead-with high-end storage platform – HPE Primera. Our HPE 3PAR installed base has a seamless upgrade path to HPE Nimble Storage and HPE Primera. (Comment: We didn’t know that 3PAR arrays could be upgraded to Nimble arrays.)
Blocks & Files:Will the XP systems remain a key aspect of HPE’s storage portfolio and, if so, how are they distinguished from the Primera line?
HPE: Yes, HPE XP Storage is a key aspect of HPE’s storage portfolio. It is our high-end platform for mission-critical workloads with extreme scale requirements, traditional use cases like Mainframe, NonStop, OPVMS and Cobol-based environments, and XP install base customers. HPE Primera is our lead-with high-end platform for mission-critical applications to support virtualized, bare metal, and containerized applications.
Blocks & Files: I believe HPE is the only supplier with both HCI and dHCI systems in its range. How does HP position its SAN storage, its HCI systems and its dHCI systems?
HPE: In the last several years, HCI has offered small and medium businesses a new value proposition, delivering infrastructure simplicity with VM-centric operations and automation – and therefore HCI (and HPE SimpliVity) is performing very well in that segment.
We’re hearing from our enterprise and mid-market customers that they want the simplicity of HCI but are unable to adopt it as they’re giving up too much to gain the agility and simplicity. This is where we position HPE Nimble Storage dHCI – a disaggregated HCI platform that uniquely delivers the HCI experience but with better performance, availability, and scaling flexibility than HCI for business-critical applications and mixed-workloads at scale.
Nimble dHCI.
In addition, in the enterprise core data center and service provider market, organizations here have the largest scale requirements and strictest SLA for performance and availability for their mission-critical applications. The right solution for them is high-end storage/SAN, so we position HPE Primera with HPE Synergy here as the core consolidation platform for mixed-workloads at scale.
Blocks & Files: What is the role of storage in HPE’s composable systems? Will all the SAN, HCI and dHCI systems evolve towards composability?
HPE: HPE has been a pioneer in delivering composability. Companies once forced into overprovisioning infrastructure are now able to create fluid pools of compute, storage, and network resources. Composability is designed for any application, at any SLA, and at any scale because all it takes is a single line of code to gather and deploy the IT assets needed to run any application, anywhere, at any scale.
From a storage perspective, our focus is to continue to deliver infrastructure that is fully automated, optimized and integrated with intelligence and composability. We drive a cloud experience on-premises with a strong foundation of composability on intelligent infrastructure for any workload. Bringing together the intelligence from an intelligent data platform with the composability of HPE Synergy, IT organizations have the absolute flexibility to support any application with the agility of the cloud in their on-premises data center—delivered as a service to scale to meet ever-changing business needs.
We fundamentally believe in flexibility and the need for workload-optimized systems designed for the use case. Composability – with the ability to support any applications across bare metal, virtualization, and containers – makes the most sense in large scale, enterprise core data center environments. For organizations who don’t need this level of resource composition and flexibility, and perhaps are exclusively needing to support virtual machines, HCI – with a software-defined framework specifically for virtual machines – will make the most sense for them.
Blocks & Files: HPE is not perceived as a mainstream supplier of its own file storage technology. It partners widely with suppliers such as Qumulo, WekaIO and others. Long term should HPE have its own file storage technology or continue to set as a reseller/system integrator of other supplier’s technology?
HPE: With unstructured data, use cases are continuing to evolve. Here’s the breakdown of our view of use cases and how we address them with our owned IP and partner solutions:
AI/ML – HPE Ezmeral Container Platform, which is inclusive of HPE Ezmeral Data Fabric (MapR), enables data scientists and data analysts to build and deploy AI/ML pipelines to unlock insights with data.
Distributed, scale-out enterprise-wide data fabric for edge to cloud – HPE Ezmeral Data Fabric (MapR) is a leading scale-out filesystem with enterprise reliability and data services, and one of the only file systems that offers batch processing, file processing, and streams processing.
Analytics, Medical Imaging, Video Surveillance, etc – HPE Apollo 4000 in partnership with scale-out software partners such as Qumulo (file) and Scality (object) provides a great foundation for these use cases.
HPC – We have market-leading HPC solutions (i.e. ClusterStor) from our acquisition of Cray
Like any market, we will continue to monitor the evolution of customer use cases and ensure we can help them address the use cases with differentiated solutions.
Blocks & Files:HPE is not perceived as a mainstream supplier of its own object storage technology. It partners with suppliers such as Cloudian and Scality. Long term should HPE have its own object storage technology or continue to set as a reseller/system integrator of these other supplier’s technology?
HPE: HPE will continue to monitor the market for object storage. Scality is the strategic HPE partner for massively scalable multi-cloud unstructured data stores – including S3 compatible object storage. The HPE partnership with Scality is global and backed by joint go-to-market. This is all in addition to a global reseller agreement through which HPE offers Scality on our mainstream HPE price list.
With unstructured data, use cases are continuing to evolve. Here’s the breakdown of our view of use cases and how we address them with our owned IP and partner solutions:
AI/ML – HPE Ezmeral Container Platform, which is inclusive of HPE Ezmeral Data Fabric (MapR), enables data scientists and data analysts to build and deploy AI/ML pipelines to unlock insights with data.
Analytics, Medical Imaging, Video Surveillance, etc – HPE Apollo 4000 in partnership with scale-out software partners such as Qumulo (file) and Scality (object) provides a great foundation for these use cases.
HPC – We have market-leading HPC solutions (i.e. ClusterStor) from our acquisition of Cray
Like any market, we will continue to monitor the evolution of customer use cases and ensure we can help them address the use cases with differentiated solutions.
Blocks & Files:How will HPE evolve its storage offer at Internet Edge sites with a need, perhaps, for local data analytics?
HPE: Enterprise edge, as we refer to the edge, is becoming more and more strategic for customers and an opportunity for infrastructure modernization. It can be a small site in a hospital or oil rig, or a distributed enterprise with remote offices/branch offices e.g. for banks or retail. Key characteristic is often no on-site resident IT staff and therefore the ability to remotely deploy, manage, operate, upgrade is really important.
We are seeing a growing market opportunity to leverage HCI at the enterprise edge. The simplified experience of a hyperconverged appliance addresses edge-specific requirements, including the need to power edge applications in a highly available, small footprint, protect data across sites, and facilitate entire lifecycle management remotely without local IT presence. HPE SimpliVity is our strategic platform to address this need with customers with numerous use cases and wins.
We also provide our customers the end-to-end lifecycle with HPE Aruba and our HPE EdgeLine servers to service all customer needs at the edge.
Blocks & Files: How is HPE building a hybrid cloud storage portfolio with data movement between and across the on-premises and public cloud environments?
HPE’s approach hinges on providing a true, distributed cloud model and cloud experience for all apps and data, no matter where they exist – at the edge, in a colocation, in the public cloud, and in the data center. Our approach helps customers modernize and transform their applications and unlock value across all stages of their lifecycle where it makes the most sense.
Our hybrid cloud storage portfolio: 1) on-premises private cloud, 2) enterprise data services across both on-premises private cloud and public cloud, and 3) seamless data mobility between on-premises private cloud and public cloud.
HPE delivers a seamless experience by extending our storage platforms to the public cloud with HPE Cloud Volumes. HPE Cloud Volumes is a suite of enterprise cloud data services that enables customers, in minutes, to provision storage on-demand and bridges the divide and breaks down the silos that exist between on-premises and public cloud.
Unlike other cloud approaches, we’re uniquely bringing customers a fresh approach to backup, with the ability to put backup data to work in any cloud with no lock-ins or egress charges when recovering that backup data from HPE Cloud Volumes Backup.
Blocks & Files: How does HPE see the market need for SCM, NVMe-oF, and Kubernetes and container storage developing. Will these three technologies become table stakes for each product in its storage portfolio?
HPE – SCM
HPE 3PAR Storage pioneered the use of SCM, showcasing its performance benefits as early as 2016. Next, in 2019, we were the first vendor to introduce SCM as a caching tier on the 3PAR high-end platforms for extreme low latency requirements.
While other storage vendors have since caught up, SCM remains a niche use case because it is cost prohibitive to deploy as a persistent storage tier.
Our focus has been to democratize SCM and delivering it more widely through broader use cases. This is why this past June we introduced SCM for HPE Nimble Storage to deliver SCM-like performance for read-heavy workloads with 2X faster response times with an average of sub 250 microseconds latency at near the price of an all-flash array (~10-15% price premium depending on the HPE Nimble Storage all-flash array model).
Over time, we believe SCM will do the job of what SSD did 10 years ago, by providing a latency differentiation performance tier within AFAs.
HPE -NVMe-oF
We believe this is the next disruption in the datacenter fabrics. As with all fabric changes, this transition will happen gradually. It will take time for host ecosystems readiness, as well as customers’ willingness to make substantial changes in their fabric infrastructure.
We are at the early stages of host ecosystem readiness. For example, Linux distributions are ready. VMware 7.0 has brought some level of fabric support (with the next release scheduled to go further).
For us, we know this disruption is coming, and are working hard with our partners to develop an end to end NVMeoF solution. We are committed to being ready once the market (partners and customers) is as well.
HPE – Kubernetes and Container Storage
As customers continue to modernize their apps, they need to modernize the underlying infrastructure and we see a clear shift toward container based applications. Kubernetes is the defacto open source standard.
HPE CSI Driver for Kubernetes for HPE Primera and HPE Nimble Storage enables persistent storage tier for stateful container-based apps, and for deploying VMs and containers on shared storage infrastructure. HPE CSI Driver for Kubernetes includes automated provisioning and snapshot management. Over time, we will extend CSI (which is an open specification) to deliver additional value-added capabilities for customers.
In addition to HPE Storage platforms for container support, HPE offers its Ezmeral Container Platform, to help customers modernise and deploy non-cloud native and cloud native applications. (See answers above.)
IBM Research and Radian Memory Systems have found SSD zoning can deliver three times more throughput, 65 per cent more transactions per second and greatly extend an SSD’s working life.
SSD zoning involves setting up areas or zones on an SSD and using them for specific types of IO workloads, such as read-intensive, write-intensive and mixed read/write IO. The aim is to better manage an SSD’s array of cells in order to optimise throughput and endurance. A host to which the SSD is attached manages the operation of the zones and relieves the SSD’s Flash Translation Layer (FTL) software of that task.
For its benchmarks, IBM Research used SALSA (SoftwAre Log Structured Array), a host-resident translation layer, to present the SSD storage to applications on the server as a Linux block device. Radian supplies SSD management software to suppliers and OEMs. The test SSD was an RMS-350, a commercially available U.2 NVMe SSD with Radian Memory Systems’ zoned namespace capability and configurable zones.
SALSA controls data placement while the SSD abstracts lower level media management, including geometry and vendor-specific NAND attributes. SALSA also controls garbage collection, including selecting which zones to reclaim and where to relocate valid data, while the SSD carries out other NAND management processes such as wear-levelling in zones.
By residing on the host instead of the device, SALSA enables a global translation layer that can span multiple devices and optimise across them. The FTL of an individual SSD does not have this capability.
IBM and Radian ran fio block-level benchmarks and recorded throughput rising to 301MB/sec from 127MB/sec compared to the same SSD relying solely on its own FTL with no SALSA software. There was a 50-fold improvement in tail latencies (latencies outside 99.99 per cent of IOs) and an estimated 3xvimprovement in flash wear.
A MySQL/SysBench system-level benchmark showed 65 per cent improvement in transactions per sec, 22x improvement in tail latencies and, again, an estimated 3x Flash wear-out improvement.
This is just one demonstration and used a single SSD. A problem with host-managed zoned SSDs is that software on the host has to manage them. This software is not included in server operating systems, and so application code needs modifying to realise the benefits of the zoned SSD. Without a standard and widespread zoned SSD specification, there is no incentive for application suppliers to add zoned SSD management code to their applications.
IBM’s SALSA interposes a zoned SSD abstracting management layer between the zoned SSDs and applications that use them. That means little or no application code has to be changed. You can read the IBM/Radian case study to find out more.
Ideally, SALSA or equivalent functionality should be open-sourced and added to Linux. This would make it easier for applications to use zoned SSDs. A 3x improvement in throughput and wear, together with tail latency and transactions/sec improvements are worthwhile gains. But without easy implementation, the technology will remain niche.
Hyperscalers and enterprise data centres helped Seagate grow revenues in its Q4, despite a pandemic-related drop in demand in several market areas.
Revenues climbed 6.3 per cent year-on-year to $2.52bn and net income was $166m for Q4 ended 3 July. Net income in the same quarter last year was $983m, thanks to a hefty boost from a one-time $702m tax benefit. Full fy20 revenues were $10.51bn, up one per cent on last year’s $10.39bn.
CEO Dave Moseley’s prepared remarks were a bit half and half: “The June quarter was led by robust cloud and data centre demand, which drove record exabyte shipments for our nearline mass capacity drives and strongly contributed to the Company’s overall revenue and solid free cash flow generation.
“However, continued economic uncertainty and COVID-19 related disruptions impacted demand in other key end markets including video and image applications, mission critical and consumer markets and also impacted profitability as we incurred higher logistics and labor costs which together weighed on our fourth quarter results.”
He told analysts on a conference call:
“Economic uncertainty and the extension of restrictive measures began playing out in other markets as the quarter progressed, causing smaller and midsized enterprise customers to scale back their IT budgets, municipalities to delay certain projects and consumers to spend more selectively. These macroeconomic factors ultimately impacted sales of our video and image applications, and led to a steeper than seasonal decline for our legacy products.”
Moseley was more effusive about the full year: “Fiscal 2020 marked a strong year of progress as we grew mass capacity storage revenue by 25 per cent and increased our overall revenue. We executed our technology roadmap, strengthened our balance sheet and delivered on our capital return program.”
Q4 summary
Operating margin 10.6 per cent
Cash flow from operations $388m
Free cash flow $274m
Fy20 summary
Operating margin 12.4 per cent
Cash flow from operations $1.7bn
Free cash flow $1.1bn
Returned $1.5bn to shareholders through dividends and share repurchases
As expected, mass capacity nearline disk drives revenues grew 35 per cent year-on-year and accounted for 63 per cent of total disk drive revenue. Shipments were 79.5EB, up 128 per cent year-on-year.
Full year nearline HDD revenues increased 25 per cent year-over-year and represented 57 per cent of annual HDD revenue.
A Seagate disk drive capacity mix table shows the growth of the nearline disk drive product sector, with EB and average capacity per drive increasing.
Mass capacity storage includes nearline, video and image applications and NAS. Legacy markets include mission-critical, desktop, notebook, digital video recorders, gaming consoles and consumer applications
Wells Fargo senior analyst Aaron Rakers told subscribers: “Legacy HDDs accounted for 36 per cent of total HDD rev. vs. 38 per cent and 51 per cent in F3Q20 and F4Q19, respectively. The legacy segment experienced weaker than seasonal demand (COVID-19 impacts).“
A nearline demand trend chart shows a recent increase;
Seagate is shipping lots of 16TB drives and has begun growing its 18TB drive business, with 20TB heat-assisted magnetic recording (HAMR) drives due by the end of the year.
Small Seagate SSD potatoes
The company made a comparatively small amount of revenue from SSDs and other non-HDD items in the quarter. HDDs accounted for $2.32bn while SSDs, enterprise systems, etc. brought in $195m. That is not to be sneezed at but represents just 8.4 per cent of Q4 revenues.
Seagate anticipates $2.3bn revenues plus or minus $200m for next quarter, down on last year’s $2.6bn. Rakers told subscribers: “Seagate expects demand for its nearline products to moderate during the September quarter. This is driven by COVID-19 related weakness continuing in video and image applications and NAS (SMB enterprise / on-premises), while data centre/hyperscale demand is expected to remain strong.”
Longer term, Seagate sees unabated growth in data at the edge and in the cloud, and this will drive secular demand for mass capacity storage.
Commvault has grown its revenues after five quarters of decline as changes to its products, sales organisation and channel program start delivering results during the Covid-19 pandemic.
First quarter revenues for fiscal 2021 ended 30 June were $173m, a 7 per cent year-on-year rise, easily surpassing the $155m to $250m revenue guidance Commvault provided three months ago.
President and CEO, Sanjay Mirchandani, was effusive in his prepared remarks: “I could not be prouder of the team. Our first quarter results are validation that our streamlined operations and reinvigorated go-to-market engine are starting to execute well. Together with our new product portfolio and robust partner program, our customers are starting to embrace our intelligent data management vision.”
Commvault revenues by quarter and fiscal year. Spot the latest uptick.
Profit came in at $2.28m, a turnaround from the $6.8m loss reported a year ago. William Blair analyst Jason Ader said this was helped by: “Changes to its expense structure – particularly related to rationalising sales-and-marketing costs – as well as temporary savings associated with Covid-19 (e.g., less sales-related travel and a shift to virtual conferences).”
He added: “Outperformance in the first quarter was due to several non-repeatable factors – i.e. reduced OPEX resulting from Covid-19-related expense reductions and large deal closures.”
Annualised recurring revenue (ARR) was $471.6m, up 9 per cent year-on-year.
Subscriptions represented 69 per cent of total software revenue in the quarter.
Software and products revenue was $76.6m, up 20 per cent year-on-year, driven by a record 41 per cent increase in larger deals in the Americas – greater than $100k in software and products revenue.
Average large deal size was $402k compared to $298k a year ago.
Services revenue was $96.4m, a decrease of 2 per cent year-on-year
Operating cash flow totalled $15.3m compared to $31.1m a year ago.
Total cash and short-term investments were $356.3m compared to $339.7m in the corresponding quarter of fiscal ’19.
Wells Fargo analyst Aaron Rakers looked at the geographic breakdown of the results : “Commvault reported a 62 per cent year-on-year increase in Americas software revenue, a 36 per cent year-on-year decrease in APAC software revenue, and a 12 per cent year-on-year decrease in EMEA software revenue.
“Total revenue in the Americas returned to growth and was up 17 per cent year-on-year while EMEA and APAC declined 2 per cent and 19 per cent year-on-year respectively.” Clearmyl there’s still work to be done in Commvault’s APAC and EMEA regions.
Commvault estimated that Q2 fy21 revenues will be $164m, slightly down on the year-ago’s $167.6m.
Ader said he remained “wary that one quarter of strong performance does not a trend make,” – he is now modelling 2 per cent revenue growth for Commvault’s fiscal 2021, up from his previous forecast of a 6 per cent revenue decline. That’s quite some turnaround.
We had a quick interview with Tom Black, HPE Storage’s inaugural boss, earlier this month. Black took on the job in January and is a 20-year networking veteran. He was promoted from HPE’s Aruba networking unit. and his career includes VP Engineering roles at HP, Arista and Cisco.
Networking guy to run storage
So, why appoint a network engineering guy to run storage? Pointing out that Pure Storage CEO Charlie Giancarlo and NetApp CEO George Kurian are former Cisco execs, Black says he thinks of storage and networking as “different dialects of the same language”.
Tom Black.
Both technologies move data, involve CPUs, memory and PCIe and can use ASIC. They are distributed systems with redundancy at every level, and “both are absolute fanatics on data integrity… both have a very large blast radius and that ripples into system design for networking and storage.”
Because of this overlap, Black says, there are similarities in the computer science concepts, design methodologies and logic involved in the two disciplines although “obviously there are different data structures”.
That means storage engineers can have meaningful conversations with networking engineers. And it also means Black, as an incoming storage boss, understands the basic whys and wherefores of a storage array technology and can ask, engineer to engineer, why choices have been made to use one technology option or direction when others are possible. He says the approach is one of “blameless inspection”.
Edging forward
But what’s going on today? One of Black’s first public HPE storage moves was a minor re-org that consolidates the SimpliVity and Nimble dHCI research and development teams.
We asked Black what HPE will do with file storage, where its technology is relatively weak, and where the company relies on partnerships such as WekaIO. He said HPE has strong partnerships and values them and also: “I don’t want to solve yesterday’s problems… I’m not going to try and catch up on someone else’s 10-year advantage.”
Tech transitions equal opportunity
Black pointed out that HPE Storage drops most of its operating profit to the bottom line and he noted that there has not been a lot of market share change between mainstream vendors in recent years. “So, storage is ripe for innovation.”
Where will he take HPE Storage? One area of interest is the internet edge – “a great opportunity,” he says.
Black sees technology transitions as times where large market share changes are made. NVMe over Fabrics is “a big transition” in the early stage of a multi-year change and promises disruptive economics.
He didn’t reveal any roadmap items but did say: “We have very aggressive next-year plans, with very innovative new programs on the docket.”
Take this at face value and HPE will introduce new file and object storage technology products, perhaps with a focus on Internet edge applications, and perhaps using NVMe-oF.
Veritas is laying the foundations for a universal data management platform, with NetBackup at its centre.
The company released NetBackup 8.3 today – and don’t let the point release fool you. This is a huge update, with a slew of new features.
NetBackup customers can standardise on a single platform covering hybrid and multi-cloud environment and save money, according to Veritas, which has combined the software with the Veritas Resiliency Platform (VRP) and CloudPoint into the ‘Enterprise Data Services Platform’ (EDSP).
Deepak Mohan, EVP for the Veritas Products Organisation, said a prepped quote: “We’re extending enterprise-grade data protection and the most robust set of recovery options to every corner of our customers’ IT environments – from on-premises physical to virtual, to cloud and even to containers.”
NetBackup 8.3 includes:
Better ransomware protection
2048-bit encryption and integration with third party key management systems (EKMS)
Role-based access control (RBAC) enhancements
Granular recovery of virtual machines with near-zero Recovery Point Objective (RPO) due to Continuous Data Protection (CDP)
Orchestrated rehearsals with zero production impact to verify the efficiency and effectiveness of recovery procedures
Instant access for SQL Databases
Expanded workload capabilities such as Microsoft SQL self-service
MSDP (media Server Deduplication Node) – direct cloud tiering and data management in multiple buckets, storage tiers and cloud providers from a single node
Image sharing in the cloud and support for Azure and S3 compatible object storage
Object storage support for client-side deduplication for writing directly to cloud storage
Cloud Snapshot enabling granular recovery.
There’s more. NetBackup 8.3 has cloud-native data protection for AWS, Azure and GCP. There is workload and data portability in hybrid and multi-clouds, with to-the-cloud and between-cloud storage tiers. Veritas has extended cloud-to-anywhere portability, adding Azure Stack to Azure Stack and Azure region-to-region, with push button orchestrated disaster recovery using Veritas Resiliency Platform integration.
Veritas has added storage cost optimisation with integrated management and reporting from Veritas APTARE IT analytics.
NetBackup 8.3 reduces discovery time from hours to minutes for large environments, with 50x speed-up for VMware vCenter and vCloud. There is 25 per cent faster dynamic NAS data protection via auto-discovery of resources and load balancing with the ability to restore data anywhere on any NetBackup target. This removing vendor lock-in, according to Veritas.
Veritas is a veteran enterprise backup supplier with more than 80,000 customers, and traditionally competes with the like of Commvault and Dell EMC. In common with these rivals, it is fighting a catch-up war with three groups of vendors.
Veeam and Acronis are backup vendors that have ridden the server virtualization wave. Actifio, Cohesity and Rubrik are in a second group that has pioneered secondary data management functions such as copy data management. They use backup as a data generating source for these functions.
A third group consists of vendors such as Clumio and Druva that provide backup as a service and in-cloud backup. Some venders in this category – HYCU, is an example – specialise in data protection for Nutanix and Azure.
Veritas is on the money in articulating the need for universal data management across on-premises, hybrid, and multi-cloud environments. The core features should include cloud protection, workload migration and disaster recovery functionality. However, this is a big ask and Veritas is in a race to deliver such a service and prevent customer erosion.
Kubernetes enables stateless containerised app movement between clouds but not stateful ones. Each cloud has its own method of automation, deployment, and monitoring for stateful apps – and that makes app migration difficult.
“No one has effectively solved this problem until now,” says Partha Seetala, founder and CEO of Robin.io, a cloud automation startup. His company has launched a service on the Robin Platform that “makes it possible to move stateful applications across clouds to support SLAs, latency, cost and other critical business objectives”.
The technology extends Kubernetes with a three-stage process:
Create application-level snapshot on the source cloud that captures all metadata, configuration, and data at a point in time.
Push the snapshot to a global repository accessible to all clouds in their infrastructure.
Pull the snapshot from the global repository to the target cloud and recreate the application, along with its data and exact configuration
The push and pull operations use a single command or click.
Blocks & Files thinks containerised app portability between multiple public clouds and private clouds will become table stakes for suppliers offering stateful application support. Think Nutanix and VMware, for example.
NetApp has gained a top 10 SPC-2 benchmark and the price performance record with an all-flash end-to-end NVMe EF600 array.
SPC-2 measures throughput and price performance, using three component workloads: large file processing; large database query; and video-on-demand. SPC-1, Its sister Storage Performance Council benchmark, focuses on storage array IOPS rather than throughput.
NetApp’s test EF600 array had 20 x 1.9TB NVMe SSDs and hooked up to the accessing servers via Emulex NVMe-over-Fibre-Channel host bus adapters.
The system achieved 31,070.79 aggregate MB/sec – the ninth fastest. Fujitsu’s ETERNUS DX8900 S3 array tops the SPC-2 benchmark at 70,120.92 MB/sec. But price performance was $24.37, while NetApp’s EF600 delivered a SPC-2 benchmark best of $3.53.
Top 10 SPC-2 throughput results.
NetApp used the EF570 array for its previous SPC-2 test run. This used a non-NVMe all-flash array and is a September 2018 era product. The EF600 , launched in August 2019, has twice its performance of its predecessor, at 2 million IOPS vs 1 million.
The NetApp EF600 comes in a 2U X 24 slot box with no expansion chassis capability. That is a limitation when it comes to SPC-2 bragging rights – a higher-capacity system would have more throughput performance.
Charting the SPC-2 results gives us a bit of a history lesson with superseded arrays included such as IBM’s XIV, TMS’ RamSan 630, and Sun’s Storage 7420. These show how throughput and price performance have improved over time.
NetApp EF600 result highlighted
Possibly the SPC-2 storage array throughput benchmark needs a revision to cope with newer AI/ML workloads.
How much life there is left in your storage? ProphetStor can tell you that. Also, academic researchers working on AI can avoid costly hardware purchases by using OpenExpress from a Korean university.
OpenExpress for NVMe
The Korea Advanced Institute of Science and Technology (KAIST) has developed Open Express, an easy-to-access open NVMe controller hardware IP modules and firmware for future fast storage class memory research. Associate Professor Dr. Myoungsoo Jung from the Computer Architecture and Memory systems Laboratory (CAMELab), tells us: “OpenExpress fully automates NVMe data processing as hardware, which is a word-first open hardware NVMe controller to the best of my knowledge.”
Jung said: “The cost of third party’s IP cores (maybe similar to OpenExpress) is around $100K (USD) per single-use source code, but OpenExpress is free to download for academic research purposes. Our real storage card prototype using OpenExpress exhibits the world’s fastest performance without a silicon fabrication (better than an Intel Optane SSD).”
All the OpenExpress sources are ready to share online.
Artificially intelligent ProphetStor
ProphetStor’s Federator.ai 4.2 predicts how long a storage system will last before it fails and the software can help schedule workloads away from storage nodes that are predicted to fail. Federator.ai works continuously on collected operation data, including the health of the disks and storage systems and application workloads. Data collected includes metrics from application logs, virtualization platforms, cloud service providers, and infrastructure.
Acronis has added Sale Sharks, the English Premiership Rugby team, to its roster of sports sponsorships. Acronis will provides Sale Sharks with backup and security software and will help the team to analyse on-pitch performance data.
Caringo has announced Swarm Cloud, a disaster recovery managed service powered by Wasabi Hot Cloud Storage. Users can instantly add a remote DR site at a Wasabi facility.
The DDR5 memory spec supports memory DIMMs of up to 2TB capacity through the maximum die capacity rising to 64Gbits, 4x higher than DDR4’s 16Gbits, and stacking of 8 dies on a memory chip. The standard DDR5 data rate is 6.4Gbit/sec, which is double DDR4’s 3.2Gbit/sec maximum speed. DDR5 will offer 2 x 32-bit data channels per DIMM with pairs of DIMMs mandatory.
FileCloud’s latest version of its enterprise file sharing and sync platform has an improved notification system with more granular controls. The release adds Automatic Team Folder Permission Updates, ServerLink Enhancements, SMS Integration & Gateways and integration with Multiple Identity Providers (IdP) plus more than 500 minor product improvement fixes.
Fujitsu is the the first vendor to offer Nutanix hyperconverged infrastructure software with certified application performance for analytics and general purpose workloads on Intel-based servers. The Intel certification applies to Fujitsu’s PRIMEFLEX for Nutanix Enterprise Cloud.
NAND bit-growth and higher average selling prices helped Intel Non-Volatile Solutions Group (NAND Flash and Optane) boost revenue 76 per cent higher year over year to $1.66bn in Q2 2020 .
Kingston Digital Europe has added 128GB capacity options to three encrypted USB flash drives: DataTraveler Locker+ G3,, DataTraveler Vault Privacy 3.0 and DataTraveler 4000G2.
Kioxia said its lineup of NVMe, SAS and SATA SSDs have successfully tested for compatibility and interoperability with Broadcom’s 9400 series of host bus adapters (HBAs) and MegaRAID adapters.
Fast object storage software supplier MinIO has gained Veeam Ready qualification for Object and Object with Immutability. This provides customers with an end-to-end, software-defined, solution for deploying MinIO as a backup, archive, and disaster recovery storage target for Veeam.
Nakivo v10 is out. Updates include backup and replication to Wasabi, vSphere 7 Support, full physical to virtual recovery and Linux Workstation backup.
Quantum has added multi-factor authentication software to the Scalar i3 and i6 tape libraries, in order to help secure off-line data against ransomware attacks.
Sixty-eight per cent of available enterprise data is unused, according to an IDC survey of 1500 execs, commissioned by Seagate The disk drive vendor has published the findings in a report: Rethink Data: Put More of Your Data to Work—From Edge to Cloud.
StorMagic has announced encryption Key Management as a Service (KMaaS), its first cloud service.
Yellowbrick Data has teamed up with Nexia, a SaaS data operations platform that helps teams collaborate by creating scalable, repeatable, and predictable data flows for any data use case. Nexia SaaS can deliver data from internal systems, ecosystem partners, and SaaS services into the Yellowbrick data warehouse.
Avinash Lakshman and Ediz Ertekin (credit:Ertekin)
Sponsored During his time working at Amazon and then Facebook, Avinash Lakshman played a starring role in taming the complex and voracious data management needs of these hyperscale cloud giants.
At Amazon, where he worked between 2004 and 2007, Lakshman co-invented Dynamo, a key-value storage system designed to deliver a reliable, ‘always-on’ experience for customers at massive scale. At Facebook (from 2007 to 2011), he developed Cassandra, to power high-performance search across messages sent by the social media platform’s millions of users worldwide.
The ideas he developed along the way around scalability, performance and resilience fed directly into his own start-up, Hedvig, which he founded in 2012 – but this time, Lakshman applied them instead to the data storage layer.
By the time Hedvig was acquired by Commvault for $225 million in October 2019, it had attracted some $52 million in venture capital funding and a stellar line-up of corporate customers including Scania, Pitt State and LKAB. As Commvault CEO Sanjay Mirchandani said on completing the deal: “Hedvig’s technology is in its prime. It has been market tested and proven.”
Tackling fragmentation
Avinash Lakshman and Ediz Ertekin
What sets Hedvig apart is its unique approach to software-defined storage (SDS), which helps companies to better manage data that is becoming increasingly fragmented across multiple siloes, held in public and private clouds and in on-premise data centres, says Ediz Ertekin, previously Hedvig’s senior vice president of worldwide sales and now vice president of global sales for emerging technologies at Commvault.
“What companies are looking for in this multi-cloud world is a unified experience of managing data storage that simplifies what is fundamentally very complex, and becoming more complex with every day that passes,” he says.
The premise for SDS is pretty straightforward: by abstracting, or decoupling, software from the underlying hardware used to store their data, organisations can unify storage management and services across the diverse assets contained in their hybrid IT environments.
It’s an idea many IT decision-makers find attractive, because it gives mainstream corporate IT access to the kind of automated, agile and cost-effective storage approaches that power hyperscale pioneers like Amazon, Facebook and Google.
According to estimates from IT market research company Gartner, around 50 per cent of global storage capacity will be deployed as SDS by 2024. That’s up from around 15 per cent in mid-2019.
As Ertekin explains, the Hedvig Distributed Storage Platform transforms commodity x86 or ARM servers into a storage cluster that can scale from just a few nodes to thousands of nodes, covering both primary and secondary storage. Its patented Universal Data Plane architecture, meanwhile, is capable of storing, protecting and replicating block, file and object data across any number of private and/or public cloud data centres. In short, Hedvig’s technology enables customers to consolidate their storage workloads and manage them as a single solution, using low-cost, industry-standard hardware.
An urgent requirement
At many organisations, a new approach isn’t just attractive, it’s an urgent requirement, says Ertekin. That’s because IT leaders are quickly discovering that established storage management approaches simply don’t sit well with the multi-cloud and modern application strategies that their organisations are adopting in order to fulfill their digital transformation ambitions.
Hybrid and multi-cloud deployments are increasing rapidly, Ertekin points out. And while the cloud-native and containerised applications that frequently underpin new, business-critical digital services may speed up innovation by supporting faster, more agile DevOps approaches, they also add to storage complexity.
“These trends are leading to more fragmentation, and with more fragmentation comes less visibility and more friction,” he says. “Traditional storage management simply can’t keep pace with new requirements.”
It leaves IT teams having to manage a vast range of disparate storage assets, unable to ‘talk’ to each other because they’re not interoperable, he says. They’re also wrestling with multiple management tools to oversee that patchwork of storage systems. On top of that, older ways of working make it extremely difficult to migrate data back and forth between on-premise systems and cloud environments, and between different public clouds. “There’s a distinct lack of portability that IT teams know they must address, but if they can’t see what data resides where, that’s going to be a big struggle,” he says.
By contrast, SDS provides companies with a virtual delivery model for storage that is hardware-agnostic and provides greater automation and more flexible provisioning. As Ertekin explains it, application programming interfaces (APIs) enable Hedvig’s Distributed Storage Platform to connect to any operating system, hypervisor, container or cloud, enabling faster, more proactive orchestration by IT admins and automating many tasks, such as provisioning, that they may have previously needed to carry out manually.
“In this way, organisations using Hedvig don’t just experience the simplification that they need, but also better resource optimisation and substantial cost reductions, too,” he says.
All this has important implications for resilience, Ertekin adds: “When you have that visibility into where data is stored and the ability to move it around as necessary, you’re better equipped to deal with the potential impact of downtime.”
“Our customers can define their data protection policies to deal with everything from a single node failure to downtime across an entire site,” he continues. “For example, in the case of a production workload, where business continuity is critical, you might decide to keep multiple copies of the data, written across multiple nodes, multiple sites or multiple geographies, so it’s never necessary to physically move data if you run into problems.”
Future challenges
At a time when it’s never felt more difficult for IT decision-makers to understand what the future might hold, a big selling point for SDS is the predictability it can bring, according to Ertekin. “With SDS, there’s more flexibility and agility to adapt to changing business and IT requirements with confidence and without having to worry about the implications for performance, scale and cost,” he says.
For a start, the issue of remote working is currently high on CIO agendas, driven by Covid-related stay-at-home orders. SDS can provide the portability to ensure that data remains accessible to employees who need it and can easily be moved or replicated to where it is needed most, while still applying strict policies to it, in order to ensure compliance with corporate governance guidelines, not to mention wider legal and regulatory obligations.
On the cost front, says Ertekin, abstracting storage management from the hardware layer using software relieves organisations of many of the time and cost burdens associated with regular infrastructure refreshes. SDS enables them to rely on commodity hardware, rather than proprietary storage appliances, and to pick and choose between public cloud providers based on pricing. It offers a scale-out solution that allows them to take a more ‘pay as you go’ approach, where matching capacity to performance and costs needs becomes easier to estimate ahead of time.
Some developments, of course, are easier to predict. The burgeoning multi-cloud and containerisation trends show no signs of slowing as companies scramble to deliver new and compelling digital experiences to their customers. In fact, more than four out of five enterprises (81%) are already working with two or more public cloud providers, according to a 2019 Gartner survey, while another study from the firm suggests that three-quarters of global enterprises will be running containerised applications in production by 2022.
As companies increasingly rely on different application delivery models, their storage capabilities will need to be just as diverse, says Ertekin. For example, there will more frequently be a need to add persistent volumes to container environments. SDS delivers the integration with container orchestrators (COs) such as Kubernetes and Docker, for example, to quickly provision the storage that containerised apps need, along with data management and protection for them.
Nor does the volume and variety of data that needs to be managed and protected look set to demonstrate anything other than continued prolific growth.
“The sources might be bare-metal, virtual servers, or containerised servers or IoT edge devices. Their locations might be centralised data centres, remote and branch offices, edge locations or multiple public clouds – or any combination of these,” says Ertekin. “And the data may be file, block or object-based. It might be streaming data, metadata, primary or secondary data. It will need to be managed for legacy applications, as well as for big data analytics, for AI and machine learning.”
This all points to a world where the ability to impose some kind of simplification will be a must-have for many firms, he says. After all, every organisation is now under pressure to achieve the same kind of standardisation, elasticity, scaling and predictability that enables the hyperscale cloud giants to cater to the needs of often millions of users, in the face of huge spikes and troughs in traffic and without missing a beat.
“But to do that,” he warns, “mainstream organisations also need to be able to take advantage of the same technology underpinnings and strategic thinking that got those hyperscale companies to where they are today. And at the heart of that is the process of making a complex task much simpler, less resource-intensive, less costly. That’s what has guided everything we do at Hedvig and continues to guide us.”
Integrated Media Technology (IMT), a tech systems integrator based in Hollywood CA, has launched its first software product. Called SoDA, the enterprise app can be thought of as data moving logistics service for AWS.
Update: GCP and Azure support plans added. On-premises cost comparison to be added at future date. 27 July 2020.
SoDA manages cost and data migration time visibility. You can use it to decide where best to move data, including comparing on-premises and public cloud tiers. A dry-run, predictive analytics feature allows customers to model the cost and duration of proposed data migrations to and from AWS. Data can be moved automatically according to policies or via manual transfer.
Greg Holick, VP of Product Management at IMT, said in prepared remarks: “SoDA predicts the cost and speed of data movement between on-premises and cloud solutions, giving customers the tools they need to control their spend and understand the time it takes for data movement.”
We asked IMT how SoDA gets and uses on-premises storage costs to make its comparisons with AWS public cloud S3 costs. A spokesperson said “Customers are typically aware of their on premises CAPEX purchases. We are looking at adding a pricebook feature, similar to how we do AWS, for each on premises storage later this year to make that comparison and we will do a cost/month basis based on a period of time.”
SoDA is delivered as a subscription service, with software running in a local virtual machine. It contains a policy engine with cost and ingress and egress transmission rates for different kinds of AWS S3 storage: Standard, Glacier and Deep Archive. SMB, NFS and S3 protocols are supported.
IMT SoDA diagram
Users set up a cloud transfer budget and monitor how individual file transfers affect this with a real-time dashboard. A budget limit can be set to prevent excess costs. Data transfer jobs are tracked and itemised so chargeback routines can be implemented.
SoDA’s subscription pricing model has unlimited data movement with no capacity limits. It has RESTful API integration with third-party media asset management packages. IMT’s datasheet and video marketing material do not say how on-premises storage costs are calculated.
IMT plans to add Azure support by the end of the year and GCP support after that.
Competition
SoDA appears to be an everyday data transfer tool – as opposed to a Datadobi-esque one-off migration project data mover. Other competing offerings include DataDynamics StorageX and Komprise’s KEDM. KEDM moves file data between NFS or SMB/CIFS sources and transfers it to a target NAS system or via S3 or NFS/SMB to object storage systems or the public cloud.
By the way SoDA is not an acronym. IMT chose the name because it’s a refreshing approach.
