Home Blog Page 100

Panasas accelerates bulk data access in latest software release

By
Chris Mellor
-

High-performance computing parallel file storage supplier Panasas now has support for dual-actuator disk drives and has added the S3 protocol to speed dataset access and provide public cloud tiering.

Panasas announced the previous PanFS v9 major release in August 2021. Just over two years later we have PanFS 10 which adds support for Seagate Exos 2×18 dual-actuator disk drives in the ArchiveStor filers, Amazon’s S3 object storage protocol, 24TB disk drives and InfiniBand HDR (200Gbps) connectivity.

Ken Claffey

CEO Ken Claffey said of the release: “We understand that our customers’ data environments demand greater performance and cost-efficiency coupled with storage solutions that are easy to install, configure, and manage at scale.”

The AI and HPC worlds are increasingly overlapping, says the company, adding that PanFS v10 has been developed with that in mind.

Claffey said: “We’re setting a new performance/cost benchmark with the new PanFS 10 capabilities. With support for Seagate’s MACH2 dual-actuator drives, organizations can now benefit from twice the performance capacity without having to evaluate a cost-multiplier of using all-flash storage solutions, which prove costly as storage requirements for emerging workloads expand.”

PanFS 10 stores large files on 3.5-inch disk drives, smaller ones on faster access SSDs and very small files (<1.5MB) and metadata in NVMe SSDs.

Seagate’s MACH2 dual-actuator drives offer around twice the bandwidth and IOPs per TB of capacity as single-actuator HDDs. However, the additional actuators use up the space occupied by a disk platter in a 10-platter chassis, meaning that they are 9-platter drives and so have a reduced capacity; Exos 2X18 18TB drives instead of 20TB or larger.

An Exos 2X18 drive, functioning logically as two 9TB drives, delivers up to twice the sequential read performance of a single actuator 18TB drive.

PanFS v10 software now supports the S3 protocol as well Panasas’ proprietary DirectFlow protocol, NFS and SMB/CIFS v3.1. Panasas’s Director Nodes translate NFS, SMB/CIFS and S3 into the DirectFlow protocol. The S3 support enables cloud-native applications, that use S3 as their primary data access protocol, to store and retrieve data in PanFS systems. 

Panasas documentation says: “Objects stored in PanFS via the S3 protocol are visible as POSIX files in PanFS via the other data access protocols (DirectFlow, NFS, and SMB/CIFS), and POSIX files stored in PanFS via any other data access protocol are accessible as Objects via the S3 protocol.”

A PanMove facility can migrate any number of files between on-prem HPC deployments and AWS, Azure, or Google plus any other S3-supporting cloud service provider.

A 20-page PanFS v10 architectural white paper can be read here. The new release of ActiveStor Ultra and PanFS 10 will be available in Q1 2024.

NetApp doubles down on Microsoft Azure partnership

By
Chris Mellor
-

After announcing a stronger collaboration with Google Cloud in August, NetApp says it has renewed its partnership with Microsoft in a bid to grow public cloud revenues.

NetApp is less successful in the cloud than it might be. Though public cloud revenues of $154 million in its latest quarter are a fraction of its hybrid cloud revenues of $1.28 billion, they are growing, while hybrid cloud has been declining for three quarters in a row.

NetApp spent around $149 million buying various CloudOps startups when Anthony Lye ran the public cloud business up to July 2022. Now it’s not earning much more than that in total public cloud business revenues and its in-house ONTAP products will be responsible for much of that. Actual CloudOps revenues have not been revealed.

The company has introduced lower cost all-flash arrays and a new reseller deal with Fujitsu in a bid for growth. It’s also improving public cloud products and operational services to help that part of its business to grow, and this Microsoft pact is part of that.

There are four software products involved: Azure NetAppFiles (ANF), Cloud Volumes ONTAP, the Blue XP data management facility, and cloud operations cost monitor/manager Spot.

ANF now provides:

  • Datastores for Azure VMware Solution  
  • Application volume group for SAP HANA 
  • Smaller 2 TiB capacity pool size 
  • Large volumes (Up to 500 TiB) 
  • Customer-managed keys  
  • Availability zone volume placement and cross-zone replication  
  • New regions: South Africa North, Sweden Central, Qatar (Central), Korea South 

NetApp says ANF is fast because of the integration of ONTAP all-flash arrays into the Azure Data Center networking infrastructure. Applications running within the Azure Virtual Network (VNET) can get sub-millisecond latency access to ONTAP data.

SVP and GM for NetApp cloud storage Ronen Schwartz said: “Through our collaboration with Microsoft, we can deliver market-leading cloud storage and cloud infrastructure operations to our customers and partners that nobody else in the industry offers.” Qumulo might have a view on that point.

The Spot offering for CloudOps on Azure now includes integrated support for Azure Kubernetes Service (AKS) environments, and a fully managed, open source service that integrates with Azure NetApp Files, BlueXP and Cloud Volumes ONTAP. Spot for Azure uses AI/ML-driven automation to deliver continuous optimization of all Azure compute with, NetApp says, increased efficiency and vastly reduced operational complexity, helping make Azure compute less onerous to manage.

As an indication of this complexity, Spot for Azure has recently gained:

  • Spot Eco support for Azure Savings Plans and all Azure account types including Pay-as-You-Go, multi-currency account (MCA), enterprise agreement (EA), and cloud solution provider (CSP) 
  • Spot Elastigroup Stateful Node for Azure 
  • Spot Security support for Azure 
  • Spot Ocean for AKS enterprise-grade serverless engine for Azure  
  • Spot Ocean for Apache Spark Azure support 
  • Instaclustr Managed PostgreSQL on Azure NetApp Files 

NetApp’s Haiyan Song, EVP and GM of CloudOps at NetApp, said in a statement: “Our expanded Spot by NetApp portfolio on Microsoft Azure and partnership with Microsoft provides customers with what they need to continuously automate and optimize application infrastructure in the cloud and drive the cloud operation and cost efficiency their businesses demand while delivering the digital experience customers expect.” 

We think Spot may well become a NetApp copilot for Azure CloudOps. Blue XP looks like potential copilot material as well.

Spot by NetApp CloudOps products are available from the Azure Marketplace, Microsoft Azure channel partners, and NetApp, and Azure NetApp Files is available through the Microsoft Azure portal as a native Azure service.

Nutanix veep talks HCI, SAN, and AI storage solutions

By
Chris Mellor
-

Interview: We had the opportunity to ask Nutanix about server SAN progress, hyperconverged infrastructure vendor consolidation, VMware and Dell HCI positioning, and its AI intentions. Lee Caswell, SVP of Product and Solutions Marketing, provided the answers.

Blocks & Files: How is it that HCI (server SAN) has not replaced external SAN and file and object storage?

Lee Caswell, Nutanix
Lee Caswell

Lee Caswell: All-flash HCI is actively replacing SAN for all workloads at a pace that is consistent with the conservative nature of storage buyers. New HCI technologies are generally additive, not a wholesale SAN replacement, which allows customers to depreciate prior investments while taking advantage of the more agile and lower cost HCI solution.

NAS and object storage replacement is starting to grow quickly now that very high-capacity storage media and capacity-dense nodes allow scale-out systems to be cost-competitive with traditional scale-up proprietary hardware designs. We expect to see customer adoption accelerate as users consolidate storage architectures to control costs, address talent gaps, and leverage common data services across files, blocks, and objects.

Blocks & Files: How would Nutanix position the strengths and weaknesses of external storage vs HCI?

Lee Caswell: External storage will continue to exist just as UNIX systems persisted when x86 servers took over the compute market. For customers with dedicated storage administrators and the resources to separately maintain storage, server, hypervisor, and network configurations, external storage is a proven, albeit complex, entity. HCI, however, will command higher market growth rates powered by a simplified operating model and the scale economics of commodity servers. 

Cloud spending has shown us that most customers simply want to stop managing storage and get on with their primary objective of speeding the development, deployment, and managing of applications and their associated data. HCI offers a flexible scaling model, a software-defined management model suitable for generalists, and a common operating model across legacy and modern applications with full cloud extensibility that will support the next wave of unstructured data growth.  

Blocks & Files: With Cisco giving up on Springpath, does Nutanix think there will be more consolidation in HCI suppliers’ ranks and why or why not?

Lee Caswell: We’re watching the HCI market evolve very quickly from a narrow on-premises modernization play to an expansive hybrid multicloud platform market. Once customers move to a server-based architecture they start to see their control plane aperture open dramatically. There are very few infrastructure vendors that have the appetite for the engineering investment and the dedicated sales attention required to support a common cloud operating model. 

It also requires an elegant scale-out architecture to deliver a hybrid multicloud platform with a consistent operating model from the edge to the core to the public cloud, across legacy and containerized apps, with full data services including database as a service. Nutanix delivers this seamless experience with one architecture while others are either rearchitecting or, in the Cisco case, partnering with Nutanix. 

Blocks & Files: How would Nutanix position its offerings vs VxRail and, separately, PowerFlex?

Lee Caswell: Nutanix Cloud Platform (NCP) uses a single web-scale architecture with integrated snapshots, replication, and DR that are common underpinnings to all our offerings. Portable licensing allows customers flexibility in moving data and apps at will across servers, including servers in AWS and Azure.  The user experience evident in our NPS of 92 and high renewal rates shows the simplicity of our offering and the excellence of our support organization. The offering includes a distributed file system and object store now recognized by Gartner as a Visionary player. 

VxRail, by contrast, has two incompatible vSAN architectures with node-locked licenses and different data service offerings. Object stores must be licensed separately from partners along with KMS solutions – both of which are included in NCP. Cloud DR systems are powered by a third architecture (from Datrium) which introduces yet another management layer. 

PowerFlex (formerly ScaleIO) is increasingly being positioned by Dell as a vSAN/VxRail alternative. However, PowerFlex is more accurately a software-defined storage (SDS) offering requiring storage specialists since it is managed by LUN and licensed by TB. Finally, since VMware VCF always requires vSAN for the management domain, introducing PowerFlex adds separate patching, maintenance, and licensing headaches. 

Blocks & Files: How would Nutanix position its offerings as being capable of providing primary storage for AI Training?

Lee Caswell: With our recently released Nutanix GPT-in-a-Box, we provide reliable, secure, and simple AI-ready infrastructure to support LLM and GenAI workloads. LLM and use case specific fine-tuning data is stored on Nutanix Files Storage and Object Storage while block storage is used to run the models for training. Nutanix recently submitted verified files and objects MLPerf Storage Benchmark results (link). Of note was our ability to deliver 25Gbps for NFS to power 65 ML accelerators (GPUs).

Blocks & Files: Ditto AI Inferencing?

Lee Caswell: A powerful feature of Nutanix AI solutions is the ability to optimize resources from core to edge. Training and fine-tuning a model is often more resource-intensive than inferencing and would happen in the core. Inference can happen in the core or at the edge, depending on the application’s needs. This flexibility opens the potential for optimization of resources. Many edge scenarios are sensitive to latency and require high-performance storage systems with the ability to function without an active cloud connection.

Backblaze hits $100M ARR, but computer backup stalls

By
Chris Mellor
-
Storage pod
Storage pod

Cloud storage provider Backblaze grew revenues 15 percent year-on-year in its third 2023 quarter but backup storage revenues declined quarter-on-quarter.

Revenues were $25.3 million in the quarter ending September 30, with a loss of $16.1 million, as Backblaze prioritized growth over profitability. This compares to a year-ago loss of $12.8 million on revenues of $22.1 million. Cash, short-term investments, and restricted cash amounted to $35.8 million at quarter end.

Backblaze revenues

Annual recurring revenue (ARR) rose 15 percent year-on-year to $100.9 million. CEO Gleb Budman said: “In Q3, we passed the $100 million in ARR milestone and are on track to achieve adjusted EBITDA profitability in Q4 through continued strong growth and efficient execution.

“Looking ahead, our recent price increase supports continued investments in the platform and positions us for profitable growth while continuing to offer customers a compelling and cost effective storage solution.”

Its computer backup business brought in $13.7 million, up 4 percent year-over-year, while the B2 cloud storage business generated $11.6 million, but B2 cloud storage ARR rose 31 percent year-over-year to $46.8 million. There have been price increases for both B2 cloud storage and computer backup storage, with the larger increase for computer backup. That may have stymied growth. Customers can pay monthly or have one or two-year contracts.

Backblaze segment revenues
B2 cloud storage revenues are on track to overtake computer backup revenues by the end of 2024

Budman said: “We’ve actually seen some customers wanting to switch from monthly to one and two year to lock in the price point ahead of the pricing.” He doesn’t see the computer backup business growth rate getting to double figures. “We’re going to have a growth rate in the upper single digits for computer backup.”

Backblaze started off selling backup storage in its cloud then branched out into general B2 cloud storage, now the fastest-growing part of its business, representing 46 percent of revenues. Budman said: “Long-term B2 is overtaking as the dominant product quickly.” Backup suppliers like HYCU utilizing B2 cloud storage will contribute to this growth.

The company is offering free egress with its B2 cloud storage, and Budman said: “We saw new customers engage with us directly because high egress fees charged by other providers are a major pain point for them.”

Budman said in the earnings call: “We believe that we’re at an inflection point with higher revenue growth expected in Q4 and 2024 … We’ve made great progress on our financial performance, particularly adjusted EBITDA and cash.”

CFO Frank Patchel added: “Next quarter, we expect to accelerate revenue growth, reduce cash usage by about half sequentially, and reach positive adjusted EBITDA … We expect Q4 adjusted EBITDA margin between a positive 1 percent and a positive 3 percent. Q4 is expected to be our first quarter of positive adjusted EBITDA as a public company.”

He also said: “Cash break-even would be in the first half of 2025.” Profitability should follow after that, it will be hoping.

Backblaze said it would use CoreSight colocation facilities in Reston, Virginia, for its expanded service in the eastern US data region.

The outlook for Backblaze’s final 2023 quarter is for revenues of $29.1 million plus/minus $400,000, a 21 percent rise from a year ago at the mid-point. Full 2023 year revenues would then be $101.6 million, a 19 percent annual increase.

GigaOm highlights key players in cloud storage landscape

By
Chris Mellor
-

GigaOm analysts have surveyed the cloud file storage area and found it necessary to split it into two distinct areas for its Radar reports: distributed cloud file storage, with a focus on collaboration, and high-performance cloud file storage. Three suppliers are present in both reports: Hammerspace, Nasuni and Panzura. CTERA leads the distributed cloud file storage area while NetApp is ahead in the high-performance cloud file storage report.

A Radar report is GigaOm’s view of the main suppliers in a market area, summarizing their attributes within three concentric rings – entrant (outer), challenger, and leader (inner) – with those set closer to the center judged to be of higher overall value. Vendors are located with reference to their scores on two axes – Maturity to Innovation and Feature Play to Platform Play. An arrow projects each vendor’s offering over the coming 12 to 18 months. The arrows represent three types: a forward mover, a fast mover, and an outperformer.

GigaOm Distributed Cloud File Storage Radar

There are just six suppliers present in the distributed file storage group: two separated outsiders in the challenger’s area, Peer Software and LucidLink; and a group of four close together in the Leaders’ ring with a balance between being innovators and generalized platform players.

The front-ranked supplier is CTERA, followed by Nasuni, both classed as outperformers, and then Hammerspace and Panzura. Hammerspace is a data orchestration supplier whereas CTERA, Nasuni, and Panzura are cloud file services suppliers with a collaboration focus.

Peer Software is described by GigaOm’s analysts as providing Global File Services (GFS) software on which customers can abstract distributed file services on top of existing storage infrastructure, and supporting scalability in the cloud. The company has an ambitious roadmap and recently introduced a new analytics engine, but it will take time to execute.

LucidLink’s Filespaces software presents on-premises applications with instant access to large S3 and Azure Blob-stored datasets over long distances, with local storage caching of the current data needed. It also works with on-prem object stores.

Analysts Max Mortillaro and Arjan Timmerman say: “Areas where we found the most significant variances were: data management, analytics, advanced security, and edge deployments. All of these areas are important, but we can’t stress enough the urgency of advanced security measures as a mitigation strategy against elevated persistent ransomware threats. Well-designed and varied edge deployment options are also critical now that most organizations must accommodate a massively distributed workforce.”

The vendors may need to provide data classification, data compliance, and data sovereignty features in the future.

GigaOm Radar for High-Performance Cloud File Storage

There are 14 suppliers who are placed in three groups:

  • IBM (Storage Scale), Zadara, and DDN (EXAScaler) all classed as more mature suppliers
  • Cloud file storage mainstreamers
  • Public cloud group

NetApp is the clear front runner in the leaders’ area, followed by Hammerspace then Qumulo. Weka, Nasuni, and Panzura are challengers poised to enter the leaders’ ring. These six suppliers are more platform than feature-focused, and represent, in our view, a mainstream group.

It is somewhat surprising that Weka, very much focused on high-performance file data access, has a relatively low ranking, being beaten by NetApp, Hammerspace, Qumulo, and Nasuni. The report authors suggest Weka’s data management needs improving: “It is expected that, in the future, Weka will improve some of its capabilities in this area, notably around metadata tagging, querying, and so on.”

IBM, Zadara, and DDN are in a top right quadrant and classed as challengers.

ObjectiveFS, Microsoft, and Amazon are a separate group of challengers, with entrants Google and Oracle set to join them in a public cloud-oriented group that is more focused on adding features than being a platform play.

Timmerman and Mortillaro say: “The high-performance cloud file storage market is interesting. It might seem obvious to many that, thanks to their dominance and massive market share, public cloud providers would provide the most comprehensive solutions. Nothing could be further from the truth.

“The primary concern with these is that, with a few notable exceptions (such as Tier 1 partnerships with vendors such as NetApp), these public cloud solutions typically need additional adjustments and improvements to meet enterprise requirements.”

Hammerspace, Nasuni ,and Panzura are all present in GigaOm’s Distributed Cloud File Storage Radar report as well as this high-performance cloud file storage report. No other suppliers have a dual report presence.

You can read the report here courtesy of Hammerspace. It provides excellent and concise descriptions of each vendor’s offerings.

GigaOm Radar reports are accompanied by Key Criteria that describe in more detail the capabilities and metrics used to evaluate vendors in this market. GigaOm subscribers have access to these Key Criteria documents.

Qumulo file system goes native on Azure

By
Chris Mellor
-

Qumulo has added a native Azure implementation of its scale-out file system, complementing its existing edge, on-premises, and public cloud presence.

Its Scale Anywhere concept matches NetApp’s data fabric vision in scope and arguably provides a more scalable and parallel filesystem than NetApp’s ONTAP in the public clouds and Dell’s APEX File Storage for AWS. 

Qumulo CEO Bill Richer said: “Unstructured data is everywhere in the enterprise and growing at an unprecedented rate, a reality that traditional storage solutions have constantly grappled with. Qumulo finally fixes this with a software platform built for hybrid IT that scales to exabytes anywhere unstructured data is generated, stored, consumed, or managed.”

The Scale Anywhere vision in a nutshell is up to exabyte-level file storage on-premises, edge, datacenters, and the public cloud. It’s a vision shared and pioneered by NetApp, Dell, and HPE (GreenLake).

Qumulo Scale Anywhere graphic
Qumulo graphic

Kiran Bhageshpur, Qumulo CTO, said in a briefing: “I look at this as the third era of data. First, there was the NetApp era, which was scale-up domain controllers connected across SAS expanders. They owned the 1990s and the early 2000s. Then there was a scale-out era, which was very much Isilon’s kind of game, and I was there. We really ran the table at that point in time, and it’s the 8,000-pound gorilla for large scale unstructured data today. I believe we are in the third era, which is scale anywhere.

Kiran Bhageshpur

“What’s changed from 15 years ago to now is the fact that data is everywhere. It’s not just datacenters, it is the edge. And the edge is very dispersed. We’ve got customers who are talking about hundreds of terabytes and mining sites, you’ve got national defense workflows with forward operating bases and geosynchronous or other satellite imagery and longtime drone imagery, which needs to be shared all across the edge. And then of course you’ve got the cloud with an absolute plethora of services.”

A file system and data services supplier has to cover all these bases. Bhageshpur told us “file in the cloud has been a joke” because it doesn’t use a cloud-native approach. That was true of the initial April Azure Qumulo implementation. “The version we put out in April was still what I would call a version 1.0 from a cloud architecture point of view as it was a lift and shift … What we are now announcing is a truly cloud-native Enterprise File System in the cloud, starting with Azure.”

Qumulo says the Scale Anywhere Platform has four components:

  • Ability to run on commodity hardware or in the (main) public clouds.
  • A Nexus unified control plane for customers to manage all their Qumulo instances anywhere.
  • Q-GNS global namespace providing a unified edge-to-datacenter-to-public cloud data plane with its data services – multi-protocol access, enterprise security integrations, snapshots, quotas, replications etc. – delivered within this geographically distributed namespace. Qumulo says customers can access remote data as if it were local for all their workflows, from the most performance-sensitive applications to active archives.
  • Azure Native Qumulo (ANQ), described by Qumulo as “the first cloud-native enterprise file system with unlimited exabyte scale, unmatched economics, and performance elasticity.”

Note that the current Qumulo implementations on AWS (Qumulo File Data Platform) and GCP are not included in this component list. They are not cloud-native. We expect that cloud-native versions will arrive in due course. Bhageshpur said: “Our view is scale anywhere. So every single cloud, every single hardware platform.”

ANQ was developed with Microsoft and data is stored in Azure Blob object storage buckets with caching to speed up access. Bhageshpur said: “The caching uses Azure ephemeral instances’ attached NVMe as a read cache, we also use a high performance managed disk as a protected cache.” Qumulo says 99 percent of ANQ I/O operations are served from the cache, improving performance and shielding file workloads from cloud object storage latencies.

  • ANQ separates performance from long-term storage retention, allowing each layer to scale independently, elastically, and seamlessly. It lets customers dynamically raise and lower their cluster’s throughput rates on demand, as needed. 
  • ANQ leverages Azure’s 12 nines of data durability and its clustered architecture so files are dependably available.
  • ANQ is almost 80 percent less expensive than the closest alternative (understood to be Azure NetApp Files), and is comparable to the fully burdened on-premises cost of file storage.
  • ANQ data services include quotas, snapshots, multi-protocol access, enterprise security integrations, and real-time data analytics.
  • ANQ can be configured and deployed directly from the Azure service portal in minutes.

Bhagesphur said of cost: “For the first time file in the cloud, rich enterprise file in the cloud is going to be comparable to the cost of object in the cloud.”

ANQ has a cloud-familiar pay-as-you-go model in its pricing structure, where economics improve as data volume grows. 

Monthly pay-as-you-go pricing is $30 per TB per month (for 1 PB and beyond). Pricing starts at $37 per TB per month (baseline commitment of 100TB), with an incremental charge per TB as data grows.

Microsoft’s Maneesh Sah, CVP Azure Storage, said: “Combining Microsoft Azure Blob Storage with Azure Native Qumulo, Microsoft and Qumulo deliver familiar on-premises file management capabilities at costs similar to object storage. Azure is the only cloud that offers the balance of performance, cost, and scalability.”

Qumulo is also announcing a consumption-based Qumulo One pricing model.

Bhageshpur said: “With our latest update to Azure Native Qumulo, customers can migrate their file-based AI/ML workloads to the cloud, and run them natively in Azure at scale, without compromising performance, features, or cost.”  He also said that Qumulo Scale Anywhere customers won’t need Hammerspace to orchestrate files within the Qumulo namespace as Qumulo includes that kind of functionality.

He thinks Qumulo operates at a level of scale beyond the capabilities of edge-caching cloud-based file data services suppliers such as CTERA, Nasuni, and Panzura, characterized as home directory-style use cases: “My view is that a lot of what Panzura, especially Nasuni, are doing … is going to get subsumed by some variation of OneDrive, Google Drive, or Dropbox … Whereas what we are looking at is hundreds of terabytes to hundreds of petabytes, really large scale data.”

The Q-GNS global namespace hits private preview from November 9 then public preview on February 1. ANQ is generally available.

Generative AI: It’s not just for the big guys

By
Clint Boulton, Dell Technologies
-

COMMISSIONED: Being stuck in the middle is no fun.

Just ask the titular character Malcom, of the TV series “Malcolm in the Middle,” (2000-2006) who struggles to stand out among his four brothers. In the earlier sitcom, “The Brady Bunch,” (1969-1974) Jan Brady feels overshadowed by big sister Marcia and younger sis Cindy.

Polite (or impolite) society has a name for this phenomenon, which describes the challenges children sandwiched between younger and elder siblings feel within their families: Middle child syndrome.

Reasonable minds differ on the legitimacy of middle child syndrome. Is it real or perceived and does it matter? Even so, it can be hard to compete with siblings – especially brothers or sisters who enjoy the lion’s share of success.

The middle children of the global economy

As it happens, the global economy has its own middle children in the form of small- to medium-sized businesses, which find themselves competing with larger enterprises for talent, capital and other vital resources.

Yet like their larger siblings, SMBs must innovate while fending off hungry rivals. This dynamic can prove particularly challenging as SMBs look to new technologies such as generative AI, which can be resource intensive and expensive to operate.

No organization can afford to overlook the potential value of GenAI for their businesses. Seventy-six percent of IT leaders said GenAI will be significant to transformative for their organizations and most expect meaningful results from it for within the next 12 months, according to recent Dell research.

Fortunately, SMBs wishing to capitalize on the natural language processing prowess of GenAI can do so – with the right approach: Using a small language model (SLM) and a technique called retrieval augmented generation (RAG) to refine results.
You may have noticed I called out an SLM rather than a large language model (LLM), which you are probably more accustomed to reading about. As the qualifiers imply, the difference between the model types is scale.

LLMs predict the next word in a sequence based on the words that have come before it to generate human-like text. Popular LLMs that power GenAI services such as Google Bard and ChatGPT feature hundreds of billions to trillions of parameters. The cost and computational resources to train these models is significant, likely putting building bespoke LLMs out of reach for SMBs.

SMBs have another option in building small language models (SLMs), which may range from a hundred million to tens of billions parameters and cost less to train and operate than their larger siblings.

SLMs can also be more easily customized and tailored for certain business use cases than LLMs. Whereas LLMs produces long form content, including whole software scripts and even books, SLMs can be used to build applications such as chatbots for customer service, personalized marketing content such as email newsletters and social media posts and lead generation and sales scripts.

Even so, whether you’re using an LLM or an SLM, GenAI models require enough computational resources to process the data, as well as data scientists to work with the data, both of which may be hard for SMBs to afford. And sure, organizations may use a pre-trained model but it will be limited by the information it knows, which means its accuracy and applicability will suffer.

RAG fine-tunes models with domain knowledge

Enter RAG, which can add helpful context without having to make big investments, thus democratizing access for SMBs. RAG retrieves relevant information from a knowledge repository, such as a database or documents in real time, augments the user prompt with this content and feeds the prompt into the model to generate new content. This helps the model generate more accurate and relevant responses for the information you wish your model to specialize in.

For example, at Dell we show organizations how to deploy RAG and Meta’s LLama2 LLM to retrieve domain-specific content from custom PDF datasets. The output was used to show how an organization might theoretically use RAG and an LLM to train a help-desk chatbot.

SMBs can use an SLM with RAG to build a more targeted and less resource-intensive approach to GenAI. Effectively, the combination affords them a very accurate tool that delivers more personalized information on their company’s data – without spending the time and money building and fine-tuning a custom model.

Getting started with RAG may seem daunting to SMBs but organizations can repurpose a server, a workstation or even a laptop and get started. They can pick an open-source LLM (such as LLama2) to begin the process. Dell calls this the GenAI easy button.

That way organizations can bring the AI to their data, keeping control of their sensitive corporate IP while freeing up IT resources as they innovate.

SMBs play an important role in the economy by contributing to innovation. Yet too often they’re relegated to Malcom or Jan status – the oft-underestimated and neglected middle children of the global economy.

By combining the right approach and technology tools, SMBs can leverage GenAI to accelerate innovation, enabling them to better compete and woo new customers – rather than feeling lost in the middle of the corporate pack.

To learn more, visit dell.com/ai.

Brought to you by Dell Technologies.

Commvault launches multi-vendor cyber resilience partnerships

By
Chris Mellor
-
Handshake
Handshake

Commvault, which just announced its all-in-one Commvault Cloud software infrastructure, has set up multiple security supplier agreements to provide comprehensive cyber resilience, security, and data intelligence capabilities.

Commvault believes collaboration is key, and is setting up partnerships with cyber security, artificial intelligence, and cloud suppliers to provide joint customers with more ways to detect, protect, and respond to potential threats and attacks, while also improving data visibility and governance. It is working with partners across the security tool chain, including: security information and event management (SIEM); security orchestration, automation, and response (SOAR); network detection and response; vulnerability, threat detection, and assessment; incident management; and data governance and privacy. 

Chief product officer Rajiv Kottomtharayil said: “By integrating with a broad ecosystem of new security and AI partners via our Commvault Cloud platform, we … can collectively and jointly bring faster, smarter, and more connected security insights to organizations around the world.”

Depiction of Commvault anti-malware security partnerships
Blocks and Files depiction of Commvault anti-malware security partnerships

The partner list includes:

  • Avira (part of Gen): AI/ML-driven threat intelligence, prediction, analysis and anti-malware technologies. 
  • CyberArk: Identity security platform.
  • Darktrace: Machine learning-based anomaly detection with integration with HEAL and Commvault. 
  • Databricks: Data lake platform for data and AI. 
  • Entrust: Post-quantum cryptography and data encryption.
  • Microsoft Sentinel: SIEM. 
  • Netskope: Zero trust-based Secure Access Service Edge (SASE) web content filtering. 
  • Palo Alto Networks: Threat intelligence repository leveraging Cortex XSOAR to shorten incident response times. 
  • Trellix: Threat detection and response with Intelligent Virtual Execution (IVX) sandbox to analyze and inspect malware in an isolated environment.

This set of integrations is impressive, but suggests the skill set needed to manage and operate these facilities will be substantial – and more so if more partners’ technologies are used. Admin staff will need to be trained and proficient with each of the suppliers’ technologies.

This suggests, in turn, given Commvault’s Arlie copilot initiative, that an even more capable copilot could be needed to help users understand the overall anti-malware capabilities of their IT environment, and the vulnerability gaps. Such an assistant would need to be trained on all the partners’ systems, be able to answer users’ questions, and generate reports and code for processes to help detect, diagnose, and fix problems in what is a highly complex environment.

If – like Commvault (Airlie), Databricks (Dolly) and Microsoft (OpenAI) – each supplier has their own copilot, they will need to interact with each other to help customers with multi-supplier security systems. Copilot-to-copilot conversations will be a whole new generative AI ball game.

Rubrik runs Ruby copilot on GenAI rails

By
Chris Mellor
-

Rubrik has jumped aboard the generative AI copilot train with Ruby to speed cyber detection, recovery, and resilience.  

Rubrik joins Druva (Dru) and Commvault (Arlie) who also have data protection copilots. A copilot is a generative AI large language model trained on the supplier’s products and services and able to interpret natural language questions and requests. It can respond in natural language and also generally produce coded statements to activate reports and procedures.

Anneka Gupta, Rubrik chief product officer, said: “Think of Ruby as the personification of a security analyst in AI, who is there to hold the customer’s hand to resolve a security incident much faster than they could do before.” 

Rubrik says it uses AI to help customers in three areas:

  • Security expertise with a guided response process that helps users navigate challenging workflows and speeds recovery from cyber incidents via Ruby.
  • Detecting anomalous activity in data across enterprise, cloud, and SaaS applications, to help customers identify malicious activity and determine the scope of a cyberattack or incident via Rubrik’s Data Threat Engine.
  • A support team, which can be more proactive and targeted in alerting potential problems before they impact an organizations’ systems via Rubrik’s Sentry AI platform. 

Ruby, like Commvault’s Arlie, uses Microsoft Azure’s OpenAI large language model. It also uses Rubrik’s best practices and the expertise of its field teams and insights from its ransomware recovery efforts. It will generate data risk alerts, and users can ask follow-up questions including whether any sensitive data was impacted, or what needs to be done to restore the environment. Ruby will provide guidance on additional questions to pose, and help customers resolve incidents more quickly.

Rubrik Ruby interaction with user
Ruby interaction with a user

It will help less skilled users deal with cyber threats. Gupta said: “Securing business data must be a company-wide imperative; every employee should be empowered with the tools to quickly respond to incidents. Our goal with Ruby is to bridge any skills gaps and eliminate the roadblocks to cyber response so that all organizations realize business continuity and preserve the integrity of their corporate data.”

Rubrik says Ruby is planned to be available in the coming months to Enterprise Edition subscribers who opt in. Over time, Ruby will expand to help customers recover even faster and more effectively from cyber attacks.

Rubrik Ruby malicious file alert
Ruby malicious file alert

Comment

Rubrik is positioning Ruby as a copilot, a helpful assistant. It is not saying it is an autonomous cybersecurity agent which can be given the responsibility of looking after an organization. That responsibility is the user’s, with Ruby as a tool.

It will be interesting to see how such a copilot performs during an actual cyber incident and helps recognize it, disarm the malware, and recover data to a clean state. A detailed case study of an incident like this will be fascinating to study

We think that having an AI-driven copilot is going to become table stakes for data protection and security vendors.

Bootnote

Rubrik’s Ruby has nothing to do with the server side, web application framework Ruby on Rails, nor with the Ruby open source programming language.

Dell unveils AI and analytics storage appliance

By
Chris Mellor
-

Dell has produced the ObjectScale XF960, an all-flash, scale-out appliance for generative AI and real-time analytics work.

Update: Removed VSAN and Tanzu support from ObjectScale. Added updated XF960 throughput numbers; 9 November 2023.

ObjectScale is Dell’s software-defined object storage system. Now it is available as a turnkey hardware appliance with limited scalability. The XF960 is the first member of an ObjectScale X-Series family and based on a PowerEdge server running ObjectScale v1.3 software. The system includes servers, network switches, rack-mount equipment, and appropriate power cables.

According to an XF960 spec sheet, each node features dual Intel Xeon CPUs within a 2RU chassis and can support up to 737.3 TB of raw capacity. A cluster has a minimum of four nodes, five nodes for high-availability, and a maximum of 16, with up to 11,796 TB of raw capacity (11.8 PB). Its features include:

  • Dual Intel gen 4 Xeon SP Processors with 32 cores per processor
  • 256 GB of memory
  • 24 x NVMe TLC 30.72 TB drives (for the initial release)
  • Dual mirrored boot drives for redundancy
  • Network connectivity with dual 100 GbE back-end and dual 25 GbE front-end speeds.

The ObjectScale software is a distributed, microservices-based, multi-node, scale-out, and multi-tenant object storage system with a single global namespace supporting the S3 API.  It has the same code base as Dell’s ECS software. More than 1,000 nodes are supported, from a three-node start, with no specific architectural limit. The XF960’s scale-out limits are a function of the hardware and not the ObjectScale software.

ObjectScale runs on customer-managed SUSE Linux Enterprise Server (SLES) operating system and Red Hat Open Shift. The v1.3 release adds new S3 bucket logging and enhanced S3 replication. Replicated objects can now be as large as 50TiB, with support for up to 50,000 parts. v1.3 supports multipart upload (MPU) using the S3 protocol. MPU enables the construction of large objects from smaller parts that can be uploaded at any time and in any order.

v1.3 enables Object Lock to be applied at the bucket level, in the absence of versioning. When enabled without versioning, Object Lock applies at the bucket level instead of to individual versions within the bucket. Consequently, overwrite and delete operations are effectively prevented. A v1.3 ObjectScale blog offers more detailed information.

A Dell XF960 blog says it’s a fully integrated, turnkey system and “the world’s most powerful object storage appliance custom-built on Kubernetes.” It envisages software-defined ObjectScale systems at edge sites sending data to XF960 appliances in the datacenter.

We’re told customers can combine Starburst, the Trino-based data lake offering, with ObjectScale “to set the foundation for a high-performance, infinitely scalable lakehouse on-premises and across multi-cloud environments.” However, the XF960 is not infinitely scalable as it is limited to 16 nodes for the initial release, due to additional networking qualifications needed for larger scale-out.

What we have here are Dell object storage nodes that can deliver can deliver up to 5.7 GBps reads and 5.0 GBps writes on http per node. This is an object store used for primary, tier 1, S3 data storage feeding data direct to AI/ML and analytics processes.

Object storage supplier Cloudian can deliver 17.7 GBps writes and 25 GBps reads from a six node all-flash cluster. That’s equivalent to 2.95 GBps read and 4.2 GBps read speeds, slower than the XF960.

MinIO promotes object storage as a primary storage tier but we don’t have any equivalent per-node read and write throughput speed numbers for comparison with Dell and Cloudian.

A Dell blog tells us what Dell thinks about the XF960.

Commvault announces Arlie GenAI copilot at Shift shindig

By
Chris Mellor
-

Commvault is unifying its data protection and resiliency offerings in a Commvault Cloud with single pane of glass management, threat prediction and recovery features, and an Arlie –  short for “Autonomous Resilience” – generative AI copilot using Azure OpenAI.

This all-in-one realignment was announced at its Commvault Shift virtual event. It is using AI to be more sophisticated and active about threat prediction and detection, helping customers use its services more quickly and easily, and offering fully managed Commvault protection services.

President and CEO Sanjay Mirchandani said: “Achieving enterprise-grade cyber resilience is more than building taller walls or deeper moats. It requires a new approach that looks holistically across the entire landscape, from best-in-class data protection and security to AI-powered data intelligence and lightning-fast recovery.”

There’s a lot to take in, and we’ve made a diagram showing the main points:

Commvault Cloud is available in Azure Marketplace, Google Cloud Marketplace, Oracle Cloud Marketplace, Salesforce AppExchange

Commvault says that Commvault Cloud powered by Metallic AI, to use its full name, unifies all of the company’s SaaS and software offerings on one platform, where visibility and control can be managed holistically via a single pane of glass. That means it provides access to Commvault Backup and Recovery for Active Directory, databases, endpoints, files and objects, Microsoft 365 and Dynamics 365, Salesforce, VMware, and Kubernetes, plus Auto Recovery, Cloud Command, Cloud Console, Metallic SaaS, CloudArchive for files and objects, Compliance, Hyperscale X, Security IQ, ThreatScore and ThreatWise.

There is a generative AI-based copilot called Arlie, short for Autonomous Resilience, which can respond to inquiries in plain language. It actually integrates on the backend with the Azure OpenAI Service and can consolidate information and reports, and provide users with personalized actionable responses.

Commvault says, for example, that “users may be able to use Airlie to verify or validate a clean point of recovery for critical systems, or generate requested code in seconds.” Note the “may.”

Airlie will provide users with:

  • Active insights: Real-time threat analysis and report summaries showing the latest information about their cyber resiliency health, removing the need to sift through filters and reports to prioritize the most pressing information.
  • Code assistance: A “no-code” way to build an integration or code an action – users simply type a description of what they want to do, and the code assistant will generate the code on the spot.
  • Custom walkthroughs: Provides context-sensitive, guided product walkthroughs that make it easy for users to set up, customize, and tune Commvault Cloud to their specifications. Users can ask “how to” questions and receive step-by-step documentation with annotated screenshots.
  • Root cause remediation: Constantly monitors the performance of Commvault Cloud using generative AI to find issues and offer real-time recommendations on how to optimize cyber resilience. 

Commvault Cloud has built-in functions that are presented as intelligent insights baked into daily workflows. These include threat prediction based on real-time predictive threat analysis to find AI-driven ransomware, including detection of shape-shifting AI malware, before they impact customers’ backups and ability to recover cleanly.

Another is Cloudburst Recovery, which uses infrastructure-as-code and cloud-scaling to automate rapid recovery of data to any location. Commvault says users will enjoy a lower TCO because of this accelerated process.

Cleanroom Recovery relies on Microsoft’s Azure to provide recovery to a cleanroom in the cloud on-demand for faster recoveries and also incident response testing.

Platinum Resilience, which is being introduced to select customers as part of an early access preview program with full availability planned in early CY 2024, provides the following:

  • Enterprise-grade backup and recovery, advanced AI-driven automation, cyber deception, recovery testing and readiness checks, advanced reporting, security audits, custom telemetry, and more. 
  • A dedicated, secured, isolated control plane for comprehensive hybrid data protection and resilience.
  • A ransomware readiness and response team available 24/7 to optimize everything from setup to ongoing testing and validation to recovery upon a cyber event.

This comes with a Commvault Protection Warranty to give everyone from IT managers to board members piece of mind that recovery from an attack is guaranteed. Click here if you are interested in joining the early access program.

Availability

The following Commvault offerings are targeted for general availability by the end of the calendar year in tandem with Commvault’s next software release: Unified management between the company’s SaaS and software offerings on one platform with Commvault Cloud, Commvault’s AI copilot Airlie, active insights, code assistance, custom walkthroughs, root cause remediation, advanced threat prediction, Cloudburst Recovery, and Cleanroom Recovery.

SaaS data protector Druva announced its Dru generative AI copilot last month.

DDN’s petabyte scale and fast Infinia object store developed from the ground up

By
Chris Mellor
-

DDN is launching an Infinia scale-out object storage software product designed from the ground up for AI and cloud environments and optimized for flash drives.

Infinia has been developed because DDN realized it couldn’t bolt on new features to EXAScaler and develop the storage qualities needed to cover edge site, datacenter and public cloud distributed storage needs. Such data will inevitably need immense scale, shared resources and storage efficiency. It will be metadata-intensive, filtered, and used by a complex mix of applications – such as those involved in end-to-end AI model development and production. Data governance will also be a concern.

Eight years ago, DDN recognized that a new storage platform needed to be developed, joining companies such as Quantum (Myriad), StorONE, and VAST Data in this regard. Like Quantum, DDN has an existing customer base which it can use for storage development strategy validation, future needs identification, and help with product development and testing.

CTO James Coomer told us: “We’ve had a very talented bunch of people from all kinds of places, some ex-parallel file system people, some people from IBM, some people from enterprise storage companies. They’ve been sitting in their basement and basically writing a whole new, very ambitious storage platform called DDN Infinia – completely ground up built, not open source, just built by DDN engineers. Absolutely everything done from scratch; we’ve been very ambitious about it.” 

Infinia is not a development of DDN’s existing EXAScaler Lustre parallel filesystem-based product. It is software-defined and has separate control and data planes. The data plane is based on key:value object storage implemented in clustered pools formed from server/storage nodes. It has always-on data reduction, encryption, snapshots and multi-tenancy, with bulk recovery and elastic IOPS. 

The node systems can be any Intel, Arm or AMD server fitted with flash drives – be they TLC, QLC or PLC. Hardware reference architectures specify node hardware characteristics. 

The clusters are built up from Ethernet-connected nodes on-premises or in the cloud and can scale out to hundreds of S3 petabytes. They use network erasure coding and are fault domain and site aware. There are no restrictions on object size or metadata, and there is multi-part upload support. DDN says there is sub-millisecond latency for Puts and Gets.

There are hybrid I/O engines autonomously optimizing for low latency, sequential streaming data,  drive IOPS and metadata-intensive workloads. There is no read/modify/write overhead, as log-structured operations are used. The storage operations are byte-addressable with wide striping for large I/O blocks. Capacity space for both metadata and data is elastic.

The Infinia keyspace has a tree structure to track site (realm), clusters, tenants, sub-tenants and data sets. It is sharded across the entire Infinia infrastructure. All tenants and sub-tenants are assigned a share of Infinia’s resource performance and capacity when they are created. This prevents noisy neighbor problems happening. The capacity is thin-provisioned and each tenant’s and sub-tenant’s data is encrypted. Service levels can be dynamically adjusted.

New tenants and services can be deployed in around four minutes. Infinia has S3 support now, and coming block, file and SQL access protocols layered on top of the data plane. These data services are lightweight and there is containerized code supporting S3, CSI and Cinder.

Infinia’s control plane looks after metadata, and is multi-tenant, has API management, a GUI, a CLI, and cloud analytics. Software upgrades involve zero downtime and their are no LUNs to administer or manual system tuning.

Infnia can be used to build an object store, a data warehouse or a data lake. Building a data lake involves setting up a new tenant on a cluster, setting capacity boundaries and performance shares for the tenant in a quality of service policy. Then storage buckets can be created, objects put in them and the data lake content built up. The self-service setup process has four overall commands and takes tens of seconds.

DDN suggests Infinia can function as an enterprise S3 object store with high-throughput and low latency, multi-tenacy support, more simplicity than Ceph and much higher performance.

Infinia is suited for outer ring use cases in AI with DDN’s existing EXAScaler storage providing an inner ring with extreme scale, efficiency and performance.

The Infinia 2023 roadmap includes a full set of S3 object services, NVMeoF block services for Kubernetes, and an SQL query engine.

In 2024 DDN intends to add a parallel filesystem designed for AI and the cloud, an advanced database, and data movement.

1...99100101...438Page 100 of 438

ABOUT US

Blocks & Files is a storage news, information and analysis site covering storage media, devices from drives through arrays to server-based storage, cloud storage, networking and protocols, data management, suppliers and standards. We publish news, views, opinions and analysis of storage industry events and developments, aiming to keep storage users, developers, consultants, marketeers, salespeople, support staff, analysts and everyone else involved in IT storage up to date with what’s going on.

FOLLOW US

© Situation Publishing, 2018-2024