Kioxia has confirmed its IPO plans are on hold as it opened Fab7 at Yokkaichi in Japan with NAND foundry partner Western Digital.
The plans were filed in August 2020 and sought to raise $3.2 billion but were delayed in September that year when the US government banned certain technology exports to China. This affected future Kioxia revenues. Now the extension of these bans has caused a likely downturn in Kioxia exports to China. This, together with a NAND oversupply situation, threatens to depress Kioxia’s revenues.
The IPO would have provided cash for beleaguered Toshiba, which owns 40 percent of Kioxia. Toshiba is talking to bidders about a potential buyout of its poorly performing and loss-making company. The flotation would also provide investment cash for Kioxia.
Bloomberg quoted president and CEO Nobuo Hayasaka as saying: “Now is not the time. We remain on the alert for the best possible timing for an IPO, but as always will also evaluate other financial options.”
New Kioxia fab
Fab7
On a brighter note, Fab7 will produce 162-layer 3D NAND chips, the sixth generation of Kioxia and WD’s BiCS technology, followed by later generations, such as BiCS 7’s 212-layer flash. The 162-layer product will ship in early 2023. Fab7’s total cost will be ¥1 trillion ($6.8 billion).
Hayasaka said: “Fab7 is the latest and most technologically advanced semiconductor manufacturing facility in Japan and will be indispensable for Kioxia’s future. Long-term global demand for memory products is expected to increase as the world consumes more data across a variety of connected devices.”
Kioxia and WD’s Fab7 cleanroom
In other words, although we’re currently seeing a relatively short-term dip in NAND demand, the medium and long-term picture is great. Kioxia and other NAND manufacturers, including Micron and SK hynix, are cutting capex and lowering production output as they ride out the supply glut and wait for customers to use up their flash chip inventories, and for smartphone and PC demand to recover.
Dr Siva Sivaram, President of Technology & Strategy at Western Digital, said: “With its innovative design and production efficiencies, the new Fab7 facility underscores Western Digital’s commitment to deliver sustainable memory and storage technologies to our customers globally. We value our tremendous relationship with Kioxia.”
This is a nod to the end of disruption to the relationship during prior WD CEO Steve Milligan’s tenure.
Fab7 uses AI techniques to increase production efficiency and has a larger cleanroom area relative to its overall space than prior Kioxia fabs, meaning a larger manufacturing area. It has an earthquake-resistant design and Kioxia says it will use energy-efficient equipment to increase its sustainability rating.
The Flux Web 3.0 decentralized and cryptocurrency-based compute network is adding storage services based on IPFS, the Inter-Planetary File System.
Web 3.0 is a concept being pushed by blockchain and cryptocurrency enthusiasts as an open and accessible alternative to what they see as the centralized and supplier-dominated internet of today – which they call Web 2.0. What Filecoin is to decentralized storage, Flux is to decentralized compute, but now it is stepping on Filecoin’s turf.
Co-founder Daniel Keller said: “At Flux, we have made it our mission to offer business and developers operating on Web2 cloud services the same experience in a Web3 environment, and the launch of FluxIPFS is our latest step in doing so… We have added FluxIFPS to the Flux platform to strengthen our current portfolio of product offerings and further establish Flux as the one-stop shop for those looking to launch decentralized applications in a cloud environment.”
IPFS is a content-addressed peer-to-peer file sharing network based on linked IPFS hosts, each holding a part of a file. Every file has a unique ID in an IPFS global namespace.
Keller wants users to be able to “deploy decentralized applications using familiar tooling found in Web2 cloud services, like decentralized file storage.” It would operate in way similar to BitTorrent with an F-drive concept.
The company is bringing FluxIPFS to its ecosystem to provide this file storage for NFTs, static web pages, file services, and images. This move brings it into competition with Filecoin, Storj, and other decentralized storage providers.
Dazz Williams, IPFS development lead, said: “Business leaders are looking for ways to speed up their digital transformation and we are pleased to offer a decentralized file-sharing network to help protect against infrastructure damage, offer immunity from data corruption, and provide an invaluable backup in the Web3 space. We will continue to help Flux grow the quality of its product portfolio by developing even greater capabilities through IPFS.”
FluxIPFS is in beta and more will be revealed by Flux in coming weeks. There is no information about the file access protocols to be used or file IO speed. Don’t bother searching for FluxIPFS in the Flux website; it’s not present, except briefly in a roadmap webpage.
The Flux network
The Flux network uses a native POW (Proof-of-Work) coin (Flux cryptocurrency) to power its ecosystem, providing incentives for hardware hosters. The network is based on Fluxnodes’ decentralized infrastructure, FluxOS cloud operating system, Zelcore self-custody multi-asset wallet and blockchain app suite, and Flux blockchain for on-chain governance, economics, and parallel assets to provide interoperability with other blockchains and DeFi (Decentralized Finance) access.
FluxNodes are operated in a decentralized manner by individual holders, and geographically spaced all over the world, so that applications are accessible by users at all times and locations. Operators of Flux nodes can choose from three tiers of hardware requirements to stand up after providing the necessary Flux capital soft-locked in their wallet. Flux says its ecosystem technology allows anyone to be rewarded for providing hardware to the network, from anywhere in the world.
DeFi is a general term for peer-to-peer financial services on public blockchains, such as Ethereum. Flux is mined using the ZelHash algorithm. ZelHash is a GPU minable implementation of Equihash 125_4, harnessing ASIC/FPGA resistance, developed by Wilke Trei of lolMiner.
Solid waste after being shredded to a uniform size.
Verity ES is a company that permanently deletes data from storage devices so they can be recycled instead of being destroyed, with three-pass overwrite deletion of an 8TB disk drive estimated to take three days.
Existing data sanitization and IT asset disposition supplier Revert has set up Verity ES – ES stands for Eradication Software – to delete drive data. This enables customers to recycle used-up and unwanted drives without compromising their ability to keep user data secure under the various compliance and regulatory obligations applicable in various geographies.
Saeed Karim.
Saeed Karim, CEO of both Revert and Verity ES, provided a canned quote saying: “The Verity ES certified, patent-pending enterprise software provides industry-leading data eradication security, economics and operational support, enabling organizations to permanently eliminate sensitive and private data extremely quickly, easily and across the widest range of devices and storage media.”
What does that mean?
Because the penalties for infringing compliance and privacy regulations can be financially severe, many if not most organizations totally destroy old drives in a shredding process that transforms the drives into fragments of scrap metal – e-waste suitable only for landfill. The Verity ES software ensues that data on a drive is replaced by meaningless signals and enables customers to then sell off unwanted drives – earning some revenue from them and reducing their e-waste burden.
Verity ES suggests that larger datacenter customers can end-of-life 500,000+ devices annually, with residual values ranging from $5 to $10,000+ per device, depending on the age, condition and media type. One customer reportedly gained $2.7 million in residual value from Verity ES’s ability to eradicate data from approximately 36,000 HDD and SSD disks. A competitor’s software failed to remove the data, according to Verity ES, which wants us to know it has a much better data eradication product.
Its software runs on a customer-built appliance and deletes data on a drive by overwriting it and doing so in three passes. A data analytics module provides real-time monitoring of operational performance using the data collected during eradication processing.
The software enables parallel data eradication processing of 500+ devices from a single appliance without any performance degradation. It supports country-specific data erasure regulations and standards for more than 25 countries. It also automatically determines the quickest and most efficient method to eradicate data on each device and media type, and supports disk drives and SSDs.
Verity says it “increases eradication success rates and yields” – which implies that the success rate is not 100 percent. Karim’s quote also said eradication took place “extremely quickly” and left us wanting to know more.
Checking the details
Glenn Jacobsen.
We asked Glenn Jacobsen, the president and COO of Revert, some detailed questions about the data eradication process.
Blocks & Files: How long does it take to eradicate data (overwrite) disk drives – eg, an 8TB drive, an 18TB drive? And how long does it take to eradicate data on for example an 8TB SSD?
Glenn Jacobsen: It depends! If the manufacturer and system vendor firmware supports embedded sanitization functionality per the ATA and SCSI standards, the time to sanitize an HDD and SSD can be reduced by up to 50 percent. With Verity, an NIST PURGE-compliant three-pass overwrite with 100 percent verification for an 8TB HDD or SSD will take approximately 70 hours (3 days), while an 18TB HDD will take approximately 150 hours (6.5 days).
Blocks & Files: How is the eradication process verified? How long does the verification take?
Glenn Jacobsen: It depends on the verification method selected. NIST stipulates a full read of a minimum ten percent representative sampling of the media, which can be a pseudorandom sampling or a predetermined selection of user addressable and reserved areas. A full verification is one full read of every bit, byte, sector (cell or page) in user addressable and reserved areas. The time for a full verification equates to the time it takes to perform a single overwrite – so for an 8TB HDD or SSD, it will take approximately 20 hours.
Blocks & Files: Are drives that are going to be eradicated mounted in a Verity ES appliance?
Glenn Jacobsen: Not at this time. Verity ES will provide the customer with the system prerequisites and consultation to build their own platforms on which to install Verity ES. That said, if there is customer demand, the sale or lease of a Verity ES appliance will be considered.
Blocks & Files:Can the Verity ES system support SATA, SAS and NVMe-attached drives? How about M.2 format SSDs?
Glenn Jacobsen: Yes, Verity ES supports SATA and SAS HDDs, SATA and SAS SSDs, NVMe SSDs, and m.2 SSDs.
Blocks & Files: What is the cost advantage of Verity ES vs drive shredding?
Glenn Jacobsen: Drive shredding depends on the method selected – on-site or off-site. On-site drive shredding typically costs $15–$20 per drive, and introduces the following issues:
Does the size of the shredded remnants fully mitigate the risk of unauthorized access to data? This is especially important for SSDs.
Where are the shredded remnants transported after the on-site shredding?
What is the disposition of the shredded remnants? Will they be recycled or landfilled? Exported to another country?
How does shredding support the sustainability goals and objectives of the business? With shredding, there are environmental concerns that must be considered, including carbon emissions (shredder, transport, recycling) and disposition of the shredded remnants (recycling or landfill?).
Off-site shredding introduces the primary risk of data breach. If drives containing PII or ePHI are released from the four walls of the datacenter, the data is no longer in control of the processing organization and technically constitutes a data breach. Chain of custody documentation typically applies to just the physical asset itself, not the data that is stored on the asset. This should be a non-starter when going through a decision-making process.
The ES data eradication enterprise software is available immediately from Verity ES.
Dell says it is providing Identity & Endpoint Protection with Microsoft Zero Trust services to protect Microsoft users.
It is now assumed that malware and other types of attack will sometimes break through perimeter security defences, like firewalls, and enter an organisation’s IT network. The appropriate strategy is to minimize the internal attack surface and provide fast and highly granular recovery services for data that has been encrypted by ransomware or otherwise corrupted. Zero trust ideas treat every user as a potential attacker and verifies their data accesses whilst granting them the least privileged access status they need, so limiting their blast radius.
Mustafa Hamid.
Mustafa Hamid, global discipline lead for the Modern Workforce in Dell Technologies Consulting Services said: “Since many of our customers rely heavily on Microsoft technologies for collaboration and Identity and Access Management, we are expanding our Zero Trust offerings by now introducing Identity & Endpoint Protection with Microsoft Zero Trust.”
An issue with zero trust approaches is that they need to be closely integrated with the IT environment that is being protected. Hence Dell is working with Microsoft to add zero trust principles in a service for identity and endpoint protection of its systems operating within the Microsoft environment.
Earlier in October, Dell announced Zero Trust cybersecurity services along with a Zero Trust Center of Excellence that uses a US Department of Defense-approved architecture. Now Dell is adding to that by supplying a suite of services for Dell + Microsoft customers that provide:
Assessment of Active Directory and Azure Active Directory and other Microsoft items to discover existing cybersecurity status and generate recommendations.
Defence using Microsoft’s Defender suite, integration with Microsoft’s Sentinel SIEM product, ServiceNow and similar services, plus Dell services to secure endpoints, MS 365 and Active Directory.
User education to increase and sustain awareness of cybersecurity risks and adopt a better security posture.
Managed services to provide incident detection and response for customers wanting to outsource these functions.
An aspect of zero trust is that a customer’s IT system is spending an appreciable amount of its resources checking user and application resource requests. We can all remember anti-virus products that slowed our PCs down because of the continuous and lengthy checks they carried out on every data access and activity.
Hamid points out that the path to Zero Trust looks different for every organization and involves a complex journey to implement this type of system architecture. If you take the attitude that, literally, data and resources are inaccessible by default, then every data and resource accessed by a user and applications they invoke needs verifying. In effect every user has a shadow security monitor, aka digital security guard, watching everything they do and checking to see if it’s permitted.
The newly announced Alta cloud-based software from Veritas combines the company’s entire cloud services and solutions portfolio to create a cloud data management platform that oversees and manages a customer’s data estate.
Lawrence Wong
We caught up with SVP and chief strategy officer Lawrence Wong to ask about the latest addition to its product line.
Blocks & Files:Veritas says the Alta View management console “provides a unified view and control of the entire data estate – across edge, datacenter, and cloud – from a single pane of glass.” If we take that literally, it includes data in every conceivable digital location: databases, data warehouses, data lakes, mainframes, IBM Power servers, all SaaS applications, backup repositories, archives, etc. I’m skeptical. Can Veritas list the data sources in “the entire data estate” that Alta View can see?
Lawrence Wong: That’s a valid point – we’re talking here about the combination of the “entire Veritas data estate”, plus the data that we can see or act upon “outside” of the Veritas estate. Alta View is underpinned by technology from our IT Analytics engine, which empowers customers subscribing to our higher tiers of service to monitor and view their data both inside and outside of the Veritas estate. Using Veritas Alta View, organizations can track their consumption and billing patterns of network, storage and compute within their tenant of the CSP (Azure and AWS), as well as receiving similar analytics and insight into traditional on-premises storage systems. Veritas Alta View supports a number of third-party data protection vendors as well as several on-premises storage systems, and has a standard interface for non-natively supported platforms which can be used to ingest a target source of data.
Alta View will initially be available to customers via early access in December 2022, with a GA targeted for Q1 CY2023. It will be delivered as a service and will provide a unified console for multi-domain management of all NetBackup domains, including domains on-premises, in the cloud and across multi-clouds. All NetBackup workloads, appliances, and data repositories will be supported. It will provide a single pane of glass for global job and policy management, critical event notifications, host deployments, and more – ultimately simplifying daily operations across the data protection estate. As we’ve said, the Veritas IT Analytics engine is integrated into Alta View for both the Veritas data estate and targets outside of the Veritas data estate. These capabilities will be available in the first release.
Blocks & Files:CEO Greg Hughes said: “Veritas Alta advances our mission to empower customers to own and control their data no matter where it resides – on-prem or in the cloud – while reducing the operational complexity and costs associated with managing multi-cloud environments.” Again, taken literally, this applies to all of a customer’s data. Is that true? If not, then exactly what data possessed by customers can Alta be applied to so that it can reduce operational complexity and costs in multi-cloud environments?
Lawrence Wong: As Greg indicated, it is very much our mission at Veritas to help our customers gain control over their data – ensuring protection, eliminating vulnerabilities, maintaining availability, gaining insight, and enabling compliance with regulatory policies.
For protection of multi-cloud environments, we do this in a number of ways. Our cloud autoscaling, delivered with Veritas’s cloud-native architecture (Cloud Scale Technology) optimizes compute resources in the cloud. The combination of our cloud intelligent policies, automated backup from snapshots, as well as our improved and highly efficient deduplication, ensure that data is stored in the most cost-effective way, in the cloud tier of the customer’s choice.
When it comes to operational resiliency, Veritas Alta empowers customers to move data simply across multi-cloud environments. This enables them to escape the complexity and cost associated with vendor lock-in. For example, with Veritas Alta, it is easy to fail over to alternative CSPs, even in different geographies. Customers can also maintain high-performance while migrating to lower-cost storage than using CSP-defined tiers.
Veritas Alta can also identify and retire stale or orphaned data across a customer’s cloud environment, and reclaim unused infrastructure, automatically classify data such as PII (Personally Identifiable Information), identify security risks, monitor communications, and perform eDiscovery across over 120 content sources, both on-premises or in the cloud.
Customers using Alta analytics can also view and analyze their cloud resource consumption from storage to compute to ensure they operate with optimal efficiency.
All of these capabilities help reduce customers’ costs, risks, and potential regulatory exposure, and all of them are part of our Alta platform moving forward.
Blocks & Files: In what way does Alta have “unmatched ransomware resiliency”? I would have thought any supplier with immutable backup stores and granular recovery would match Alta. And ones offering ransomware recovery guarantees could more than match Alta?
Lawrence Wong: While many competitors offer immutability capabilities, that does not mean they are always able to recover from an attack. Veritas has an 100 percent ransomware recovery rate, albeit this is not something we promote aggressively in our marketing, but rather we share it directly with customers in our engagements – and, of course, we shared it privately with you during our IT Press Tour briefing. The reason for this is that we feel this is something our customers care deeply about and, were we to we tout broadly, for example, in a press release, we would essentially just be painting a bullseye on our back.
Regarding the Ransomware Guarantee comment, we have found in our conversations with customers that they care more about delivering ransomware resiliency than they do marketing gimmicks such as ransomware guarantees that typically prove to be riddled with disclaimers/caveats. Our focus here remains on delivering on what our customers NEED so that they can focus on getting back to business.
With that 100 percent recovery rate under our belt, we also think that it’s time to move the conversation on. Ransomware resiliency shouldn’t be a binary metric where you either do or don’t get your data back. Instead, we’re focused on making the process of bouncing back as fast and as simple as possible for our customers. Imagine if airlines were claiming that every flight eventually got to its destination with all the passengers still onboard? That’s the level that many of these ransomware “guarantees” pitch themselves.
Customers using competitive products are coming to us saying, “we can recover 2TB per hour today – which is great, but we have 400TB of critical infrastructure to restore and can’t be down for 200 hours.”
Veritas is solving this challenge for customers with features like:
Scalable Instant Access
Parallel streaming to restore large databases more quickly
Server elasticity to burst on restores
We believe that these issues are going to be a growing concern for businesses that haven’t standardized on Veritas for their data management. Staying ahead of these challenges is a huge focus for us, and will remain so going forward … as we continue to deliver against this in our product capabilities.
Blocks & Files: Our understanding is that Alta represents a bundling and integration of existing Veritas cloud-based products such as:
Enterprise Vault.cloud SaaS archiving, eDiscovery and Supervision platform
Merge1 to capture and archive high-risk data across all communication platforms
Advanced eDiscovery
Veritas Advanced Supervision – compliance framework for capturing content in regulated communications through Merge1 integration
Lawrence Wong: Veritas Alta is much more than bundling of our existing product set. It’s about delivering a cohesive multi-cloud platform for our customers. Alta will provide a single place to interact with all of our cloud-first services from our protection, application resiliency and compliance portfolios. It also includes new services, such as Veritas Alta View.
The services you mention in your question are all from our compliance and governance portfolio. Through Veritas Alta Data Compliance and Governance, one of the pillars under Alta, we can now deliver integrated support for organizations that want to adopt an “as-a-Service” approach to their data capture, archiving, surveillance, and discovery. It is worth noting that our Alta Data Compliance and Governance SaaS product has more features and capabilities than its corresponding on-premises standalone software versions (for example, Enterprise Vault, Merge1, etc.).
Blocks & Files: Is the Alta in-cloud backup capability a new cloud-native data protection product written by Veritas or is it an implementation of NetBackup or BackupExec in the cloud?
Lawrence Wong: Alta Data Protection is powered by Cloud Scale Technology, Veritas’s cloud-native architecture that also underpins NetBackup. Specifically, this is a Kubernetes microservices-based architecture and deployment of NetBackup that runs in AKS and EKS. We debuted Cloud Scale Technology in NetBackup 10 and continue to evolve it in each release. Alta Data Protection is available to customers with a choice of consumption model – as a customer-managed software deployment, and with selected workloads, as SaaS today. We will be releasing additional workloads that can be consumed as SaaS when we formally launch our full Backup-as-a-Service offering later this year.
Everything in Veritas Alta Data Protection is based on what NetBackup already supports, extending our 30 years of enterprise-class experience and hardened capabilities to the cloud.
BaM – Block access Memory provides high-level abstractions for accelerators [GPUs] to make on-demand, fine-grained, high-throughput access to storage while enhancing the storage access performance. To this end, BaM provisions storage I/O queues and buffers in the GPU memory.
It was developed by a collaboration of industry engineers and academic researchers at NVIDIA, IBM, the University of Illinois Urbana-Champaign, and the University at Buffalo. The approach maximizes the parallelism of GPU threads and uses a user-space NVMe driver, helping to ensure that data is delivered to GPUs on demand with minimal latency.
Logical view of BaM design.
A research paper, GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture, says BaM stands for Big accelerator Memory, and it “features a fine-grained software cache to coalesce data storage requests while minimizing I/O amplification effects. This software cache communicates with the storage system through high-throughput queues that enable the massive number of concurrent threads in modern GPUs to generate I/O requests at a high-enough rate to fully utilize the storage devices, and the system interconnect.”
It goes on to say: ”The GPUDirect Async family of technologies accelerate the control path when moving data directly into GPU memory from memory or storage. Each transaction involves initiation, where structures like work queues are created and entries within those structures are readied, and triggering, where transmissions are signaled to begin. To our knowledge, BaM is the first accelerator-centric approach where GPUs can create on-demand accesses to data where it is stored, be it memory or storage, without relying on the CPU to initiate or trigger the accesses. Thus, BaM marks the beginning of a new variant of this family that is GPU kernel initiated (KI): GPUDirect Async KI Storage.”
There is a BaM software cache in the GPU’s memory. The research paper discusses GPU thread data access to it, and says: “If an access hits in the cache, the thread can directly access the data in GPU memory. If the access misses, the thread needs to fetch data from the backing memory [NVMe SSDs]. The BaM software cache is designed to optimize the bandwidth utilization to the backing memory in two ways: (1) by eliminating redundant requests to the backing memory and (2) by allowing users to configure the cache for their application’s needs.”
To fetch data from the NVMe SSDs: “The GPU thread enters the BaM I/O stack to prepare a storage I/O request , enqueues it to a submission queue , and then waits for the storage controller to post the corresponding completion entry. The BaM I/O stack aims to amortize the software overhead associated with the storage submission/completion protocol by leveraging the GPU’s immense thread-level parallelism, and enabling low-latency batching of multiple submission/completion (SQ/CQ) queue entries to minimize the cost of expensive doorbell register updates and reducing the size of critical sections in the storage protocol.”
A doorbell register is a signaling method to alert a storage drive that new work is ready to be processed.
It says BaM is implemented completely in open source, and both hardware and software requirements are publicly accessible.
Veritas has announced its cloud-based Alta data protection and security software and Alta View dashboard to review and manage data in the public cloud, datacenters, and edge sites.
Alta “combines our entire cloud services and solutions portfolio” to create a cloud data management platform enabling customers to protect their application data in the public clouds”, says Veritas. They can use the Alta View dashboard to view and manage their data wherever it lies.
Greg Hughes
Veritas CEO Greg Hughes said: “Veritas Alta advances our mission to empower customers to own and control their data no matter where it resides – on-prem or in the cloud – while reducing the operational complexity and costs associated with managing multi-cloud environments.”
Taken literally, this applies to all of a customer’s data but that might be questionable. We think it is to do with data that Veritas can see and it’s directed at protecting public cloud data. We asked Veritas to clarify this.
Christophe Bertrand, Veritas Enterprise Strategy Group practice director, said: “Organizations are often confused about their data protection SLAs when they move their data and/or applications to the cloud, and may think that their responsibility to ensure the protection of these digital assets is somehow mitigated or just the vendor’s responsibility.
“The reality is much different: the data is in fact always the organization’s responsibility. In a multi-cloud environment, which is becoming the norm, this data protection disconnect can easily become exacerbated, in particular when organizations use many different and poorly orchestrated data protection and data management mechanisms. Limiting the number of standalone, bolt-on solutions is much needed, which is the challenge that Veritas is taking on with its Alta unified management platform.”
Druva? Clumio? OwnBackup? Nope, no need. The idea is to extend Veritas’s data protection and security scope to the public cloud, so persuading Veritas customers there is no need to buy other suppliers’ products to do this. Cloud-native Alta creates “the most comprehensive cloud data management platform built for any environment … Alta application resiliency solutions support all major cloud infrastructures, whether virtual, containerized, or bare metal,” claimed Veritas.
Alta functionality includes:
Data protection and ransomware resiliency
Five nines application availability
Application portability across clouds
Optimizing storage capacity usage
Increasing storage IO performance
Data compliance-as-a-Service to capture data from all communication platforms and automate content classification
Portfolio simplification
Our understanding is that Alta represents a bundling and integration of existing Veritas cloud-based products such as:
Enterprise Vault to capture and archive high-risk data across all communication platforms
Veritas Advanced Supervision – compliance framework for capturing content in regulated communications through Merge1 integration
The data protection capability is, we understand, NetBackup 10 and its Cloud Scale technology. Agaim, we have asked Veritas to confirm this.
Alta View
Veritas said the Alta View management console “provides a unified view and control of the entire data estate – across edge, datacenter, and cloud – from a single pane of glass.” If we take that literally, the “entire data estate” then includes data in every conceivable digital location; databases, data warehouses, data lakes, mainframes, IBM Power servers, SaaS applications, backup repositories, archives, etc. We have asked Veritas for the detailed scope.
Alta View is a cloud-based management console which uses a Veritas analytics engine, with AI and machine learning capabilities, to provide autonomous data management actions, lessening the need for manual activities. It looks to be a nicely powerful window through which to view and manage the data using the Alta products.
Customers can use Alta with on-premises, IaaS, hybrid, virtual, and SaaS consumption models. Veritas has a stack of downloadable PDF documents describing aspects of Alta, and you can start with a Solution Overview.
Panzura claims it’s the leading cloud file data services supplier, having overtaken CTERA and Nasuni in providing a global file environment.
Jill Stelfox
Multi-location businesses, particularly ones with remote workers, need to share files and manage their file data estate. The traditional way to do this is with filers at every on-premises location – but that does not suit remote access, and ensuring a single version of file truth with a single master file copy is difficult. A newer way is to have a cloud-based and centrally managed global file data service, with edge caching for speed, that supports file sharing and collaboration and is built for that purpose. Panzura, founded in 2008, helped pioneer this, and then encountered difficult times before being bought by private equity in May 2020.
CEO Jill Stelfox briefed Blocks & Files, and said she was part of the Profile Capital Management private equity investment group that bought Panzura in May 2020. “Since then, we’ve grown 485 percent” annually, she tells us, by providing “enterprise hybrid, multi-cloud data management.” Panzura is cash flow positive and this, she said: “puts us in a unique position in this space, I think we’re the only ones that are cashflow positive. … running a financially sound company might be unique in Silicon Valley.”
The company sees itself as providing a cloud-based distributed and global primary and secondary file data access and management platform that negates the need for on-premises filers. Its software is divided into four layers.
Panzura 4-layer software cake
The base pink layer is a set of global services to move and migrate data. The black layer is where the CloudFS (Cloud File System) unstructured data storage and management platform is located. CloudFS provides a global file namespace with distributed fie locking using a global metadata store. It uses snapshots to save file state and has global deduplication capabilities. You can read a technical white paper here.
A data services layer (yellow) sits above this, with a data management dashboard accessing search, audit, analytics, and governance facilities. Stelfox explained: “The number one reason we did Panzura Data Services is because CIOs are struggling with the answer to ‘What is where?’ And with [our] standard data services, you can know where all your data is.”
The top blue layer contains seven Panzura-written Smart Apps for data access, management, collaboration, and monitoring. They include:
An edge application to manage security and intellectual property on a mobile device such as an iPad or laptop;
Company Connect, which is the ability to set up a quasi-DMZ (de-militarized zone) – a safe place where companies can share files, even if they don’t have Panzura. This is used, for example, in selling mortgage applications to securities companies;
Panzura Protect, which provides ransomware alerts within two seconds of an attack taking place, telling you if it’s internal, where it’s coming from and if data is being encrypted;
Deep content search.
More are coming, with Stelfox saying: “We’re working on a global metadata catalog. That’s actually not complete yet, although we do have all the logistics and provisioning. You can choose where you want data to be; move this to an S3 bucket, move that to a different datacenter. We give you recommendations based on our AI/ML layer.”
Market positioning
Stelfox said: “We’re going after three different segments: data storage – that’s our original CloudFS products that we’ve purchased – and then data services and data intelligence.”
A couple of diagrams summarize Panzura’s view of its positioning versus its competition. First the overall view:
And then a more detailed view:
The competitive environment is formed from three kinds of vendors. First there are “the legacy guys, the NetApps and Isilons, and the’’re not really doing hybrid cloud.” For example, “the ONTAP offering … is really just pushing files to cloud. It’s not a true hybrid cloud system that connects files between servers and the cloud. It’s just a sort of migration tool.”
“Then there’s the niche players, the CTERAs and Nasunis, and they do have a product similar to CloudFS. It doesn’t scale to the size of Panzura, but it’s similar. But they don’t have any of the data services pieces that we do.”
Hammerspace, another niche player, has functionality similar to Panzura Data Services, “where they can see data on storage platforms and and bring insights around your data. But they don’t have the full storage capability of the data storage layer.”
The third group is the “Qumulos and the Cohesity that definitely do have a storage layer. They also have a little bit of data services. But all of that is on secondary data and not primary data – which is, in some respects, easier, because … you’re not worried about interrupting the user flow or segregating speed of user access to files or speed of rewrites; that gets saved.”
In her view: “It’s OK if it takes five minutes to do something [on secondary storage] because nobody cares. But can you imagine if it took five minutes [to write] a file? It’s just unacceptable.”
Stelfox said: “Qumulo talks about having a little bit of the data layer, like Cohesity does, but they they don’t do it for real-time data. And it’s not a robust protocol. But we’ve seen them in the field and competed against them in the field.”
In general, though: “We actually compete against NetApp and Isilon much more than Nasuni and CTERA.”
Comment
Imagine CTERA, Nasuni, Panzura, and Hammerspace offering a set of functionalities to global users which we place on the surface of a black box. They all have basically, a “cloud” approach as opposed to one centered on on-premises filers.
The “how” of providing these services is inside the black box and software engineers and architects can argue about the best way to architect and construct such services. They can explain how their approaches optimize performance, scale, costs, user productivity, and so forth.
Without objective benchmarks, the only way a customer can evaluate the different approaches in detail is with pilot installations and bake-offs – checking for functionality, ease of use and scalability.
Astera Labs demonstrated how datacenter operators can improve performance and reduce TCO (Total Cost of Ownership) with its CXL, PCIe 5.0, and Ethernet Connectivity solutions at the OCP Global Summit. The Leo Memory Connectivity Platform for CXL 1.1 and 2.0 is in pre-production sampling with strategic partners and customers and it is ready for deployment in cloud servers, the company said. Aries PCIe 5.0/CXL Smart Retimers are the industry’s most broadly deployed retimers and have passed interoperability testing with all major PCIe endpoints.
…
Block data migrator Cirrus Data Solutions said its Cirrus Migrate Cloud is now integrated with Oracle Cloud Infrastructure (OCI) and publicly available on the Oracle Cloud Marketplace. Cirrus Migrate Cloud enables organizations to move block data between any storage environments freeing enterprises from vendor lock-in, the company claims. Oracle Cloud customers now have the ability to move their on-premises mission-critical databases and applications to the OCI cloud.
…
Observability pipeline supplier Cribl and Cloudian announced a data analytics tieup that integrates Cribl Stream and Cloudian HyperStore object storage to provide an observability platform on an on-premises, S3-compatible data lake. This enables organizations to ingest, parse, restructure and enrich large data volumes in flight, so they get the right data in the formats they need securely behind their firewall. Users can convert logs into metrics, reduce cost, and increase search speed.
…
DataCore announced the new Perifery Panel for Adobe Premiere Pro to streamline access to content for DataCore Swarm and Perifery users on Adobe Premiere Pro, designed to improve the efficiency of media production workflows. It allows users to enrich the metadata of searched objects and speeds up search for media assets and has a seamless connection to DataCore’s Perifery-powered appliances and Swarm object storage software platform, the company said.
…
The immudb, the only immutable enterprise-scale database with cryptographic verification, today added FIPS 140-2 compliant cryptography support, meeting Federal Risk and Authorization Management Program (FedRAMP) requirements, as well as a performance boost and synchronous replication. Codenotary co-founder and CTO Dennis Zimmer said: “We”ve achieved a 1080 percent performance increase with this latest release of immudb which can handle millions of transactions per second and serve as a high performance alternative to blockchains and other ledger technologies. We are seeing rapid adoption with 15 million downloads and have more than doubled the number of GitHub stars over the past year to nearly 8,000.”
…
Immuta announced product enhancements and integrations to deliver more data security and monitoring across cloud platforms at scale. These include native integration with Google BigQuery, expanded integrations with Snowflake, including external OAuth support, audit log data export into Amazon S3, and better policy onboarding for Databricks. Read more in a blog.
…
Intel announced v2.2 of its DAOS high-performance computing parallel filesystem, adding Rocky Linux 8 and Alma Linux 8 support. CentOS Linux 8 support is removed. Support for the libfabric/tcp provider is added (replaces libfabric/sockets). UCX support is added as a Technology Preview. Release notes are here.
…
TrueNAS opensStorage vendor, iXsystems has opened its new and largest campus in Maryville, Tennessee (outside of Knoxville). iXsystems grew TrueNAS revenues 70% year-over-year in 2021. It launched the next generation of its TrueNAS R50 Unified Storage Appliances with a modular architecture enabling controller upgrades or replacements. TrueNAS R50 systems can grow to over 1PB of HDD capacity and up to 60TB of NVMe flash in 4U of rack space. With expansion shelves, a system can expand to up to 5PB of hybrid storage capacity with 100GbitE networking. TrueNAS SCALE clustering and enclosure management are available for scale-out customers, and migration from scale-up is also available.
…
A Los Alamos Nuclear Labs (LANL) Tycho supercomputer consists of 2,600 Sapphire Rapids processor nodes with DDR memory. The filesystem, for the first time in the lab’s supercomputing history, will comprise only solid-state drives.
…
NVMe/TCP storage array supplier Lightbits Labs has been assigned patent 11,467,730 for “system and method for managing data storage on non-volatile memory media.” This describes a QLC-optimized caching policy that only writes to the flash media when it is the right time to do so and so improves flash endurance.
…
Cloud file services supplier Nasuni has announced an expanded European presence in DACH (Germany, Austria and Switzerland), appointing Michael Wappler as Channel Sales Manager and Ann Thueland as Head of EMEA Marketing. Nasuni’s revenue in DACH increased by 29 percent in the first half of 2022, compared with the whole year of 2021, Nasuni said. The number of Nasuni users in the region has doubled since 2019.
…
NetApp has announced a new partner program: NetApp Partner Sphere. The program’s tiers range from Approved to Preferred, Prestige and Prestige Plus – assessing, validating and recognizing partners based on value and competencies. Benefits and support will grow as partners ascend tiers, with the partners generating the most impact seeing the broadest and most customized benefits, including solution specific marketing campaigns and enablement, proposal-based market development funds, and preferred value-based incentives. As part of these changes NetApp will shift away from Specializations to identify partner capabilities and utilize Solution Competencies aligned to key NetApp focus areas: Cloud Solutions, Hybrid Cloud and AI & Analytics. These competencies will be included in the NetApp Partner Sphere program requirements.
…
FPGA-based analytics processor and all-flash system supplier NeuroBlade has qualified Kioxia as an FL6 (SCM-class SSD) NVMe drive supplier for the NeuroBlade Hardware Enhanced Query System (HEQS). The FL6 Series bridges the performance gap between DRAM and NAND flash memory. It delivers 60 DWPD (Drive Writes Per Day) endurance and performance of up to 1.5 million random read IOPS and 400K random write IOPS, with 29μs read latency, and 8μs write latency, respectively.
…
Nutanix announced general availability of Nutanix Cloud Clusters (NC2) on Microsoft Azure’s dedicated bare metal nodes. NC2 on Azure enables customers to deploy and manage their workloads in their own Azure account and VNet enabling them to keep the operating model simple and consistent between Azure and on-premises, Nutanix told us. It is available in North America Azure regions, with additional global Azure regions to follow in 2023.
…
Red Hat‘s Stratis filesystem, providing local storage management for Linux, has been updated to v3.3 and the Stratis daemon supports extra space made available on a device, useful when a RAID array is expanded. There is a a stratis pool extend-data CLI command to get the pool to use additional space on devices. Thin pool metadata fragmentation should be reduced in v3.3 as well. Read more in a blog.
…
Streaming database startup RisingWave Labs announced a $36 million A-round of funding led by multiple partners, including the investment arm of a leading global gaming firm, Yunqi Partners, and several individual investors. It brings total funding to over $40 million for this developer of the RisingWave open-source, cloud-native SQL streaming database for real-time applications. Yingjun Wu, founder and CEO of RisingWave Labs, was a software engineer at Amazon Web Services, where he was a member of the Redshift data warehouse team, and before that a researcher at the Database group in IBM Almaden Research Center.
…
Seagate has announced the launch of Lyve Cloud Analytics platform that includes storage, compute, and analytics, to help Lyve Cloud customers lower the total cost of ownership (TCO) with their DataOps and MLOps (machine learning operations). Seagate said it enables enterprises to activate their stored data in an open data lake architecture for analytics at petabyte scale, accelerating time to implementation and innovation, and yielding up to 40 percent cost savings compared to other available offerings.
…
Seagate has built an evaluation version of an NVMe hard disk drive with no bridges and no emulation needed for the OCP Summit. It demoed an NVMe HDD JBOD at the 2021 OCP Summit. The OCP organization is due to release an NVMe HDD spec soon.
…
StorPool Storage announced an update to its hosted analytics instrument that collects hundreds of metrics per second to deliver insights into each StorPool storage system’s performance, reliability and availability. This is at a per-second granularity for the previous 48 hours and per-minute for the most recent 365 days. The update adds authentication with Google accounts, per-customer login to see all clusters, and a general performance boost, with accelerated graphics rendering speed. It also impacts the cloud-based monitoring console that converts the collected metrics into per-node and whole-system status reports and alerts. These two instruments, included free of charge in each StorPool Storage license, make up the StorPool Analytics and Monitoring Suite that is used to get insights into the overall condition of the 30+ country global fleet of StorPool storage systems.
…
Kubernetes app data protector Trilio has extended Red Hat OpenShift support with TrilioVault for Kubernetes v3.0. It provides data protection policies for “fleets” of Kubernetes clusters in alignment with Advanced Cluster Management (ACM), and near-instantaneous OpenShift application portability and recoverability across any cloud or storage via Continuous Restore. This offers faster levels of replication, restoration and migration of Kubernetes data and metadata from any cloud or storage platform to another, Trilio said. TrilioVault also supports the OpenShift Service on AWS (ROSA) and Microsoft’s Azure OpenShift Service (ARO).
…
Cloud storage supplier Wasabi said it has integrated of its cloud storage service with Telestream DIVA, an object content management system for digital media, allowing users to transfer media assets from their on-premises storage systems to Wasabi for secure cloud storage that is accessible from any location.
…
HPE’s Zerto operation released findings from its 2022 Ransomware Strategy Survey conducted at VMware Explore US in August/September 2022. It revealed gaps in companies’ data protection and ransomware strategies. They should Buy Zerto offerings to reduce their risk of interrupted business operations in the event of a ransomware attack, HPE said.
Gartner’s latest Distributed Files and Object Storage Magic Quadrant lists two newcomers, “visionary” Cohesity and “challenger” VAST Data.
Update: VAST Data statement added. 24 Ocober 2022. And, courtesy of Pure Storage, the full report can be downloaded here and it is ungated. 28 Oct 2022.
There were 15 suppliers in the 2021 report, when Pure Storage was promoted to ‘Leaders’ box status, and both Nutanix and Weka entered for the first time. Now we have 17 suppliers with Cohesity and VAST making their first appearance. Here are the previous two year’s distributed files and object storage MQs:
And the new one:
Cohesity CMO Lynn Lucas said in a statement: “Our customers love SmartFiles as a great way to optimize cost, scale, and efficiency for their unstructured data… SmartFiles allows [customers] to manage, secure, and defend their data with next-level intelligence. We believe this recognition from Gartner underscores the value and innovation that the Cohesity platform provides to customers looking to simplify unstructured data management.”
Based on a balance between Ability to Execute and Completeness of Vision, Dell leads the Leaders’ box with IBM positioned in second place and Pure in third (Pure was fifth behind Scality a year ago. Now it has overtaken Scality and fourth placed Qumulo).
The SmartFiles product is based on Cohesity’a scale-out Span File System, which can receive and present files using NFS, SMB and S3 protocols at the same time, and tier files to the cloud.
Cohesity is also ranked as a Leader in the 2022 Gartner Magic Quadrant for Enterprise Backup and Recovery Software Solutions.
VAST Data has a lower Completeness of Vision score than Cohesity but a higher Ability to Execute. It leads the businesses in the Challengers box based on the balance between Ability to Execute and Completeness of Vision. Huawei, Hitachi Vantara and Cloudian are the other Challengers.
Co-founder and CMO Jeff Denworth said: “Today, we achieved a major milestone in our mission by landing on the 2022 Gartner File and Object Magic Quadrant. Our debut is particularly notable because of our placement… where we’ve landed the highest on the quadrant of any new vendor since the inception of the MQ. … We are out-executing with a plan built on a vision that is both anchored on the future of computing as well as pragmatic in terms of its potential to be executed upon.”
DDN and Inspur are the two Niche Players.
“Visionary” NetApp has failed to break into the Leaders’ quadrant this year, and Nutanix has joined the quadrant close to the boundary of the Leaders’ box. Weka and Quantum are visionaries still, as is Red Hat, acquired by IBM yet listed separately.
MQ explainer
As we wrote 12 months ago, the files and object storage MQ is produced once a year and features the well-known Leaders, Challengers, Niche Players and Visionaries quadrants in a chart with Ability to Execute and Completeness of Vision axes. Suppliers are scored on various attributes of these two concepts and then placed in the MQ diagram according to their summed and weighted scores, with a presence towards the top right being desirable. That area has the highest ability to execute and completeness of vision.
Samsung has followed up its 980 Pro with the even faster 990 Pro, which beat Solidigm’s recently announced P44 Pro as a gamer’s gumstick SSD.
Update. Solidigm P44 Pro layer count corrected to 176. Solidigm internal testing showing 7.4GB/sec performance added. 22 Oct 2022.
The prior 980 Pro is a M.2 PCIe 4 drive built with 112-layer 3D NAND in TLC (3 bits/cell) format. It delivers up to 800,000/1,000,000 random read/write IOPS, 6.9GB/sec sequential writes, and 5GB/sec sequential reads across its PCIe 4×4 interface.
The 990 Pro uses newer 176-layer V-NAND, again in TLC format, and pushes out up to 1.4 million/1.55 million random read/write IOPS (at queue depth of 32) and sequential read/write throughput of 7.45/6.9 GB/sec. There are 1 and 2TB versions available now with a 4TB model coming next year. DDR4 SDRAM is used for caching; 1GB for the 1TB variant and 2GB for the 2TB. The average power consumption is 5.4W at 1TB and 5.5W at 2TB.
Performance comparisons
This newer 990 Pro is a much faster drive than the 980 Pro, and edges out Solidigm’s P44 Pro in performance terms:
A Solidigm spokesperson took some exception to this and said: “From a pure datasheet comparison, the 990 might look superior, however, Solidigm specs our datasheet to account for all motherboards and systems in the market across all supported platforms. Thus, the data sheet numbers are typically conservative compared to what the device is able to achieve. Please note In our internal testing we have been able to hit 7.4GB/sec Seq Read on Solidigm P44 Pro.”
The 990 Pro beats Western Digital’s Black SN850 as well, with its 1,000,000/720,000 random read/write IOPS and 7.0/5.3 GB/sec sequential read/write throughput. Fantom’s Venom8 is also just about surpassed, with its 7.4/6.9GB/sec sequential read/write throughput.
In fact, this Samsung gaming SSD is faster in IOPS and bandwidth terms than Kioxia’s storage-class memory FL6 drive, with its 1.5 million/400,000 random read/write IOPS and 6.2/5.8 sequential read/write throughput. The FL6’s latency is 29µs read and 8µs write. Samsung hasn’t revealed the 990 Pro’s latency details.
The 990 Pro uses an in-house Samsung Pascal controller and comes with or without a heatsink, which has fancy RGB lighting that comes with a lot of gear aimed at gamers.
The drive can be used with Sony’s PlayStation 5 console and laptops compliant with the PCI-SIG D8 standard. It comes with Samsung Magician management software with diagnostic functions, software updates, and so forth.
The 990 Pro supports AES 256-bit and TGC/Opal encryption. It has retail pricing of $169.99 for the 1TB version and $289.99 for 2TB; and $189.99 and $309.99 for the heatsink variants respectively.
Big Blue is active afresh in the archive-storing tape library business, with its Diamondback product for web-scale enterprises and hyperscalers, claimed to be the industry’s densest library.
IBM says it has 1.8x more petabytes per square foot compared to competitor tape storage and is specifically targeted at Google, Microsoft, AWS and Meta (Facebook). We’re told they “heavily rely on this technology to keep their customers’ information safe.” IBM hopes it can extend the tape library’s addressable market to a new category of enterprise data aggregators – the “new wave” enterprise hyperscalers.
A Diamondback frame.
IBM’s Scott Baker, VP and CMO for IBM Infrastructure, issued a statement saying: “The IBM Diamondback Tape Library provides critical protection against a variety of threats, helping minimize datacenter floor space requirements and which can reduce organizations’ carbon footprint. It’s part of an end-to-end data protection and security solution that IBM can deliver.”
It’s got all the traditional tape storage benefits of physical air-gapping, long-term storage of up to 30 years, lower carbon footprint than disk storage – we’re told high-density hard disk drives (HDD) consume 20 times more energy per year than the Diamondback tape storage library. It also has good streaming IO performance, multiple drives and robotic tape cartridge handling.
IBM says it’s designed it to provide industry’s most dense tape storage. It comes as a frame (tall rack) the same floor width as an OCP rack, with a 5U top rack option, and is designed to be used as a ultra-high-density stand-alone library or as part of a Redundant Array of independent Libraries (RAIL). You add another and independent frame to get more capacity. There are 1,584 cartridge slots in total, with up to 36 slots for cleaner and diagnostic cartridges, and swap space, enabling 1,548 active tape slots.
Diamondback supports the LTO format; LTO-9 tape drives with LTO-8 and LTO-9 tape media. Data and WORM formats are supported as is mixed media. There are plans to integrate all IBM tape formats in the future.
The maximum capacity per frame is 27.8PB uncompressed, 69.66PB compressed in 7.6 square feet of floor space. With up to 14 tape drives there is a 20.16TB per hour I/O rate.
The library can be shipped with media pre-installed and there is a 30-minute installation time from time on floor to power-up and auto-configuration. The product has a self-service design with <2 minute average CRU (Customer Replaceable Unit) parts replacement, finger-only screws and QR code guidance to assist service personnel.
IBM’s Spectrum Archive software product supports Diamondback and the company says the system is a quarter of the total cost of a public cloud archival service for equivalent capacity.
Competition beat
Quantum has its i6H Scalar library product for webscale enterprises and hyperscalers, and it has a Redundant Array of Independent Libraries (RAIL) architecture like Diamondback. Quantum has not publicly revealed i6H tape slot and drive counts. But the library is delivered as a fully assembled 48U rack which contains eight component chassis looking like Scalar i6 modules.
We know a 6RU i6 module supports up to three full-height LTO tape drives and 100 storage slots, for up to 4.5PB (compressed) of storage per module. A rack of eight i6 chassis would then total 24 tape drives and 800 cartridge slots, and store 14.4PB uncompressed and 36PB of compressed data using LTO-9 tapes.
IBM’s Diamondback stores almost twice as much data as a fully populated rack of Scalar i6 chassis.
Spectra Logic has its Tfinity library for hyperscalers and the like. It scales out to 45 frames (large racks) with 55,990 cartridges, and supports the LTO-9 format, amongst others. That means 2.5EB of compressed data. Divided by 45 we arrive at 55.6PB per frame, which is less than Diamondback’s 69.66PB compressed data per frame.
The Diamond back tape library is generally available now. Check out IBM’s Diamondback web pages for more information.
