Ceph news: IBM Storage Ceph has introduced several new features to Ceph’s orchestrator that simplify the deployment of IBM Storage Ceph Object and associated services. Enhancing the Ceph Object Gateway and Ingress service specification files enables an out-of-the-box, production-ready RGW setup with just a few configuration steps. These enhancements include automated SSL/TLS configurations, virtual host bucket access support, erasure coding for cost-effective data storage, and more. More details in an IBM Storage Ceph blog.
…
According to a BlueSky post regarding the DAOS User Group, the DAOS (Distributed Application Object Storage) development team has transferred from Intel to HPE. The code base is still managed by the DAOS Foundation. A comment was that: “After what happened to Lustre, HPE ensuring DAOS’s continuing survival is to be lauded.”
…
DDN and VAST Data could be salivating at the news that xAI boss Elon Musk has told the Memphis Chamber of Commerce that he wants to expand his Colossus supercomputer complex to 1 million GPUs. Colossus powers xAI’s Grok generative AI system and could provide AI facilities to other Musk businesses, such as Tesla and its vehicle self-driving efforts. Both DDN and VAST supply storage for Colossus. Musk has been known to over-promise and under-deliver – but then he can over-deliver as well.
…
Search company Elastic announced GA of Elastic Serverless, powered by a re-architected Elasticsearch built on an “industry-first Search AI Lake optimized for real-time applications. It combines vast storage with low-latency querying and all of the strengths of Elasticsearch’s AI and search capabilities.” It “is engineered to tackle high-volume, complex, and high-performance workloads – from retrieval-augmented generation (RAG) to threat detection. By decoupling compute from storage, indexing from search and using cost-effective cloud-native object storage, the architecture scales seamlessly while retaining the rapid, low-latency querying performance and AI relevance capabilities of Elasticsearch.”
…
Colo and datacenter/digital infrastructure supplier Equinix has promoted 14-year company veteran Bruce Owen to be its permanent EMEA President from being its interim one.
…
Futurum’s Stephen Foskett, who runs the Gestalt IT podcast, put out a panel session and blog titled “There’s a Gulf Between Storage and AI.” He reckons: “There is a significant gap between storage companies and their ability to effectively support AI infrastructure … The discussion highlights a significant gap between storage companies’ current capabilities and the demands of AI applications.”
Tell that to VAST Data, the Nvidia GPUDirect suppliers, the vector database companies, Dell, DDN, Datadobi, Komprise, Hammerspace, Arcitecta, Couchbase (below), and the many other storage suppliers who are all diving into the generative AI and machine learning pool.
There were three panel members: Kurtis Kemple, Brian Booden, and Rohan Puri. Kemple is a developer relations director at Slack. Booden calls himself a “Trusted Analytics Advisor & Media Partner” and runs the DataGlow business intelligence consultancy in Edinburgh, UK. Puri is a staff engineer at Samsung Semiconductor, eight months in from being a graduate teaching assistant and software intern. We think the gulf they identified is, in actuality, pretty shallow.
…
The US has made it difficult for more than 140 Chinese organizations to obtain high-bandwidth memory (HBM) using US IP. This affects HBM sales made by Micron, Samsung, and SK hynix. The Chinese organizations also face restrictions in getting accelerators (GPUs) and advanced semiconductor chip fabrication tools. China is encouraging its companies to become self-sufficient in HBM, accelerators, and semiconductor fabs.
…
German hosting and cloud provider Hetzner is expanding its portfolio with S3-compatible Object Storage. Hetzner’s Object Storage has a monthly base price of $5.99/€4.99, which includes 1 TB of storage and 1 TB of egress traffic. Storage space and traffic in excess of the quota is charged on a pay-as-you-go basis at $0.0080/€0.0067 per TB-hour for storage space and $1.20/€ 1.00 per TB for outgoing traffic. Data can be stored in compliance with the EU’s General Data Protection Regulation (GDPR) in Hetzner’s datacenters in Falkenstein, Nuremberg, or Helsinki.
…
HPE has launched its MSA Gen7 Storage System targeted at small and medium-sized businesses and offices, as opposed to HPE Alletra Storage MP, which targets medium-to-large enterprise customers. HPE says its MSA Storage has been the trusted choice for small to medium-sized businesses and remote office/branch office environments. Highlights include:
- Enhanced Performance: Up to 2x faster random read speeds (up to 783,000 IOPS) with throughput reaching 14.1 GBps, and scalability up to 7.37 PB to support growing business needs.
- Streamlined Management: Features including Smart Assist, online firmware updates, and the HPE MSA Health Check tool, reducing downtime and improving system reliability.
Get more information in an HPE article.
…
Self-hosted cloud SaaS backup supplier Keepit has been awarded “Best Green Initiative” by the Business Awards UK, 2024 Corporate Sustainability Awards. Keepit’s datacenters in the Americas and EMEA have been powered by 100 percent renewable energy since 2023.
…
Marvell has a Bravera SC5 SSD controller “supporting up to 16 NAND Channels for enabling next generation cloud storage solutions.” The MV-SS1331 and MV-SS1333 products support 8 or 16 high-performance NAND channels respectively. They are NVMe 1.4b compliant and feature end-to-end data protection with support for up to four PCIe lanes, or a dual port up to two PCIe lanes per port.
…
NetApp’s Spot Eco got a big nod from research house GigaOm. A study, commissioned by NetApp, revealed the cloud savings that can be achieved by shifting resources from on-demand to reserved instances and savings plans by Spot Eco. It covered eight enterprise companies from seven industries, both single-cloud and multi-cloud, with annual cloud spend ranging from $1 million to $70 million. GigaOm found that these eight companies collectively saved $201,700,000 in annual cloud costs. A table showed that a similar saving was achieved in single-cloud and multi-cloud enterprises:
…
Quantum has launched the Scalar i7 RAPTOR tape library system, originally announced in March. It has 2,008 usable data slots in a single tape library cabinet or frame that delivers over 36 PB of storage using LTO-9 (native 18 TB/cartridge). All components are hardened and customer serviceable and replaceable in less than five minutes. It has the highest 80 PLUS Platinum power supply efficiency rating. Quantum says ActiveScale and the Scalar i7 RAPTOR together deliver cost-effective, S3 Glacier-compatible cold storage services. By deploying multiple Scalar i7 RAPTORs in a Redundant Array of Independent Libraries (RAIL) architecture, ActiveScale manages and optimizes data placement across the libraries for industry leading availability, performance, and storage efficiency.
The Scalar i7 RAPTOR has flexible growth with capacity-on-demand starting at 100 slots going up in 100-slot increments. There are up to 20 LTO full-height tape drives, more than any other scale-out library. It integrates with StorNext data services such as FlexTier to deliver full data lifecycle management from data creation to long-term retention and recall. As data moves across storage media, from Non-Volatile Memory Express (NVMe) or solid-state drive (SSD) to hard disk drive (HDD), tape, or cloud, StorNext provides continuous access. Data requiring preservation over time can be copied or moved manually or by policy to the Scalar i7 RAPTOR for the lowest cost long-term retention.
The Scalar i7 RAPTOR is now available for general distribution, with initial units already in customer testing and certification. For more information about the Scalar i7 RAPTOR and Quantum’s infrastructure offerings for AI, click here.
…
The Korea Times reports that SK hynix plans to increase dividend payouts by 25 percent per share by 2027 as part of a broader strategy to boost corporate value amid the chipmaker’s rapid growth in the AI chip market. The report mentions: “SK hynix has been rapidly expanding its presence in high-bandwidth memory (HBM), a key component for AI accelerators. Its latest HBM3E products are currently supplied to US AI chip giant Nvidia, solidifying its leadership position in the industry over its rival Samsung Electronics.”
…
Bloomberg has an article profiling SK Group Chairman Chey Tae-won, headlined “SK Group’s Billionaire Scion Bet on a Failing Chipmaker and Won Big.” That failing chipmaker became SK hynix.
…
New Zealand’s National Institute of Water and Atmospheric Research (NIWA) has chosen to store its data on VAST Data storage in primary and secondary datacenters, helping it prepare for GPU-based scientific workloads. NIWA has over 20 petabytes of historical weather data that can be used by AI and machine learning. It says VAST Data Platform’s active-active clustering capabilities and global namespace allow synchronization and utilization of data across multiple sites. The deployment process is under way, with full production readiness anticipated by May 2025.
…
Datacenter virtualizer VergeIO launched ioReplicate, a DR facility delivering “infrastructure-wide recovery that ensures organizations can recover from disasters quickly and confidently – in just three mouse clicks.” It encapsulates entire workloads, including virtual machine configurations and networking settings, into portable Virtual Data Centers (VDCs). These VDCs can be replicated across VergeOS instances or to third-party providers, enabling recovery in any environment. It uses global deduplication and ioClone snapshots to ensure WAN-optimized replication with minimal bandwidth usage. Replication frequency is configurable, supporting near real-time updates to meet strict recovery point objectives (RPOs).
It includes VergeFabric, which virtualizes networking capabilities such as firewalls, routing, and micro-segmentation. Its dynamic routing protocol, Border Gateway Protocol (BGP), ensures seamless traffic redirection to the DR site during outages, without requiring third-party failover tools.
…
Cloud object storage supplier Wasabi has joined SINET (Science Information Network), an information and telecommunications network built and operated by the National Institute of Informatics (NII) as an academic information infrastructure for universities and research institutions throughout Japan. Currently, more than 1,000 universities and research institutions nationwide are subscribers, and more than 3 million researchers and students are using SINET. Universities and research institutes using SINET can now use Wasabi cloud storage over SINET’s high-speed and secure closed network. Wasabi supports data strategies of universities and research institutions globally, including the GakuNin Cloud Deployment Support Service, and academic information networks such as AARNet in Australia and JANET in the UK.
AWS re:Invent
AWS made plenty of announcements at its 2024 re:Invent conference event in Las Vegas. The storage-related ones were …
AWS Clean Rooms – AWS Clean Rooms announced support for collaboration with datasets from multiple clouds and data sources. This launch allows companies and their partners to easily collaborate with data stored in Snowflake and Amazon Athena, without having to move or share their underlying data among collaborators. With this, customers can collaborate with any company leveraging datasets across AWS and Snowflake, without any party having to move, reveal, or copy their underlying datasets.
…
Amazon Elastic VMware Service – AWS announced the preview of Amazon Elastic VMware Service (Amazon EVS). Amazon EVS is a new, native AWS service to run VMware Cloud Foundation (VCF) within your Amazon Virtual Private Cloud (Amazon VPC).
…
Data Transfer Terminal locations in Los Angeles and New York, where customers can bring in storage devices and get the contents uploaded to AWS.
…
EC2 I8g instances – AWS is announcing the general availability of EC2 storage optimized I8g instances, which offer the best performance in EC2 for storage-intensive workloads. They are powered by AWS Graviton4 processors that deliver up to 60 percent better compute performance compared to previous-gen I4g instances. They use third-gen AWS Nitro SSDs, local NVMe storage that delivers up to 65 percent better real-time storage performance/TB, while offering up to 50 percent lower storage I/O latency and up to 60 percent lower storage I/O latency variability. I8g instances offer instance sizes up to 24xlarge, 768 GB of memory, and 22.5 TB instance storage. They are for real-time applications like relational databases, non-relational databases, streaming databases, search queries, and data analytics. I8g instances are available in the US East (N. Virginia) and US West (Oregon) AWS regions.
…
External AWS Outposts storage – Customers can attach block data volumes backed by NetApp ONTAP and Pure Storage FlashArray on-premises storage arrays to EC2 instances on AWS Outposts directly from the AWS Management Console. This enhancement is available on Outposts racks and Outposts 2U servers at no additional charge in all AWS Regions where Outposts is available, except the AWS GovCloud regions.
…
FSx Intelligent Tiering – AWS announced GA of FSx Intelligent-Tiering, a new storage class for Amazon FSx that costs up to 85 percent less than the FSx SSD storage class and up to 20 percent less than traditional HDD-based NAS storage on premises. This brings full elasticity and intelligent tiering to NAS. The storage class is available on Amazon FSx for OpenZFS. With FSx Intelligent-Tiering, customers can bring to FSx for OpenZFS a broad range of general-purpose datasets, including those with a large proportion of infrequently accessed data stored on low-cost HDDs on-premises.
…
Nvidia NIM microservices are now available on AWS, supporting optimized inference and lower latency for generative AI application deployments. Developers can access Nvidia NIM microservices on Amazon Bedrock Marketplace, SageMaker JumpStart, and AWS Marketplace.
…
S3 default data integrity protections – S3 updates the default behavior of object upload requests with new data integrity protections that build upon S3’s existing durability posture. The latest AWS SDKs now automatically calculate CRC-based checksums for uploads as data is transmitted over the network. S3 independently verifies these checksums and accepts objects after confirming that data integrity was maintained in transit over the public internet. Additionally, S3 now stores a CRC-based whole-object checksum in object metadata, even for multipart uploads, which helps you to verify the integrity of an object stored in S3 at any time. Default checksum protections are rolling out across all AWS Regions in the next few weeks. To get started, users can use the AWS Management Console or the latest AWS SDKs to upload objects.
…
S3 Storage Browser – AWS announced GA of Storage Browser for S3, an open source component that can be added to web applications to provide authorized end users, such as customers, partners, and employees, with access to easily browse, download, and upload data in S3 directly from their own applications. Storage Browser for S3 is available in the AWS Amplify React and JavaScript client libraries. End users can now search for their data based on file name and can copy and delete data they have access to. Storage Browser for S3 automatically calculates checksums of the data end users upload and blocks requests that do not pass these durability checks.
…
Will Amazon add Nvidia’s GPUDirect for object storage – S3 over RDMA? It would be madness not to.
…
Storage-optimized I7ie instances – AWS announced GA for high-density Storage Optimized I7ie instances designed for large I/O intensive workloads. They are powered by fifth-gen Xeon Scalable processors with an all-core turbo frequency of 3.2 GHz, offering up to 40 percent better compute performance and 20 percent better price performance over existing I3en instances. I7ie instances have the highest local NVMe storage density in the cloud for storage optimized instances and offer up to twice as many vCPUs and memory compared to prior-gen instances. Powered by third-gen AWS Nitro SSDs, I7ie instances deliver up to 65 percent better real-time storage performance, up to 50 percent lower storage I/O latency, and 65 percent lower storage I/O latency variability compared to I3en instances.
I7ie instances are for workloads requiring fast local storage with high random read/write performance at very low latency consistency to access large datasets. A 16 KB torn write prevention feature enables customers to eliminate performance bottlenecks. They deliver up to 100 Gb of network bandwidth and 60 Gb of bandwidth for Amazon Elastic Block Store (EBS). They are available in the US East (N. Virginia, Ohio), US West (Oregon), Europe (Frankfurt, London), and Asia Pacific (Tokyo) AWS regions. Customers can use these instances with On Demand and Savings Plan purchase options.
…
Cohesity is a recipient of a 2024 Geography and Global AWS Partner Award and named as Global Storage Partner of the Year.
…
Data protector Druva announced new Threat Hunting capabilities for Amazon EC2 and Amazon RDS Air-Gapped Backups, enhancing data protection and security for AWS workloads. These complement Druva’s existing coverage for AWS workloads, including EC2, EBS, Redshift, DocumentDB, and Neptune, and enable businesses built on AWS to keep their backup data effectively isolated from their primary environments – protecting them from credential breaches and evolving cyber threats. (Druva has expanded its AWS workload offerings with enhanced support for Amazon RDS. With this, customers can air-gap their data and completely isolate backups from their AWS environment.) More information here.
…
Rubrik introduced its upcoming Cloud Vault for AWS. Back in 2022, Cloud Vault was Rubrik’s SaaS archival service built on Azure, maintaining immutable and instantly recoverable copies of data in a secured, isolated cloud location, using Azure Blob storage, fully managed by Rubrik. Cloud Vault for AWS will offer an isolated, virtually air-gapped environment, be fully managed, clean recovery point identification, flexible storage tiers, single bill for all expenses, and be available through the AWS Marketplace.
…
Data warehouser Teradata is strengthening its relationship with AWS to deliver “rapid-start” GenAI use cases when leveraging Teradata VantageCloud on AWS, integrated with Amazon Bedrock. A fully managed AWS service, Amazon Bedrock allows enterprises to quickly adopt the latest GenAI innovations with easy access to their choice of high-performing foundational models from leading AI companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon. When combining VantageCloud’s open and connected framework with Amazon Bedrock, enterprise customers can rapidly leverage the foundational models of their choice with data in VantageCloud.
…
DBaaS supplier Tessell showed its RDS+++ Solution, enhanced with AWS FSx for NetApp ONTAP integration. With support for Azure NetApp Files already in place, Tessell now enables businesses to deploy database services on ONTAP solutions across both AWS and Azure. Tessell’s managed DBaaS offers high availability with multi-AZ replication, application-consistent backup and recovery, advanced cloning capabilities, and comprehensive monitoring for optimized database operations.
