Storage news roundup – 30 March 2023

Avid‘s NEXIS | F2 Solid State Drive (SSD) flash storage engine is available now. It accelerates media workflows, including finishing of 4K, 8K and HDR content, color grading, VFX and animation. Capacity scales from 38.4TB to 307.2TB per engine, which is >6GBps with media packs. Media protection and high availability are achieved with a redundant storage controller and hot spare SSDs. Dual redundant 100 Gbps Ethernet connections per storage controller are standard. The NEXIS | F2 SSD is compatible with all current Avid NEXIS systems. When used with NEXIS online or nearline storage, F2 SSD administrators can seamlessly move a workspace between performance tiers, maintaining read and write access while the media is moving. 

William Blair analyst Jason Nader, who has written at length about Confluent and Apache Kafka in prior reports (our initiation and our deep-dive into Confluent Cloud), has concluded that the need for modern streaming software is accelerating as organizations look to use real-time data to build smarter applications and glean insights. Support for real-time capabilities is increasingly becoming table stakes across many industries. Through its support for the open-source software project Apache Kafka, Confluent has become the category leader in the data streaming ecosystem, having built an at-scale business for helping organizations harness their real-time data streams.

Lakehouse supplier Databricks announced the opening of offices in Tel Aviv and Zurich, in addition to expanding its footprint in  London and opening a new office in Munich. It also announced the coming availability of Databricks infrastructure in the AWS France (Paris) Region in the first half of 2023. The AWS-based Databricks infrastructure in France will support the growing customer base and demand for the Databricks Lakehouse.

The DNA Data Storage Alliance, a SNIA Technology Affiliate, formed in 2020 by Illumina, Microsoft, Twist Bioscience, and Western Digital,  announced the appointments of CATALOG Technologies and Quantum Corporation to its governing board. David Turek will represent CATALOG and Don Doerner and Turguy Goker will represent Quantum. The Alliance says it’s achieving its mission by: 1) educating the storage ecosystem and the public on this emerging technology; 2) identifying key technical challenges in the underlying technologies in order to drive funding and research which facilitate commercialization; and 3) developing standards and specifications (e.g. encoding, physical interfaces, retention, file systems) that enable the emergence of an interoperable DNA data storage product ecosystem.

An EU-funded 10-party consortium from Spain, France, Italy, Finland, Israel and Switzerland is working on a 3-year EXTRACT project to provide a distributed data-mining software platform for extreme data across the compute continuum (edge, cloud and high-performance computing (HPC) environments). Deriving value from raw data requires the ability to extract relevant and secure knowledge. Current practices and technologies are only able to cope with some data characteristics independently and uniformly. The aim of EXTRACT is to create a complete edge-cloud-HPC continuum by integrating multiple computing technologies into a unified secure compute-continuum. It will do so by considering the entire data lifecycle, including the collection of data across sources, the mining of accurate and useful knowledge and its consumption. It will be validated in two real-world use-cases:

  • A Personalized Evacuation Routing (PER) System will serve to guide citizens in an urban environment (the city of Venice) through a safe route in real time. The EXTRACT platform will be used to develop, deploy and execute a data-mining workflow to generate personalized evacuation routes for each citizen, displayed in a mobile phone app, by processing and analysing extreme data composed of Copernicus and Galileo satellite data, IoT sensors installed across the city, 5G mobile signal, and a semantic data lake fusing all this information.
  • The Transient Astrophysics with a Square Kilometer Array Pathfinder (TASKA) case will use EXTRACT technology to develop data mining workflows that effectively reduce the huge amount of raw data produced by NenuFAR radio-telescopes by a factor of 100. This will allow the populating of high-quality datasets that will be openly accessible to the astronomy community (through the EOSC portal) to be leveraged for multiple research activities.

IBM says Storage Fusion will provide application resilience on Red Hat OpenShift with Fusion 2.5, which has a GA date of 30th March 2023. The enhancements are:

  • Fusion Software gets Data Foundation Advanced features, Metro DR (with Data Foundation + Red Hat ACM) and Fusion on zCX (with Data Foundation)
  • Fusion HCI System gets Rack-less HCI System (bring your own rack), 3-zone HA cluster, and run multiple OpenShift clusters on HCI System with IBM Cloud Satellite. 
  • Fusion Software and HCI System get backup to IBM Storage Protect (SP), offload backup from SP to tape or disk. They integrate Discover as a Fusion Cataloging Service, and improve the usability and resiliency of the Fusion Backup/Restore service. 

More details here.

MariaDB has a new version of its managed SkySQL database with autoscaling which scales resources up when demand surges and back down when demand normalizes to save costs. It has also introduced serverless analytics to uncover insights on current data without the need for ETL and while paying for only what is used. SkySQL enables autoscaling of both compute and storage in response to changes in demand. Rules specify when autoscaling is triggered, for example when CPU utilization is above 75 percent over all replicas sustained for 30 minutes, then a new replica or node will be added to handle the increase. Similarly, when CPU utilization is less than 50 precent over all replicas for an hour, nodes are downgraded. Users always specify the top and bottom threshold. It works with Xpand, MariaDB’s distributed SQL database, adding and releasing nodes as needed.

Language searcher and vector database builder Nuclia is adding generative AI to its product. It says that with Nuclia generative answers you will be able not only to get semantic results but also answers based on company documents. Imagine a ChatGPT-like functionality for your internal information. Users will be able to query Nuclia in (almost) any language and get the answer in the same language they queried. They’ll be able to generate reports based on internal documents. This tutorial tells you more.

TerraMaster D6-320.

TerraMaster has released a new D6-320 6-bay external hard disk enclosure with USB 3.2 Gen 2 protocol, 10Gbps data transmission bandwidth, and up to 132TB (22TB x 6 drives) capacity. The Read/write speed is up to 1,030 MBps with 6 x WD Red 8TB drives. With a single SSD (WD Red 1TB) read speed could reach 510 MBps. D6-320 is equipped with a USB Type-C interface and a USB type C cable (1 meter in length), and could be compatible with a variety of computer interfaces: USB 3.0/USB 3.1/USB 3.2/USB4 /Thunderbolt 3/Thunderbolt 4 (to connect it to the USB type-C interface on your computer, you need to purchase another USB type C-type C cable).

Cloud storage provider Wasabi is partnering IBM with its Cloud Satellite to allow enterprises to run applications across any environment – on-premises, in the cloud or at the edge – and enable users to cost-efficiently access business data and analytics in real time. Cloud Satellite provides a distributed cloud architecture that brings the scalability and flexibility of public cloud services to the applications and data that run in a user’s private cloud. Boston Red Sox baseball team is a user. The Red Sox plan to use Wasabi cloud storage across its hybrid cloud infrastructure while piloting IBM’s Cloud Satellite to house data including player video, analytics, surveillance data, IoT, and more, across the Fenway Park stadium.