It’s all about speed this week; Amazon is announcing new stuff before its December Re: Invent conference. DDN has spruced up its arrays and Intel is launching a data moving semiconductor block to offload server CPUs.
There is more, including a tape industry tidbit and 128-layer NAND. Read on.
AWS Data Exchange
This service brings data from validated sources to Amazon customers. They can find, subscribe to, and use third-party data in AWS in a secure way. Qualified data providers include Reuters, Change Healthcare, Dun & Bradstreet and Foursquare.
AWS Data Exchange removes the need to build and maintain infrastructure for data storage, delivery, billing, and entitling. AWS Data Exchange becomes their data delivery system to enterprises, scientific researchers, and academic institutions.
Currently they may receive the data on shipped physical devices (tape, disks) or use old style FTP transfer methods from several hosts, or use specific APIs. Amazon says it can sweep all this away and replace it with faster, smoother data delivery through its cloud.
Data providers can, Amazon says, reduce or do away with sales and marketing effort, as well as technology to store, deliver, bill for, and entitle data for their customers.
Investigate AWS Data Exchange here.
DDN beefs up storage arrays
DDN touted new generation SFA arrays with greater performance At SC19 in Denver this week. The high performance computing storage lineup includes SFA18KX and SFA7990X hybrid systems and SFA400NVX and SFA200NX all-flash arrays.
DDN doesn’t say what the performance features will be. We understand that NVMe SSDs are coming to the hybrid systems, which currently support SAS disks and SSDs only. They may also get multi-actuator disk drive support, with two read/write heads per disk platter instead of one. This will increase their bandwidth.
We wonder if PCIe Gen 4, with twice PCIe Gen 3 speed, will be adopted as well. That would boost bandwidth nicely.
DDN is preping new data management features for its EXA5 Lustre array. It is also getting an S3 object storage interface that can be accessed in parallel with its existing NFS/SMB file interfaces. That means these DDN file storage arrays will be able to act as object storage arrays to a S3 API-using application.
The company’s WOS object storage system supports S3, Swift, NFS, SMB, Spectrum Scale and Lustre.
Data Streaming Accelerator
Intel has launched a Data Streaming Accelerator (DSA) to speed data movement (copy and transformation) in coming Intel CPUs and applying to/from volatile memory, Optane memory-mapped I/O, and through a Non-Transparent Bridge (NTB) device to/from remote volatile and persistent memory on another node in a cluster.
This replaces Intel’s 2006 era QuickData Technology and offloads data moving functions from a server’s main CPU.
The DSA has a PCIe-compatible OS programming interface and is controlled with a device driver.
Think of it as a semiconductor block added to a System-on-Chip (SoC). It supports shared virtual memory operation and Intel Scalable I/O Virtualization.
Download the architecture specification here.
Short items
Aerospike, a real-time NoSQL data platform supplier has announced a $32m Series D fundraising round.
Data analytics company Databricks and automated data governance supplier Immuta have integrated their products to provide analytics on sensitive data in the cloud. They say this is policy-driven and automated, with data made secure, anonymised and compliant for cloud analytics and machine learning.
Databricks also announced API integration with AWS Data Exchange, a new service for Amazon Web Services customers to securely find, subscribe to, and use third-party data in the cloud.
The UK’s Science and Technology Facilities Council (STFC) has deployed a high performance computing architecture to support computationally intensive analysis including machine learning and AI‐based workloads using Excelero’s NVMesh elastic NVMe block storage product.
GigaOm has published Data Storage Architectures for Machine Learning and Artificial Intelligence.
HPE announced the HPE Container Platform, claimed to be the industry’s first enterprise-grade Kubernetes-based container platform for cloud-native applications and monolithic applications with persistent storage. It uses BlueData and MapR technology.
HPE has added Cray products to its own set, such as positioning Cray supercomputers above the Apollo line. Cray’s Slingshot interconnect is on the combined product list, as is ClusterStor E1000 storage array for HPC workloads, differentiating it from HPE’s existing Nimble, 3PAR and OEMed Hitachi VSP 5000 arrays.
Email and data security vendor Mimecast has acquired DMARC Analyzer, a SaaS-based supplier offering domain-based message authentication, reporting and conformance setup, management and analysis.
The UK’s Object Matrix, with object storage, and Germany’s Arvato Systems, with media asset management, have teamed up to enable media companies to instantly access, manage, browse, and edit clips from their digital archives.
SK hynix is sampling 128-layer 3D NAND – the first public appearance of a 3D NAND chip with more than 100 layers. This is the company’s sixth 3D NAND generation and the sample chips have a 1Tbit capacity. They are built into 1TB UFS and 2TB M.2 sample drives with TLC (3bits/cell) flash. There is also a 16TB EDSFF E1.L ruler format drive sample.
(Clockwise from top left) 16TB E1.L eSSD, 2TB cSSD, 1TB UFS 3.1
The UK Science and Technology Facilities Council has deployed a Spectra TFinity ExaScale Tape Library in its Scientific Data Centre facility at the Rutherford Appleton Laboratory in Oxfordshire. Initial capacity for the scalable system is 65PB. It has 48 drive bays and a mixture of 17 open-standard LTO-8 tape drives and 16 IBM TS1160 tape drives.
The UK’s StorMagic announced it has validated its virtual SAN SAN product with HPE DL325 and DL360 ProLiant Gen10 rack servers running both VMware vSphere and Microsoft Hyper-V hypervisors. The company has joined HPE’s partner program.
The Tape Storage Council has released its State of the Tape Industry report, which highlights the current trends, use cases, and technology innovations occurring within the tape storage industry. We found it bitty and uncompelling. Get it here.
