Home Blog Page 159

Tiger Technology enters digital pathology cloud file services market

Cloud file services vendors – CTERA, Egnyte, Nasuni, Panzura – are meeting new competition in the specialized image digital pathology area in the form of Bulgaria’s Tiger Technology.

Digital pathology refers to images of pathology specimen glass slides. The slides contain small slices of tissue, which may have been stained to highlight specific features. A doctor or other medical professional can inspect them to check for signs of disease or deterioration. Such images are taken by special digital scanners and need to be zoomable to fine detail, as if the viewer was looking through an actual microscope.

Alexander Lefterov

Alexander Lefterov, Tiger Tech’s founder and CEO, told B&F: “We are working with different scanning machine vendors and their workflows [for example Phillips]. A module in its scanner talks to our API to access cloud data. We’re middleware vendors and integrate our software with these front-end devices.”

Tiger Technology provides a hybrid multi-cloud file namespace for Windows Servers and enables space-saving file tiering from on-premises servers to cheaper file and object stores with ancillary backup, archive, file sync, business continuity, and disaster recovery benefits. It has several products: Tiger Store on-premises file sharing; Tiger Pool combines multiple volumes into a single pool; Tiger Spaces file sharing among work group members; and the Tiger Bridge cloud storage gateway, syncing and tiering product.

In the digital pathology field, the Tiger Bridge software is integrated with the scanner and enables captured images to be sent to a public cloud repository, from where they can be accessed anywhere in the world. That means remote consultants can check images and help a local medical team come to a patient treatment decision.

There is another reason the images are sent to the cloud. Actual slides containing bio-samples may need to be kept for 10 years or longer and that means they have to be refrigerated to prevent decay and degradation. Refrigeration costs money. A digital scan can be archived in the public cloud more cheaply than such refrigeration. 

Each scanning manufacturer, such as Philips, OptraSCAN, Epredia and Leica Aperio, has unique technology and therefore unique software integration is needed. The API integration hooks the scanner up to an on-premises file server with Tiger’s software installed. The Tiger Bridge software shoots the images up to the cloud and is also used to retrieve them for later access.

The images are huge, potentially 1 million by 1 million pixels, approximately 2GB per slide, and up to 10 exabytes of such images could be ingested per year. The scanning machines are expensive but even so, the digital images they generate are still less expensive to keep than sticking slides in a fridge for ten years. 

Proscia digital pathology image at 30x zoom

The image data can’t be encrypted as that prevents analysis and provides a kind of lock-in. The data must also be anonymized to preserve patient confidentiality. Lefterove says it needs keeping in a vendor-neutral format as well so that, again, customers are not locked in.

Lefterov said that digital pathology image file collaboration has three phases:

  • Solidification – sending data to the cloud for cost-optimized archiving, and continuous data protection with disaster recovery,
  • Access Extension – with multi-site synchronisation and sharing and remote workflow support,
  • Augmentation – AI and machine learning for analytics and insights to augment human analysis.

Tiger’s cloud storage costs around $5/TB/month, with Lefterov saying: “We are an enabler of cloud storage – and can’t cost more than the cloud storage provider.“

In effect Tiger has found a specialized niche cloud file services and collaboration sector within the healthcare market. Established vendors CTERA, Nasuni and Panzura each have a healthcare market presence already, as does Egnyte. Tiger says its edge is that it has the digital pathology scanner and workflow integration, and the capability of handling the metadata needed for the huge digital pathology images.  

Storage titans face Turkish competitor inspired by VAST Data’s architecture

NGX Storage, a Turkish storage outfit, is developing a scale-out NVMe block storage array with a top-level hardware architecture similar to VAST Data’s Universal Storage scale-out filer array.

Privately owned NGX, founded in 2015 and based in Ankara, builds unified block, file and object NGX-H hybrid flash+disk arrays and NGX-AFA all-flash arrays. It has more than 200 systems deployed, mostly in Europe, and technology smarts include deduplication, inline compression, zero detection, thin provisioning and Random Flash Sequential Disk. This has incoming data always written sequentially to the hard drives with random operations served from SSDs. Its systems have massive DRAM caches with almost 95 percent of data served from memory – shades of Infinidat.

Pradeep Ganesan.

NGX has engineering, research and development carried out in Bangalore and Canada (Vancouver). This is not a a startup – NGX funded the new array’s development over the past two and a half years using ongoing product sales.

NGX architect Pradeep Ganesan, who is based in Canada, presented the scale-out block array development to an IT Press Tour audience in Lisbon. A look at the architecture diagram shows an uncanny similarity to VAST Data’s disaggregated storage array design:

It uses high-availability stateless storage controller nodes accessing NVMe SSDs across an InfiniBand RDMA fabric, with each node seeing all the SSDs in the array, and with controller nodes and storage capacity separately scalable. The controller nodes handle metadata storage service with a focus on IOPS, throughput and client connections.

Bullet point attributes are:

  • Write-anywhere block storage – no block is overwritten. 
  • SSD-friendly log-structured write scheme
  • 9GB/sec controller throughput
  • User space NVMe driver – kernel bypass
  • Uses Optane Persistent Memory on the storage nodes – not for controller write buffering; now looking at other options
  • Intel’s open source Storage Performance Development Kit (SPDK) provides user space NVMe driver
  • Sub-millisecond latency with highly parallel direct access to the TLC SSDs in the storage nodes
  • I/O Path locks avoided through message passing
  • NVMe-oF client access support
  • Full block stack as a user space library – pluggable API module

Protection is by NVMe bdevs (block devices) which are added to a Replication Group entity – which is another block device.

The initial version of the product is expected to arrive in the January/February period next year. It will scale to tens of petabytes initially with hundreds of PB in the future. A software-only version will enable NGX to offer it as a public cloud resource.

This is the first storage array design we have seen that has taken VAST Data design ideas on board. As a scale-out block array it will compete with Dell EMC’s PowerStore, HPE’s Alletra 9000, IBM’s FlashSystems and Pure’s Flash Array. Unless NGX adds file access it will not be a direct competitor to VAST Data.

Bringing the best of all technology worlds to next-gen laptops

Think about your wishes as an end user of smartphones and laptops for a moment: in an increasingly remote world, what are the benefits and drawbacks of both types of device? Would you ever believe it is possible to bring the best of both worlds – (low power, long battery in smartphones) and high-performance capabilities and productivity tools (laptops) – to each?

That reality is closer than it might appear. Expert co-design efforts among device, processor, memory, and storage makers, along with powerful new technologies that boost power and performance capabilities have arrived.

All of this comes at the right time, especially following the shift to remote working which began in 2020. It’s natural that people now used to flexibility almost by default might want many features of a smartphone to carry over to a laptop. From lightweight portability to long battery life, these devices changed how we work, play, and communicate. Laptops provide better productivity tools and options, and following the height of the pandemic, users demand the best of all worlds in terms of their devices.

That impetus has significantly contributed to market growth. Research firm Gartner estimates there were 6.2bn PCs, laptops, tablets and mobile phones in use across the world in 2021, a figure predicted to increase by 3.2 percent to 6.4bn in 2022. Those figures mask an anticipated decline in the number of desktops, however. By contrast, laptop usage is expected to increase by 8.8 percent year on year, and many new devices sold will rely on innovative form factors that make them more attractive devices to own and use.

Even before the pandemic there was increasing demand in the client computing world for outstanding user experience. This includes delivering devices that could be thinner, lighter, and faster with far longer battery life all presented in a cost-effective way.

That pent-up clamor for devices that captured the power and portability of mobile devices alongside superior performance that trickled down from the data center exploded during Covid and continues today. And at the core of these innovations in client devices is memory.

From smartphones to the world’s largest data centers, memory is the foundation for performance and efficiency. In between those form factors are laptops, which can benefit from the best in data center and mobile devices, particularly with the use of low-power DRAM (LPDRAM) technology.

Energy efficient, high-performance memory is at the heart of architectural changes that have hit the broad range of technology, from smartphones and laptops, desktops to data centers. We now have changes in the types of processor cores (some for pure performance, others energy-efficient, and even accelerators for AI/ML) as well as I/O controllers and control elements all embedded into a single chipset. To make all this work seamlessly and efficiently is a memory architecture for all these elements that provides a pool of high-capacity, low-power DRAM that enables the best-in-class performance per watt customers across the client world demand.

LPDRAM drives architectural choices

That group of power efficient memory in ultra-thin and light laptops is almost always LPDRAM due to its ability to balance high performance and power efficiency in a thin form-factor. That one architectural choice changes the entire playing field for what’s possible when it comes to designing, building, and delivering lightweight, portable devices with long battery life that end users currently demand.

This will continue to evolve as manufacturers like Micron push the innovation envelope. The company recently introduced what it says is the world’s first 232-layer NAND technology. This means higher density, lower power storage which allows for the integration of more storage in an energy efficient manner, enabling more intelligence in many types of devices. These devices consume less power and can last longer between charges.

The more efficient storage along with improvements that Micron’s LPDDR5 memory delivers enable a best-in class user experience for laptop users. Consider too the performance/watt gains LPDDR5 based memory architecture can bring when integrated into CPUs using System-in Package technology, and what you have is a highly compact system that enables ultra-thin laptops. The power and space savings derived from this memory subsystem means even more room for larger batteries, which can extend device battery life for the user by as much as 12-18 hours, all without the use of power-hungry fans in a laptop for instance.

LPDRAM is the very definition of a technology enabler, especially for client devices. While ideal for smartphones, the same concept can be scaled to laptops and even large-scale data center networking as well as at the heart of on-device AI. In short, LPDDR5 memory, as delivered by Micron, can bring game-changing power efficiency and performance to the broader client world with higher capacities, performance, and ultimately, longer battery life.

“This represents a renaissance in the client market that opens a new world of architectural innovations including how the CPU and memory sub-system work together to enable lighter, thinner, faster performance with longer battery life at the right cost,” says Viral Gosalia, Director of Strategic Marketing at Micron’s Compute and Network Business Unit.

“Micron was the first to bring 1α LPDRAM to the market, and one of the first to deliver LPDDR5 mobile solutions, and recently announced its industry-leading 1β LPDDR5X mobile memory as well,” he continues. “That’s the right memory technology for client applications, which along with the architectural innovation of integrating CPU, GPU, I/O controllers and AI/ML accelerators along with LPDRAM based memory architecture, delivers best-in-class performance per watt and helps meet end-user experience expectations”.

Collaboration with x86/Arm OEM partners

That 232-layer NAND technology supports I/O speeds of 1.6GB/s now and 2.4GB/s in the future. That’s fast enough to meet the high throughput needs of even the most demanding of applications and services.

It’s also been specifically designed to deliver high performance data storage capabilities which can support a range of different workloads running on a wide range of different client devices. Everything from data center servers to mobile devices, consumer electronics and PCs, says Micron.

To that end, Micron is working closely with ecosystem partners on the x86 and Arm architecture based platforms to design memory subsystems targeted for client applications. This ranges from productivity suites, video conferencing, and even gaming and entertainment applications. In a world after the pandemic, the expectations for a single device to fulfill multiple functions is perhaps higher than it’s ever been, which makes Micron and its collaboration with OEMs and CPU players matter more than ever.

So with memory at the root of architectural innovation for lightweight, thin, power efficient devices, it’s no surprise that Micron is working closely with CPU vendors and OEMs alike as they look to design and build their next generation form factors. “There is deep collaboration with ecosystem players across the product development cycle — from product definition to validation, leveraging expertise and strategic assets including joint labs. This enables us to optimize our memory and storage solutions across platforms for targeted workloads,” says Gosalia.

Being able to deliver devices that couple the best of smartphones and laptops takes engineering expertise at the memory level. With added densities, lower power, smaller footprints, and higher performance, the rest of the system can also be optimized. This is why Micron works closely with its OEM, design, and processor partners – to deliver the next generation of what’s possible.

Sponsored by Micron.

Storage news ticker – December 15

Storage news
Storage news

Apache Cassandra v4.1 is a milestone release that’s mainly a feature update geared toward preparing for the major changes for developers in 5.0. Significant core services changes have been made to make them pluggable so different options can be used without firing up a compiler. Limits and operations boundaries can be defined in a feature called Guardrails. There are pluggable interfaces for storage formats, networking, and security. There are new native functions to convert unix time values into C* native types: toDate(bigint), toTimestamp(bigint), mintimeuuid(bigint) and maxtimeuuid(bigint) – critical for IoT-style workloads. New floor functions can also be used to group by a time range. The big, breaking changes are lining up behind 5.0, such as a new indexing engine, major changes in the CQL protocol to support new queries, and of course, ACID transactions.

 IT Infrastructure supplier DH2i said its DxEnterprise multi-platform smart high availability clustering software has earned Red Hat OpenShift certification on Red Hat Enterprise Linux.

HPE announced its GreenLake for Private Cloud Enterprise, which offers expanded container deployment options for Kubernetes with Amazon EKS Anywhere from AWS, and infrastructure-as-code and cloud-native toolchains to support customer’s DevOps and continuous integration and continuous deployment (CI/CD) environments. There are six workload-optimized instances for general purpose, compute, memory, and storage available on GreenLake for Private Cloud Enterprise with rate card pricing that delivers a pay-as-you-go consumption model. The GreenLake for PCE ecosystem now includes GreenLake for Red Hat OpenShift Container Platform – available in Q12023, and the recently announced GreenLake for VMware. Consumption analytics enhancements deliver improved usage and cost analytics of applications for AWS, Azure and GCP. GreenLake for Data Fabric and Ezmeral Unified Analytics are available through an Ezmeral early access program.

Infinidat Field CTO Ken Steinhardt is retiring from Infinidat on Dec 31.

iXsystems announced TrueNAS SCALE 22.12 “Bluefin”, the second major version of its hyperconverged open source storage OS. It provides: FIPS-140 Security Kubernetes CSI Driver; new cluster management APIs to reduce downtime when replacing a gluster node; Docker OverlayFS; Virtualization Improvements; USB device pass-through and CPU pinning; Apps GPU Acceleration and iX-Storj service; and Globally Distributed Storage with Web 3.0 technology. It also announced the clusterable TrueNAS R30 system with 16 NVMe Gen4 SSDs and two dual-port 100GbE NICs in a 1U package. Each node is capable of over 30GB/sec bandwidth and a cluster scales out to hundreds of GB/sec for clustered file and object workloads.

Kioxia CM6 and CD6 PCIe 4.0 NVMe SSDs and PM6 Series of 24GB SAS SSDs have gained Windows Server 2022 Software-Defined Data Center (SDDC) certification, when using Storage Spaces Direct. The Storage Spaces Direct feature of Windows Server creates a software-defined storage setup, combining SSDs on clustered servers and allowing the sharing of storage resources in converged and hyperconverged IT infrastructures. Users can scale out storage capacity by adding more drives or more servers in the cluster. Storage Spaces Direct automatically onboards the new drives and rebalances the storage pool. 

Lenovo announced globally it reached six percent market share and #5 in total market share for the first time, according to IDC’s December 2022 Storage Tracker. It is also #1 Entry Storage provider in EMEA based on revenue. Lenovo is at #1 in the “Price Bands 1-4” category, for the first time. Lenovo EMEA grew faster than the rest of the EMEA market – +36.8%  year-on-year in vendor revenue terms – for this time period.

We heard Luke Quick, VP OEM Sales, was laid off from composability startup Liqid along with other OEM sales team members. Sumit Puri, CEO & Cofounder, Liqid, told B&F: “The commitment to Liqid’s Dell strategy remains strong as ever. We had a minor workforce reduction earlier this week that affected a pair of OEM team members. However, we still have a strong Dell OEM team and plan to grow the greater OEM organization in 2023.”

NetApp says Amazon FSx for NetApp ONTAP now has FedRAMP Moderate authorization in US East (N. Virginia), US East (Ohio), US West (N. California), and US West (Oregon), and FedRAMP High authorization in AWS GovCloud (US) Regions. NetApp is moving for authorization of other NetApp services. NetApp Cloud Insights and CloudCheckr (part of Spot by NetApp) are currently in process for FedRAMP.

Software-defined storage supplier OpenDrives has appointed Izhar Sharon as CEO. He brings 25 years of enterprise data storage expertise leading revenue growth and company scale for companies such as Infinidat (ex-president), IBM and Dell EMC. His appointment builds the debut of OpenDrives’ next-generation enterprise IT and cloud management solutions, and also follows a series of strategic hires from Quantum, Kyndryl, Unisys and DreamWorks Studios. Ex-CEO David Buss, who led OpenDrives as CEO in achieving 92% year-over-year revenue growth during his three-year tenure, becomes a strategic advisor to the board.

Aaron Passey

Qumulo announced that one of its original founders, Aaron Passey, has returned to the company and be an integral part of the engineering leadership team. He will work closely with chief technology officer Kiran Bhageshpur. Passey left his Qumulo  CTO role in August 2016, later joining Dropbox as a principal engineer, leaving in April 2021. He returned to Qumulo this month. A Passey LinkedIn post said: “I have some unfinished business to attend to. … we aren’t the dominant player in the storage industry. We haven’t redefined the industry from the ground up. We still have some work to do. … My new role is the most senior engineer in engineering. I will help build a culture of innovation in service of the customer.”

Rubrik announced that existing board member John W. Thompson, former Microsoft chairman and Symantec CEO, is now the lead independent director of Rubrik’s board. 

Samsung Electronics and NAVER Corp are jointly developing semiconductor products tailored for hyperscale artificial intelligence (AI) models using Samsung’s computational storage (Smart SSD), processing-in-memory (PIM) and processing-near-memory (PNM), and Compute Express Link (CXL) technology. NAVER will refine its >200 billion parameter HyperCLOVA hyperscale language model and improve its compression algorithms to create a more simplified model that significantly increases computation efficiency. Samsung Electronics announced on Dec. 12 that it had built a computing system by adding its PIM chip to 96 x AMD MI100 GPUs. Business Korea reported that When Samsung  trained the T5 language model algorithm with this system, its performance improved 2.5x and its power consumption dropped by 2.67x, compared to when PIMs were not used.

Colin Presly, senior director, Office of the CTO at Seagate, has made some predictions for 2023. (1) Domain-specific silicon will increasingly be used to manipulate data movement.  (Think hardware-based compression and dedupe and AI- and ML-based data manipulation methods.) (2) Emerging interfaces will simplify internal storage systems complexity. (Think CXL, NVMe & NVMeoF). (3) HAMR technology will become more widely commercially available. (4) Cloud-native software will gain more momentum.  (5) The “Reduce, Reuse, and Recycle” mandate will become more important.

  • Reduce. At the device level, we want to reduce the data movement via techniques like compression. By reducing the data written, the total power consumed is also lowered. 
  • Reuse. At storage device level, reuse comes from using secure erase to wipe the data from the storage device before returning that drive back for resale. 
  • Recycle. Data storage companies look at ways to recycle the material from different components in the device and use back into the production cycle. 

 …

Storage Review reports that a Seagate ppresentation at an OCP summit revealed its NVMe-interface disk drives are getting close to engineering sample status.

StorPool RapidCompute graphic

StorPool Storage announced that RapidCompute, Pakistan’s first enterprise-class public cloud computing platform, has deployed its primary storage platform as part of an IaaS offering for local enterprises and small businesses. RapidCompute wanted to upgrade its infrastructure from NexentaStor and evolve its storage system to gain more flexibility and operational efficiency. Two primary storage systems are already deployed. One is a 9-node deployment connected over iSCSI to a CloudStack environment managing XenServer hypervisors. The other is a 5-node deployment connected to OpenStack with KVM. The full case study can be found here.

Toshiba has announced 1, 2 and 4TB versions of its Canvio Basics hard disk drives. These are in 2.5-inch format and for external use hooked up to PCs and notebook. Their USB 3.2 Gen 1 interfaces, which deliver power, allow transfer speeds of up to 5.0Gbit/sec to be supported and they are compatible with legacy USB 2.0. Their dimensions are 78mm x 109mm x 14mm for the 2TB and 1TB versions, weighing <150g, and 78mm x 109mm x 19.5mm dimensions for the 4TB version, which weighs 217.5g. The drives come with a matt black finish, are ready to use with Windows, and will be available in 1Q 2023.

Global file and object access supplier Vcinity announced GA of its Vcinity Access object offering and version R3.2.3 of its Ultimate X (ULT X) file service system. Vcinity Access allows cloud native applications that use S3 from AWS or S3-compliant object storage to remotely access and instantly operate on that data across hybrid or distributed cloud environments, with local-like performance. Vcinity Access features include 3-node HA VM scale-out cluster, offering scalable performance for hybrid on-prem (VMware ESXi) and AWS environments, deployment automation and integration into common cloud and on-prem automation platforms, plus diagnostic capabilities using real-time monitoring virtual sensors and visualization tools.

ULT X r3.2.3 includes hardware clustering support, Amazon EC2 F1 instance support, automation and API support, and file system enhancements.

Active data replication supplier WANdisco has signed its second follow-on agreement worth a further $13.2 million with a large European automotive components supplier. The combined Commit-to-Consume contract value from the three agreements with this Client in 2022 now totals $25.3 million. Terms were originally signed with this Client to replicate automobile sensor data to the AWS cloud. The initial data amount that was to be transferred has again exceeded initial requirements, driven by a significant customer win by the Client. WANdisco says it can execute enterprise-scale migrations extremely quickly –  1 tebibyte every 1 minute and 27 seconds (12,638MB/sec) – and with zero downtime, disruption, or data loss.

VMware’s Cloud Flex Storage in AWS squares up to FSx for ONTAP

VMware has created a scalable cloud file service for its VMware Cloud on AWS, taking on NetApp’s FSx for ONTAP in provisioning storage for its VMs and Tanzu containers.

The Cloud Flex Storage service is based on VMware’s July 2020 acquisition of Datrium and its disaggregated hyperconverged storage software tech. The news was announced in a VMware blog by Oliver Romero, the senior product marketing manager for the software.

He writes: “Cloud Flex Storage is built on a mature, enterprise-class file system that has been developed and production-hardened over many years, dating back to Datrium’s D­HCI storage product.”

This file system, already used in VMware’s Cloud Disaster Recovery service, has a two-tier design that allows for independent scaling of storage performance and capacity.

The Cloud Flex Storage service is delivered as a service, natively integrated into the VMware Cloud. NFS data store capacity is provisioned with clicks from the VMware Cloud Services Console.

Users can provision up to ~400TiB (439.805 TB) of storage capacity with ~300K IOPS per file system. Cloud Flex Storage “provides always-on data services such as at-rest encryption, deduplication, compression, and data integrity checks.”

VMware says users get a storage SLA of 99.9 percent availability within an AWS availability zone and regional S3 replication for data durability.

It has, Romero says, a predictable pay-as-you-go consumption model or a term subscription billed at a predictable $/GiB rate.

Sazzala Reddy.

A blog by Sazzala Reddy, VMware’s Cloud Storage CTO and Datrium co-founder and ex-CTO, provides more information about Cloud Flex Storage’s internal design. Reddy says it has a Log Structured File System (LFS) design with app-centric snapshots, instant cloning, and immutability, making it a multipurpose file system.

For example, “OLTP databases need very high random IO performance, OLAP databases need high sequential IO performance, and file shares have a mix of 20%/80% hot/cold data.” Ironically “The original idea for LFS was first proposed in 1992 by Mendel Rosenblum, who also happens to be the founder of VMware.”

Cloud Flex Storage has “a 2-tier design: a capacity tier, and a performance tier. All data is stored in S3 (capacity-tier), and we use EC2 with NVMe for IO performance (cache-tier). This 2-tier design helps us decouple storage capacity from storage performance and be able to dynamically scale them independently.” 

“All incoming data is converted to large ~10MB sequential segments, and these large segments are stored as S3 objects, and S3 is excellent at large sequential IOs. This allows data to be stored in S3 at high speed.” 

Reddy says: “Cloud Flex Storage will be offered as supplemental storage to vSAN for capacity-heavy workloads.”

SDDC = Software-Defined Data Center.

Romero adds: “Longer-term, our vision for VMware Cloud Flex Storage is to deliver an enterprise-class storage and data management as-a-service for the multi-cloud.”

That’s quite clear in its implications. We should see the Cloud Flex Storage product available for Azure and the Google Cloud Platform and on-premises.

It will have added data management features, with Romero saying: “We plan to support a broad range of workloads by enabling multi-dimensional scaling of compute, storage performance, and storage capacity – while delivering a seamless and consistent data management experience across clouds.”

VMware Cloud Flex Storage is now generally available. 

SingleStore claims 100x speed boost for JSON queries

SingleStore claims to have accelerated queries up to 100x for transactional apps working with JSON semi-structured data.

This is part of SingleStore’s v8.0 release of its unified real-time transactional and analytic database, SingleStoreDB. It has hundreds of customers including Siemens, Uber, Palo Alto Networks and SiriusXM. SingleStore was ranked as a strong performer in the Forrester Wave: Translytical Data Platforms Q4 2022 report.

Raj Verma, SingleStore
Raj Verma

CEO Raj Verma said in a statement: “Real-time has been baked into our foundational design from very early on and the continued innovation with the latest announcement sets us apart as the world’s only unified database that allows you to transact and reason with data in real time in a multi cloud hybrid distributed environment.”

Version 8.0 adds a fast seeking JSON for columns feature which provides the speed increase.

Users also get autonomous dynamic workspace compute scaling when running SingleStoreDB in AWS, Azure and the Google Compute Cloud. Workspaces can be suspended and resumed to lower costs. There are real-time and historical monitoring capabilities for up-to-the second visibility into cluster performance.

SingleStoreDB already supports Common Table Expressions (CTE) – virtual tables containing a SELECT statement result set, and v8.0 adds Recursive CTE support. This makes it easier to perform analytics on hierarchical data such as in-database graph and tree expansions using one SQL statement.

Version 8.0 also adds a native client for Python, enabling developers to create up to 10x faster connections to the database, compared to a vanilla MySQL connector such as PyMySQL. It makes support for Wasm (Web Assembly) available to both Cloud and Self-Managed deployments, thus opening up availability of this feature to all customers and making it easier to port code libraries or routines (in Rust, C or C++, and soon other languages) into SingleStoreDB.

There is also extended Single Sign On support for federated authentication using OAuth.

SingleStore is running a 60-minute interactive webinar on December 15 at 10am PT to discuss this SingleStoreDB release and its development roadmap.

Bootnote

JSON (Java Script Object Notation) is a self-describing file and data interchange format using text to transmit and store data objects. JSON data is formed from name/value pairs written in between double quotes like this: “lastName”:”Doe”. Because its syntax is similar to JavaScript object code, a JavaScript program can convert JSON data into native JavaScript objects and process them.

Ondat beats Pure’s Portworx in container storage tests

Ondat has come in ahead of Pure’s Portworx in performance and latency terms when provisioning container-native storage to three databases.

An Architecting IT test, Performance Benchmarking Database Platforms and Cloud Native Storage Solutions for Kubernetes, compared five container storage providers: Ondat, Portworx, OpenEBS/cStor, Rancher Labs’ Longhorn and Rook/Ceph. It looked at performance consistence as well as latency and speed.

Analyst Chris Evans
Chris Evans

Principal analyst Chris Evans, who ran the tests, said: “Container-native storage has emerged as a new category of software-defined storage where the traditional features of data storage (resiliency, data protection, scalability) are built into containers running inside a Kubernetes cluster. This enables deployments to use local storage resources, whether that be physical storage in an on-premises solution or native storage (including NVMe) in public clouds.”

He continued: “Inefficiency in … public clouds translates into additional costs. Those costs compound as systems scale. For this reason, container-native storage must deliver high performance with low latency and minimal overhead to minimize costs, all while maintaining data integrity.”

Testing was performed on a Kubernetes cluster with the vendor software installed. The configuration of each test was based on one Kubernetes master server and three Kubernetes data servers. The testbed was a four-server configuration:

  • Dell R640 Server with 64GB DDR4-2400 DRAM
  • Dual Intel Xeon Silver 4108 CPUs at @1.80GHz
  • One WD Gold NVMe SSD (960GB capacity) data disk
  • One 136GB RAID-1 HDD pair (O/S drive, hardware RAID)
  • 10GbE Networking
  • Ubuntu Server 20.04, Kubernetes 1.22

When tested with Postgres, the data showed Ondat has 32 percent faster database transaction throughput than Portworx. OpenEBS was the slowest product. Rook/Ceph beat Portworx with Postgres in non-replication mode, as a chart shows:

Ondat Postgres

A Postgres latency chart shows similar ranking of the suppliers:

Ondat Postgres

A Redis throughput test with 100 writes to 1 read and replication switched on showed that Portworx beat Ondat in operations per second terms. However, a repeat test with no replication and a 70 write/30 read ratio saw Ondat beat Portworx:

Ondat Redis

The test report concludes: “Ondat continues to deliver the most efficient performance for throughput and latency, compared to all the competitor solutions in this test.” Get your copy and read the more detailed results here. An Ondat blog discusses the test as well.

US to slap trade restrictions on China’s YMTC

The US is set to place Chinese NAND maker YMTC on its trade blacklist in days, according to the Financial Times.

The Department of Commerce’s Bureau of Industry and Security (BIS) placed YMTC on its Unverified List (UVL) in mid-October. This is because YMTC was believed to be “at significant risk of being or becoming involved in activities contrary to the national security or foreign policy interests of the United States.”

Unless they pass US checks for activities such as not supplying China’s military organizations, UVL classified companies are moved to the full Entity List 60 days after UVL status begins. And that 60 day period is almost up for YMTC.

The FT reports that YMTC supplied NAND chips to Huawei, which would violate US export controls. YMTC had also apparently been stockpiling necessary equipment in preparation for Entity List status.

Entity List status means US business cannot sell products or services to such companies unless an export license is granted. That will restrict YMTC’s ability to produce its current 232-layer Xtacking. 3.0 NAND and develop a follow-on process.

YMTC NAND chips
YMTC NAND chips

Reuters reports that China has complained to the World Trade Organisation that US chip technology export controls are illegal. This has little to no chance of successfully getting the US to reverse its anti-China technology policy.

The US is negotiating with Japan and the Netherlands to set up a three-way deal to stop advanced chip making equipment, built by companies headquartered in those countries, from being sold to businesses in China.

SemiAnalysis consultant Dylan Patel wrote: “The US is forcefully decoupling the entire advanced technology supply chain before China insources it. Advanced logic, such as AI and HPC chips, are restricted. Equipment for 16nm or more minor logic chips, non-planar logic chips including FinFET and Gate All Around, NAND with 128-layers or more, and DRAM with 18nm half-pitch or less are restricted.”

In Patel’s view, the US has declared an economic war against China concerning advanced semiconductor manufacturing equipment. As US, Korean and Japanese chipmakers advance their DRAM and NAND technologies with denser and faster chips, the intention would seem to be that Chinese manufacturers will not be able to keep pace. This will deny that level of equipment to China’s organizations and is supposed to protect US national security interests – unless China somehow manages to develop the tools and technology needed itself.

Backblaze pours cold water on immersion cooling for most disk drive setups

Storage pod
Storage pod

Iceotope has demonstrated immersion cooling of disk drives and reckons it could enable fewer drive failures and lower total cost of ownership. A Backblaze technologist isn’t convinced.

Backblaze, which provides cloud backup and general storage, has designed its own drive chassis called Pods, and has tens of thousands of disk drives in operation. Backblaze keeps detailed statistics about its drive population over their lifetime, cataloging failures and tracking their working life and costs. 

We asked co-founder and CEO Gleb Budman what he thought about the Iceotope immersion cooling and he asked one of his technical staff to respond. The tech spokesman said: “Any kind of liquid cooling (immersion or otherwise) is basically trading complexity and difficulty in servicing for a more consistent temperature (fewer peaks and valleys) and higher heat dissipation capacity. Typically, this trade-off isn’t particularly valuable unless you have extremely high heat production (GPU compute is notable here, though CPU and RAM can get there), constant load, or both.”

This is not the situation at Backblaze because its cloud storage operation is disk-based and not compute-intensive: “In our case, we don’t use particularly high-end CPUs or RAM, we don’t use GPUs at all, and while our hard drives are constantly in use, unlike SSDs, heat is more a factor in their reliability than their performance.

Backblaze, like most everybody else, uses fan-assisted air-cooling for its server and disk chassis. Any new cooling system would need to be added to this setup and would have a cost. The tech guy told B&F: “Now, if this was a cheap and accessible solution that could just be dropped into a system, it’s… possible it’d be interesting to us, but judging by the PDF (and some prior attempts I’ve seen at this) I think the cost of implementing alone would wipe out any lengthened life of the drives it gave us.”

Backblaze does not like lock-in, it buys drives and servers from whichever manufacturer offers the best deal in terms of its requirements at purchase time. Mr techie added: “Given the diagrams there, it looks like much variation in the side of the drive or in its characteristics might mean it doesn’t fit or work as well, which… is basically always the trade-off with liquid cooling. A perfect fit system can outperform air cooling, but air cooling is much more adaptable.”

He then talked about the skill level of Backblaze technicians, saying air-cooled kit is relatively easy to fix if problems occur but liquid cooling needs plumbing skills: “Generally speaking, the datacenter techs are the start of the Operations tree in terms of skill level. We expect them to be able to do things like swap out drives and service systems because it is usually 1-2 screwdrivers worth of tools, and some basic tab/click based connectors. Easy peasy.”

“Liquid cooling, immersion or otherwise, takes a lot more care and non-zero specialized knowledge to make it work right. Having done custom loops for cooling on my home systems in the past, I can tell you I wish I had more plumbing experience when dealing with fittings, getting air bubbles out of the system, etc.”

The lliqid cooling system technology is a retrofit to a drive chassis. It’s not designed in from the start and therefore less costly: “If SMCI (or someone like them) were to come out with systems pre-prepared for this and we just dropped things in at a slight premium, we might be able to make it work. Though… by that time, if we’re starting to swap to SSDs, it kind of becomes less useful again.”

That’s because SSDs have a different form factor and may not take kindly to being drowned in liquid. Also: “SSDs tend to wear out more by usage, and less by temperature and not really at all by vibration. So even if this cooling extended the life from heat damage, it’d likely still be use that killed us first.”

His big issue with immersion cooling is liquid leakage: “I would have… non-zero concerns about catastrophic failure scenarios here. In air-cooling, worst thing that can happen is the fans break, or run in reverse, or some such. This makes the system overheat, but we have monitoring for that, and can just fix it. In any kind of liquid-cooling scenario, catastrophic failure means liquid on components that don’t want to be wet, which is a lot harder to recover from, and depending on what gets wet, may knock us down a lot more hardware.”

His summary of immersion cooling for disk drives is that “for high-volume, low-cost setups (like us), there’s not really any way I can think of that it’d make financial sense, to say nothing of the fit for the existing workforce, datacenters, etc.”

It might make sense “for high-performance, low-quantity setups” and could be interesting for them. But for Backblaze? Not today.

Komprise speeds SMB data migration to cloud 

Komprise is claiming to give SMB small file uploads to the cloud an up to 25x speed boost with its Hypertransfer for Elastic Data Migration SaaS.

Its Elastic Data Migration (EDM) product reduces the number of networking handshakes with NFS file transfers and is particularly effective with small files where the data payloads are moved quickly and the handshakes take up a disproportionate amount of the overall data set movement time. Now Komprise has worked the same handshake reduction magic on SMB file transfers.

Kumar Goswami, Komprise
Kumar Goswami

Co-founder and CEO Kumar Goswam said: “With Komprise Hypertransfer you get measurably faster SMB cloud data migrations – shaving weeks off migration timelines while minimizing the chance for errors or data loss. Komprise Hypertransfer makes cloud migrations feasible.”

The company notes that Microsoft SMB (Server Message Block) user data transfers, electronic design automation (EDA) and other multimedia workloads often contain lots of small files. It says the SMB protocol requires many back-and-forth admin handshakes that increase overall network session time. Microsoft has reduced the chattiness of SMB over time from its introduction with SMB 1.0 in 1990. SMB 2.0 in 2006 saw the command and sub-command total being lowered from over 100 to 19. SMB 3.0 in 2012 reduced chattiness and accelerated transfer speed even more by adding the SMB Direct Protocol, which used remote direct memory access, and SMB Multichannel, meaning a session could use multiple TCP/IP connections.

SMB diagram
Microsoft SMB session authentication diagram

Komprise says its Hypertransfer technology uses dedicated data channels, mitigating the SMB protocol issues, and so is comparable to SMB Multichannel. Its own testing showed that Hypertransfer improves data transfer rates across the WAN by 25x over other alternatives for SMB datasets with predominantly small files. The other alternatives would include Rsync, the basic Linux file copying tool.

A comparison with Rsync was the basis for Komprise’s claim that its Elastic Data Migration (EDM) product improved NFS small file transfer time by up to 27x compared to alternatives. Now, two years later, we have an up to 25x SMB small file transfer speed boost presented in the same way.

Hypertransfer, Komprise says, also strengthens security and ransomware defenses since data transfers from source to target pass over private channels.

The EDM offering is available as a separate service or as part of Komprise’s Intelligent Data Management offering. The Hypertransfer EDM release gives Komprise admins flexible migration configuration settings to handle read-only sources and sources with access-time tracking disabled. Users can also enable and disable data integrity checks to either ensure data accuracy or improve migration performance further.

Why hybrid data storage needs a single managed console

HPE Alletra 5k.

Sponsored Feature: Data is often analogised as the ‘lifeforce’ of the modern enterprise as it circulates around the body corporate, from analytics teams to business units, sparking technological innovation and energising commercial initiative.

What was once seen ostensibly as static accumulations of records, data has become transformed into valuable digital assets, analysed and repurposed to drive new business.

But as these assets have enlarged in volume and complexity, storing and managing them has created ongoing challenges for IT planners. Cloud-hosted repositories seemed to offer a solution, but there remain significant numbers of organisations that, according to given business imperatives, want to run critical applications and workloads off-cloud, on-prem.

Data now has to be highly accessible and mobile, so that it can be moved between applications as workloads require. To enable this, storage solutions providers have been tasked with re-engineering the way their clients’ all-important data is kept and managed.

With data stores located across multiple locations both on- and off-cloud, it can prove challenging to ensure security and compliance, as well as to meet carbon reduction targets by consolidating and optimising the infrastructural locations where data is kept, says Matt Shore, Business Development Lead, Data Services at HPE.

“Putting the term ‘locations’ into context here is instructive,” he explains. “Increasingly, the location where enterprise data exists affects its accessibility, mobility and therefore its value. With today’s hybrid and multicloud environments, data can end-up stored on diverse platforms, each with its own management tools, with limited interoperability between them.”

This means storage administrators have to use different management systems for on- and off-cloud data operations, resulting in an onerous resource overhead – especially with line-of-business applications and software developments that must access data from various platforms.

This situation also leaves key datasets largely locked into the infrastructure where they were generated and/or first stored. Moving all data into one, cohesive data repository, so it can be analysed to inform business decisions, becomes clunky.

These inhibitors informed the evolution of HPE’s GreenLake – a self-service, HPE-managed platform that combines the simplicity and agility characteristic of public clouds with the security, governance and compliance, and performance advantages of on-premises IT infrastructure.

Leveraging HPE’s SaaS-based Data Services Cloud Console – that delivers cloud operational agility and unified data operations – GreenLake flexibly optimises all kinds of enterprise applications and workloads.
Working with HPE’s Alletra family of storage platforms, the solutions work together to enable storage administration teams to manage their on-premises data using cloud-native tools that would not otherwise be available to them.

Optimized for any application generation

HPE Alletra is HPE’s family of all-flash and hybrid block storage systems. Optimised for any generation of application, HPE Alletra offers architectural flexibility without the complexity of traditional storage management.

“Cloud sets a standard for agility with on-demand access, resource elasticity, and charges for services based on actual usage,” says Nigel Williams, HPE’s WW Storage Field CTO for UKIMEA. ”We were finding that clients wanted the cloud management tools experience while managing data on their on-premises data storage platforms, while also managing data stored with their appointed Cloud Service Provider.”

Williams adds: “They encountered more and more storage complexity because those disparate workloads had to be moved and configured using different management consoles. And that complexity was proving onerous, leaving IT teams with less time for business development projects. It’s precisely the kind of predicament that the HPE GreenLake/Alletra combination was designed to solve.”

Acknowledgement of the advantages of a unified cloud-native console is at the heart of HPE Alletra’s value proposition, explains Shore. “With HPE’s Data Services Cloud Console we are stepping away from the ‘single pane of glass’ conception of a management console, because it’s just not accurate,” Shore says. “What we’re providing is a single management console experience. It’s an important distinction.”

It’s a distinction that’s resonating with clients. Leading manufacturer Stelmet, for instance, recently chose HPE Alletra because of how it facilitates the company’s need to respond to any workload demand and break down complex data silos. As a result, the Stelmet can significantly accelerate IT-driven services and free-up resources that were previously required to maintain, upgrade, and tune storage in remote locations.

Williams adds: “The thinking behind HPE Alletra’s technological and pricing options is not about trying to ‘out-smart’ pure-play public cloud offerings. Rather, we’re positioning the products to be the orchestration layer for enterprise IT infrastructures that need to span on-premises and public cloud and hybrid operations.”

Move on up: data mobility

The thinking behind the HPE Alletra family goes further than providing a flexible storage platform that enables cloud-based data assets to also fold into HPE’s single management console experience. Its functionality also takes account of how OPEX considerations are a greater factor in data storage solutions strategies.

This is why the HPE Alletra platforms are also positioned to help organisations that are moving on from a previous ‘cloud-first’ strategy, in favour of what Shore calls a ‘cloud-costed’ approach.

“There are advantages that come with running your IT in the cloud, but that’s not the whole story,” says Shore. “For multiple reasons, clients tell us that ‘cloud-first’ often turns out not to be the most propitious strategic choice for their total business requirements. They’re reverting selected workloads and applications to on-premises. For this reason HPE’s guiding principle is the hybrid model. It’s the growth in this requirement that our HPE Alletra platforms are designed for also.”

Onboard security has become another operational imperative, Williams adds. “The HPE Alletra family comes with extensive cyber-security features. We can provide lots of ability to include extra layers of security for a client organisation’s IT security policy. HPE Alletra’s backup data immutability is a key tool in our ability to protect backup data against attacks, such as ransomware.”

Backup data immutability prevents a backup being deleted or modified by a malicious threat before the client-configured retention date.

“We say with a conviction borne of experience that snapshot immutability can play a strong part in the fight against cyber-threats such as ransomware,” Williams says. “All HPE Alletra’s cyber-security features can be made part of an HPE-managed service, for clients that do not have the security skills resource in-house.”

Why simplicity costs

Pricing and OPEX/CAPEX trade-offs are also becoming increasingly important to informing cloud decisions, Shore reports. “We realise this, and it’s why with HPE GreenLake, clients can consume our HPE Alletra options as a service for pay-per-use, scale-up-and-down freedom. Or they can have the service managed for them, as a turnkey solution. They can also have a mix of subscription and usage-based services, if that delivers the financially agility they’re looking for.”

According to Shore, OPEX financials are becoming a decisive factor in storage administrators’ on-cloud/off-cloud decision-making.

“Received notions of ‘simplifying IT’ need to be unpacked,” Shore explains. “Say an organisation wants to simplify its IT infrastructure, including its storage strategy. That’s understandable – but at what financial cost? Cloud will simplify management to a degree, but will impose a charge for doing so. And those charges are not proving upwardly immobile.”

Organisations like the cloud experience and all its operational conveniences, but such features can come at a much higher price than they had originally anticipated, adds Williams. “Clients who started out with a wholehearted ‘cloud-first’ strategy are now reassessing the implications of that decision from a more strictly commercial perspective,” says Williams. “This is why the recent rise in popularity of hybrid cloud has been so rapid.”

Energy savings: every little helps

COP targets have brought IT operations into sharp focus in regard to meeting emissions reduction targets. While fully carbon neutral computing solutions are still generations away, many vendors are committed to wringing maximum performance gains for the energy their products use – including storage products. And as more stored data is accessed by applications that demand advanced CPU architectures and storage bandwidth, storage hardware is using more energy than it used to.

It’s why HPE Alletra is powered by the latest generation of AMD’s EPYC processor range. These are engineered for utmost energy efficiency, so that organisations can run compute-intensive workloads without necessarily bumping-up compute energy consumption, which contributes to reduced OPEX costs.

“For years, AMD has prioritised the ‘performance-per-watt’ principle in its CPU design and development,” Williams points out. “The latest AMD EPYC architectures that power HPE Alletra are designed to provide optimum cores-per-processor. Having them at the core of our storage solutions makes energy efficiency a differentiator that delivers sustainability gains to both technology partners and their customers.”

AMD gives the HPE Alletra portfolio “an enormous core count per-watt,” adds Shore. “We take full advantage of the fact that AMD’s power-per-watt is class-leading. The AMD EPYC 7002 series processors that power the HPE Alletra 6000, for example, are able to scale from 8 to 64 high-performance cores, yet pull much less power than any other processor we looked at.”

Clearly, the importance of effective storage has been elevated by a combination of value-driven data analytics and the necessity to recover and restore most recent data and applications in the event of a cyber-attack.

Hard-pressed IT teams need all the help they can afford, and the ability to more easily manage disparate data assets stored across cloud and non-cloud environments will be well received.

To find out more on how to simplify data management with HPE Storage click here.

Sponsored by HPE & AMD.

Catalogic protects Azure and GCP VMs against ransomware

Data protector Catalogic has expanded its Guard Mode ransomware protection coverage and now protects virtual machines running in Azure Stack and the Google Compute Engine

Catalogic has its Cloud Backup-as-a-Service offering for containerized applications on-premises or in the cloud, and its DPX suite for bare metal and virtualized workloads running on-premises. It can back up to on-premises object stores or cloud storage. The DPX family includes vPlus for Microsoft 365, which covers Exchange Online, SharePoint Online, OneDrive for Business, and Teams. The vPlus for Open VMs offerings supports Citrix Hypervisors, KVM, Nutanix Acropolis or AHV, Oracle VM, Proxmox, RHEV/oVirt, Scale Computing HyperCore/HC3, and XenServer, along with Amazon EC2.  

Ken Barth.

Catalogic CEO Ken Barth said: “Backups are now the last line of defense against cyber security threats such as ransomware and malware. Catalogic is committed to stretch left and offer proactive security solutions for our data protection customers. … With these new enhancements to DPX GuardMode combined with DPX instant recoveries, Catalogic customers will be among the best prepared to detect and recover from a cyber attack.”

“Stretch left” is Catalogic-speak for protecting Kubernetes and cloud environments. We think it comes from the containerized DevOps world where “Shift Left” is jargon for automating cloud-native application test, management and operational processes and doing them early on in an application’s life cycle. Stretch left sounds DevOpsy and modern, and would logically include SaaS applications. More on this in a minute.

Catalogic data protection universe.

Catalogic has introduced v4.9 of DPX, having announced v4.8 in July. This included Guard Mode for Windows, which detects ransomware by monitoring file access behavior patterns – local or networked – and comparing them to over 4,000 known and constantly updated ransomware threat patterns with so-called honeypots used as decoys for ransomware attacks. Once threats are detected, admin staff receive alerts.

It has introduced Guard Mode for Linux servers and Samba shares. Backup administrators can also benefit from an increased quality of alerts, where DPX GuardMode measures the level of file entropy and compares known magic signatures on files suspected to be impacted. A file’s entropy level indicates the amount of random data in a file. A magic signature refers to a file’s type – the first few so-called magic bytes in its data that identifies its format.

Catalogic says GuardMode uses active, live forensic techniques instead of analyzing backup data that lags security incidents by several hours, days and even weeks.

One thing that we suspect Catalogic has on its roadmap is protection for SaaS applications beyond M365. It is going to stretch even further left, we envisage, to cover things like Salesforce, ServiceNow, GSuite, Box and Dropbox. Otherwise, companies like OwnBackup and Druva will start eating its SaaS lunch.