The German Climate Computing Centre (DKRZ) is replacing its HPSS hierarchical file management system and migrating from Oracle StorageTek to LTO tape.
DKRZ provides high performance computing, storage and services for simulation-based climate science. Computing time, storage and archive capacity, consultancy and visualisations are free of charge to German research scientists. DKRZ says its data management system is one of the largest in the world.
Prof. Thomas Ludwig, CEO at DKRZ, supplied a quote: “This new system will enable DKRZ to expand and modernise essential data services to our research community, while also seamlessly transitioning to a new open-standard platform that can grow as we do. This will provide a solid foundation to ensure researchers have on-going access to the critical data needed for climate modelling and improved climate projection”
DKRZ signed a €32.5m contract in June with Atos for a BullSequana XH2000 supercomputer. This increases compute power “by 5”, compared with the current Mistral supercomputer, provided by Atos in 2015. Mistral uses the Lustre file system with ClusterStor 9000 and L300 storage, Oracle/StorageTek SL8500 tape libraries, Swift object storage and the HPSS system.
HPSS is software for scalable, policy-based Hierarchical Storage Management (HSM). HSM manages file storage across fast-access, online drives and back-end tape or public cloud object storage to optimise performance and bulk capacity costs. The HPSS software is the responsibility of a consortium of IBM and more than 20 HPC users worldwide.
DKRZ’s new BullSequana is based on AMD EPYC x86 processors, about 3,000 computing nodes with a total peak performance of 16 PetaFlops, 800TB of main memory, a 120TB DDN storage system, and Nvidia InfiniBand HDR 200G interconnect technology. The new setup is expected to go live in mid-2021.
The HPSS HSM will be replaced by StrongBox, a company that provides native support for tape libraries from any vendor. Using StrongBox’s StrongLink software, DKRZ will automate multi-tier storage workflows, virtualizing flash, disk, and LTFS tape to support at least 120PB per year of new HPC research data flows.
The project involves managing the online data and migrating 150PB of legacy HPSS tape data to LTO format tape. StrongLink’s metadata-driven workflow engines will virtualize 1PB of new high-performance tier-1 cache, and five legacy tier-2 tape libraries. It will automate day-to-day tiering operations and the background HPSS-to-LTFS migration. Tape IO is to be parallelised across 90 tape drives in the five libraries, which house more than 75,000 media slots.
Users will continue to access data as before, with StrongLink presenting a global normalised view of all data across all storage types.
Floyd Christofferson, CEO of StrongBox Data Solutions, offered a quote: “StrongLink was designed to provide customers like DKRZ with a vendor-neutral platform that could seamlessly automate data management and high speed data movement across otherwise incompatible storage types at any scale.”
StrongBox Data Solutions was set up in 2016 to buy the beleaguered CrossRoads Systems and its StrongBox archiving technology. This contract is a big deal for the private equity-owned firm which only added LTFS support in June.
