GPUs have an insatiable want for knowledge and maintaining these processors powered generally is a problem. That is one of many principal the explanation why WEKA final week launched a brand new line of information storage gadgets that may transfer knowledge at as much as 18 million IOPS and serve 720 GB of information per second.
The newest GPUs NVIDIA It could actually ingest knowledge from reminiscence at unimaginable speeds, as much as 2TB of information per second for the A100 and three.35TB per second for the H100. Such a reminiscence bandwidth, utilizing the most recent HBM3 normal, is required to coach the biggest giant language fashions (LLMs) and run different scientific workloads.
Retaining PCI buses saturated with knowledge is crucial to using the complete capability of GPUs, and that requires a knowledge storage infrastructure that may sustain. the individuals of WEKA They are saying they’ve finished simply that with the brand new WEKApod line of information storage gadgets they launched final week.
The corporate provides two variations of the WEKApod, together with Prime and Nitro. Each households begin with clusters of eight rack servers and about half a petabyte of information, and may scale to assist a whole bunch of servers and a number of petabytes of information.
The Prime line of WEKApods is predicated on PCIe Gen4 expertise and 200Gb Ethernet or Infiniband connectors. It begins with 3.6 million IOPS and 120 GB per second learn efficiency, and goes as much as 12 million IOPS and 320 GB learn efficiency.
WEKApod specs
The Nitro line is predicated on PCIe Gen5 expertise and 400Gb Ethernet or Infiniband connectors. Each the Nitro 150 and Nitro 180 are rated at 18 million IOPS of bandwidth and may obtain knowledge learn speeds of 720 GB per second and knowledge write speeds of 186 GB per second.
Enterprise AI workloads require excessive efficiency for each studying and writing knowledge, says Colin Gallagher, vp of product advertising and marketing at WEKA.
“These days, a number of of our opponents have claimed to be the very best knowledge infrastructure for AI,” Gallagher says in a video on the WEKA web site. “However to take action they selectively cite a single quantity, usually one to learn knowledge, and omit others. For contemporary AI workloads, a efficiency knowledge determine is deceptive.”
It is because, in AI knowledge pipelines, there’s a essential interplay between studying and writing knowledge as AI workloads change, he says.
“Initially, knowledge is ingested from varied sources for coaching, loaded into reminiscence, preprocessed, and written again,” Gallagher says. “Throughout coaching, it’s constantly learn to replace the mannequin parameters, save checkpoints of varied sizes, and write the outcomes for analysis. After coaching, the mannequin generates outcomes which can be written for later evaluation or use.”
WEKAPods use the WekaFS file system, the corporate’s high-speed parallel file system, which helps quite a lot of protocols. The gadgets assist GPUDirect Storage (GDS), an RDMA-based protocol developed by Nvidia, to enhance bandwidth and scale back latency between the server NIC and GPU reminiscence.
WekaFS has full assist for GDS and has been validated by Nvidia together with a reference structure, WEKA says. WEKApod Nitro can be licensed for Nvidia DGX SuperPOD.
WEKA says its new gadgets embody quite a lot of enterprise options, similar to assist for a number of protocols (FS, SMB, S3, POSIX, GDS and CSI); encryption; backup/restoration; snapshots; and knowledge safety mechanisms.
Particularly for knowledge safety, it says it makes use of a proprietary distributed knowledge safety encoding scheme to guard in opposition to knowledge loss attributable to server failures. The corporate says it provides the scalability and sturdiness of erasure coding, “however with out the efficiency penalty.”
“The accelerated adoption of generative AI and multi-modal restoration augmented technology functions has permeated the enterprise sooner than anybody might have predicted, driving the necessity for reasonably priced, high-performance and versatile knowledge infrastructure options that provide low latency. extraordinarily low and drastically scale back the price. by generated tokens and may be scaled to satisfy the present and future wants of organizations as their AI initiatives evolve,” mentioned Nilesh Patel, chief product officer at WEKA, in a press launch. “WEKApod Nitro and WEKApod Prime provide unparalleled flexibility and choices whereas delivering distinctive efficiency, energy effectivity and worth to speed up your AI tasks wherever and everytime you want them to run.”
Associated articles:
Legacy knowledge architectures holding GenAI again, WEKA report says
Object and file storage have merged, however variations between merchandise stay, says Gartner
