Behind ATTO SiliconDisk: A High-Performance Storage Gap

This article is part two of a multi-part series that describes the impetus and technology behind ATTO SiliconDisk RAM-based storage appliance. For more details, visit https://www.atto.com/products/silicondisk.

Collectively, the prevailing storage options leave a gap in the storage pyramid thereby limiting the performance of the latest data-hungry applications. The size and complexity of data pools, the need to access data more quickly and the value of the data itself have only further magnified the problem. Real-time data (RTD) and artificial intelligence (AI) operations are among the most demanding processes today.

RTD is data that is available for processing immediately after it is generated or collected. This data is typically a mix of data freshly collected processed against an active dataset. A basic example would be real-time economic data converted into real-time economic analysis.

International Data Corporation (IDC) projects RTD to comprise nearly 30% of all data by 2024 and that 60-70% of the Global 2000 will have at least one mission-critical real-time workload in 2021. Presently, real-time data application performance is outpacing the ability of conventional memory and storage architectures to supply data to the application.

AI models have expanded in complexity by 50 times in less than two years, while the rate at which the hardware needed to process the models has lagged behind. Like RTD applications, the needs of a system optimized for AI have outgrown the traditional memory/storage paradigm.

Systems architected to process real-time data and artificial intelligence workloads are complex. The CPU is not the sole destination for data nor the sole generator of data used and reused throughout these processes. GPUs, DPUs, and even FPGAs and ASICs are employed throughout systems to process data and add to the demands on data I/O. System memory can only store so much before data needs to be offloaded into storage and often leads to a data-greater-than-memory problem in these architectures.

This is why the first tier down after RAM is so critical. The data offloaded to storage by these applications is often needed again immediately for additional processing. However, offloading to even SSDs can quickly create a significant bottleneck and the subsequent performance drop can be, by some estimates, several thousand times slower or more depending on the storage type.

All of this represents the consequential gap in the storage pyramid that the industry is trying to fill. Solutions range from offloading storage processes, various methods of SSD connectivity, enhancing storage operations through software-defined memory, to JBOFs managed by DPUs and new, faster types of Flash storage with DRAM-like characteristics.

ATTO SiliconDisk™ is an ultra-low-latency RAM-based storage appliance that provides near-RAM level performance over an Ethernet fabric. It fills the gap between RAM and storage, providing the performance needed by emerging computing loads.