Using AI allows us to quickly analyze data and provide results based on pre-determined criteria established during application training. It allows the user to quickly answer questions such as ‘is this product defect-free?’ or ‘is this a military target of interest?’ and even ‘is the person at an airport checkpoint on a no-fly list?’

Answering these questions among others requires ultra-fast ingest of data, typically from a remote location. That data must then be moved across a network, avoiding congestion and latency, and into the specialized hardware engines that evaluate it. Some of this data must be shared across multiple systems, while other data ultimately ends up being saved on a more permanent basis for evidentiary or future training purposes.

Typical storage systems aren’t built to handle the sheer volume of data being thrown at it in VCA systems. They are unable to keep up and latency limitations eliminate any possibility of being able to provide timely analysis. The key challenge is getting data to the AI engine reliably and quickly enough and then writing completed meta-data to a database.

At the server level, options for performance improvements typically involve a forklift upgrade or the addition of multiple servers configured to act only as storage. These options are exceedingly expensive and amount to a “kitchen sink” approach that does not efficiently address the core issue which is storage latency.

Since the storage is likely already shared, flash-based storage, the performance is already as good as it can get. Adding JBOFs might add capacity, but not speed and would not address storage latency.

Additionally, traditional Ethernet networks can be both a generation or more outdated and challenging to configure for optimum performance, creating limitations in how quickly data can get to the AI engine in a VCA ecosystem.

Most storage vendors have little understanding of these applications and the particular challenges to these environments. Fortunately, ATTO Technology are experts in storage ecosystems that support digital video and real-time analysis applications and we have solved storage-related problems for high-profile customers over the past 33 years.

The key to this solution is to utilize the fastest shared storage available while reducing latency in the storage network. The fastest shared storage available today is ATTO SiliconDisk.

SiliconDisk is a scalable, sharable state-of-the-art RAM-based storage appliance that delivers faster performance than flash-based SSD storage solutions, and with much lower latency. Data is instantly stored and retrieved with incredible speed, mitigating the read and write latency that leads to inefficiencies in daily operations.

Because SiliconDisk sits on the network, data is available to all users simultaneously without unnecessary read and write operations slowing down overall network performance. The persistent RAM-based SiliconDisk would be a cost-effective solution because it maintains its reliability even with an almost infinite number of drive writes per day – something that would degrade traditional storage in short order.

Solving the network issues is fairly straightforward using RDMA Ethernet protocols to raise QoS (Quality of Service) and eliminate network latency. ATTO recommends using the NVMe-oF (NVMe over Fabrics) protocol, a relatively new technology that is starting to be used in latency-sensitive applications, like real-time video analysis, to provide the needed performance. This brings typical network latency down to less than 10 microseconds, a 100x improvement over the more than 1-millisecond latency common to traditional network architectures.

Storage investments decisions involve much more than just acquisition costs, especially with complicated applications such as VCA. Compare an investment in SiliconDisk storage to other solutions involving servers relegated to simple storage devices and you’ll find it strike the right balance in cost versus performance, reliability and ease-of-integration. In the end, users would quickly recoup their investment and drive their application performance to new heights.

In digital video analytics applications, the ultimate metric is the number of images that can be ingested, evaluated and decisions made in the shortest amount of time – and traditional storage just can’t keep up. Only SiliconDisk provides the throughput and latency needed to support real-time analytics.

SiliconDisk provides all of this while still being 100% shareable. Since it’s available to all designated hosts on a network, large files do not need to be stored in multiple places and valuable network bandwidth isn’t consumed with unnecessary copy operations. Data is available close to where it’s needed at all times – with no performance penalties.

Finally, since SiliconDisk does not wear out, the cost of drive replacements for production could all but be eliminated.