Optimize the ingest and processing of data for ultra-fast performance.
Video content analysis (VCA), also referred to as video analytics, video intelligence and intelligent video analysis, has become even more valuable over the past several years as advanced artificial intelligence (AI) engines have added new capabilities, allowing it to process more information, more accurately – and doing it all in real-time.

This has resulted in explosive growth in applications used in a diverse number of industries such as industrial manufacturing, military and security. Developing and implementing ultra-low latency storage networks is the greatest challenge to extracting the most value from VCA. Unless the data extracted from video can be processed, analyzed and used to develop timely decisions and recommendations, it is for naught.

Incorrectly manufactured products resulting in expensive scrap, military operations becoming more dangerous as boots-on-the-ground information becomes stale, and failing to identify security risks in time to mitigate the problems are all examples of why high-performance storage is the most vital component in making VCA work well.

A single data frame of high-definition video often represents more than 1GB of valuable data, more when multiple views of the same object must be evaluated simultaneously. Solid-state disk (SSD) storage struggles to keep up with the constant reading and writing of such large data sets. Additionally, they lack resiliency and wear out quickly, resulting in frequent and expensive upgrades costing upwards of $100K, as well as the hit to total cost of system ownership (TCO)

VCA applications need a block storage solution that is faster than flash storage, shareable and easy to manage. It should also seamlessly integrate into existing storage systems so as not to interrupt ongoing projects.
The Obstacle
Using AI allows us to quickly analyze data and provide results based on pre-determined criteria established during application training. It allows the user to quickly answer questions such as ‘is this product defect-free?’ or ‘is this a military target of interest?’ and even ‘is the person at an airport checkpoint on a no-fly list?’

Answering these questions among others requires ultra-fast ingest of data, typically from a remote location. That data must then be moved across a network, avoiding congestion and latency, and into the specialized hardware engines that evaluate it. Some of this data must be shared across multiple systems, while other data ultimately ends up being saved on a more permanent basis for evidentiary or future training purposes.

Typical storage systems aren’t built to handle the sheer volume of data being thrown at it in VCA systems. They are unable to keep up and latency limitations eliminate any possibility of being able to provide timely analysis. The key challenge is getting data to the AI engine reliably and quickly enough and then writing completed meta-data to a database.

At the server level, options for performance improvements typically involve a forklift upgrade or the addition of multiple servers configured to act only as storage. These options are exceedingly expensive and amount to a “kitchen sink” approach that does not efficiently address the core issue which is storage latency.

Since the storage is likely already shared, flash-based storage, the performance is already as good as it can get. Adding JBOFs might add capacity, but not speed and would not address storage latency.

Additionally, traditional Ethernet networks can be both a generation or more outdated and challenging to configure for optimum performance, creating limitations in how quickly data can get to the AI engine in a VCA ecosystem.

Most storage vendors have little understanding of these applications and the particular challenges to these environments. Fortunately, ATTO Technology are experts in storage ecosystems that support digital video and real-time analysis applications and we have solved storage-related problems for high-profile customers over the past 33 years.
The Solution
The key to this solution is to utilize the fastest shared storage available while reducing latency in the storage network. The fastest shared storage available today is ATTO SiliconDisk.

SiliconDisk is a scalable, sharable state-of-the-art RAM-based storage appliance that delivers faster performance than flash-based SSD storage solutions, and with much lower latency. Data is instantly stored and retrieved with incredible speed, mitigating the read and write latency that leads to inefficiencies in daily operations.

Because SiliconDisk sits on the network, data is available to all users simultaneously without unnecessary read and write operations slowing down overall network performance. The persistent RAM-based SiliconDisk would be a cost-effective solution because it maintains its reliability even with an almost infinite number of drive writes per day – something that would degrade traditional storage in short order.

Solving the network issues is fairly straightforward using RDMA Ethernet protocols to raise QoS (Quality of Service) and eliminate network latency. ATTO recommends using the NVMe-oF (NVMe over Fabrics) protocol, a relatively new technology that is starting to be used in latency-sensitive applications, like real-time video analysis, to provide the needed performance. This brings typical network latency down to less than 10 microseconds, a 100x improvement over the more than 1-millisecond latency common to traditional network architectures.
The Outcome
Storage investments decisions involve much more than just acquisition costs, especially with complicated applications such as VCA. Compare an investment in SiliconDisk storage to other solutions involving servers relegated to simple storage devices and you’ll find it strike the right balance in cost versus performance, reliability and ease-of-integration. In the end, users would quickly recoup their investment and drive their application performance to new heights.

In digital video analytics applications, the ultimate metric is the number of images that can be ingested, evaluated and decisions made in the shortest amount of time – and traditional storage just can’t keep up. Only SiliconDisk provides the throughput and latency needed to support real-time analytics.

SiliconDisk provides all of this while still being 100% shareable. Since it’s available to all designated hosts on a network, large files do not need to be stored in multiple places and valuable network bandwidth isn’t consumed with unnecessary copy operations. Data is available close to where it’s needed at all times – with no performance penalties.

Finally, since SiliconDisk does not wear out, the cost of drive replacements for production could all but be eliminated.

Ready to Try SiliconDisk?
If you’re ready to try ATTO SiliconDisk, we are happy to oblige. Fill out the form below or contact us directly at +1-716-691-1999 to begin the free evaluation process.