Choosing the Right SAN Fabric and Protocol
Designing the right SAN effectively amounts to a choice between using Fibre Channel (FC) or Ethernet. Ethernet SANs are nearly always implemented in the context of a converged network architecture; FC networks are always utilized as dedicated SANs.
The choice depends on the answers to these questions: How many application instances need to access the shared storage simultaneously? What are the storage performance needs of each application instance? How much latency is acceptable? Will jitter affect the goals of the application? As a dedicated storage network, FC SANs provide significantly better performance in most real-world use cases than Ethernet-based alternatives, albeit with higher CapEx and OpEx costs. However, FC SANs are highly scalable and are immune to network congestion, enabling FC SANs to be built with thousands of nodes without impacting throughput, latency or jitter.
For enterprise applications where the size of the SAN is in the thousands of ports, or where application instances require both high throughput and low latency, FC SANs continue to be the lowest risk choice. The ability to scale and grow FC SANs without impacting performance offsets the higher CapEx. Because FC SANs are dedicated to storage traffic, they can consistently provide high throughput, low latency, and extremely low jitter.
Where the cost of shared storage is the driving factor and performance and data delivery reliability are not critical, iSCSI is the common choice albeit at the expense of reduced throughput, typically 40%-50% of line rate. While the theoretical scale of Ethernet is unlimited, iSCSI over Ethernet networks start to experience congestion and "noisy neighbor" issues well before they reach even half of their theoretical throughput.
In smaller contexts, or where there are budgetary constraints, Ethernet is the only remaining choice beyond FC. Designers need to be aware of the limitations of Ethernet in this application in order to effectively minimize the impact of those limitations.