Comparing Fibre Channel Switched Network and Gigabit Ethernet
users.nni.com
Purpose
McDATA Corporation sales persons are frequently asked to compare Switched Ethernet client/server networks with Fibre Channel Storage Area Networks. This paper will explain the basic concepts of Ethernet, and describe the similarities of and the differences between Switched Ethernet networks and Fibre Channel Switched Fabric SAN’s. A working knowledge of FC-SW (Fibre Channel, Switched Fabric) is assumed.
Ethernet Background
Traditional Ethernet
Historically, LANs grew and proliferated in a shared environment characterized by several LAN access methods. For instance, the MAC (Media Access Protocol) protocols for Ethernet, Token Ring and FDDI (Fiber Distributed Data Interface) each have arbitration rules that determine how data is transmitted over a shared physical media type. Most traditional Ethernet networks use unshielded twisted pair copper wire (UTP).
Traditional Ethernet LANs run at 10 or 100 MB/S over a common bus-type design. Stations physically attach to this bus through a hub, repeater or concentrator, creating a broadcast domain. Every station is capable of receiving all transmissions from all stations, but only in a half-duplex mode. This means that stations cannot send and receive data simultaneously.
Furthermore, nodes on an Ethernet network transmit information following a simple rule: they listen before speaking. The Ethernet environment allows only one node on the segment to transmit at any time due to the CSMA/CD (Carrier Sense Multiple Access/Collision Detection) protocol.
Although this manages packet collisions, it increases transmission time in two ways:
1) If two nodes begin speaking at the same time, the information collides and they both must stop transmission and try again later.
2) Once a packet is sent from a node, an Ethernet LAN will not transfer any other information until that packet reaches its endpoint.
These two factors slow down Ethernet networks. Countless hours have been lost waiting for a LAN to free up. The bigger the network segment, the longer the wait.
Switched Ethernet
A new class of interconnect products has emerged that can boost bandwidth on overburdened, traditional LANs while working with conventional cabling and adapters. These are known as LAN switches and are available for Ethernet, token ring, and FDDI. A switch provides the capability to increase the aggregate LAN bandwidth dramatically, because it allows for simultaneous switching of packets between its ports.
Ethernet switches segment a LAN into many parallel dedicated lines that enable a contentionless, scalable architecture. All nodes always have an open line to the network. A switch port may be configured in two ways:
1) as a segment with many stations attached to it
2) with a single station connected to a port.
In Scenario 1, the segment is a collision domain and the rules of contention based on CSMA/CD are in effect. The rule is that only one conversation may originate from any individual port at a time, regardless of whether there is one or many stations connected to that port. That is, all ports still listen before they speak, and cannot speak until the line is clear.
In Scenario 2 a single LAN station is connected to a switched port therefore it may operate in full-duplex mode on dedicated media. Full-duplex (or sending and receiving at the same time ) does not require collision detection because it always has an open line to the network, hence there is a suspension of CSMA/CD protocols. A single device resides on that port, and therefore no collisions will be encountered. Full-duplex switching enables traffic to be sent and received simultaneously. Aggregate throughputs for 10 MB/S Ethernet networks jump to 20 MB/S , from 100 Mbps to 200 Mbps.
Fibre Channel and Switched Ethernet - A Comparison
Network Speed
Fibre Channel and Switched Ethernet operate at 1 Gb/S or 100 MB/S, full duplex. The opportunity for Fibre Channel speed to surpass Switched Ethernet is real with plans for 2, 4, 8 GB/S already in the standards committee. Switched Ethernet speed increases are not in committee and are not expected any time soon.
Supported Media Types
<edited>
Reliability
Bit Error Rate (BER) is the ratio of received bits that contain errors to the total number of received bits. Ethernet network copper links have a BER of 1 in 106. Fiber-optic Fibre Channel links have an error rate of 1 in 1011.
Fibre Channel uses IBM’s 8B/10B encoding scheme to overcome oscillator synchronization and switching threshold differences between the sending and receiving nodes. Using 8 bit bytes, every combination of “1” and “0” is legal. Two extra bits are encoded into each byte, a code set with a greater number of combinations is achieved, and the most desirable bit pattern may be selected. This ensures that every character has some transitions to account for node oscillator differences and there are an equal number of “1” and “0” bits in each character to account for switching threshold differences between sender and receiver.
These characteristics allow design of inexpensive Fibre Channel transmitter and receiver circuitry that can operate at a BER of better than 1 in 10 12.
Media Access
Fibre Channel Switched Fabric and Switched Ethernet transports can, when properly configured topologically, operate with dedicated media. A virtual, point to point path is assured between any nodes on both of the respective Ethernet and Fibre Channel fabrics
Both transports can add blocking segments to their fabrics. Multiple clients can be configured to share a single switched Ethernet port, in which case, CSMA/CD is used as in the traditional Ethernet architecture. Similarly, Fibre Channel fabrics can deploy Fibre Channel Arbitrated Loop (FC-AL) segments connected to the fabric by a FL port. FC-AL elements gain shared media access in a way more similar to Token Ring than CSMA/CD, but the concept and the non-deterministic nature of this access is the same as traditional Ethernet.
Protocols Transported
Perhaps the most significant area of difference between Switched Ethernet and Fibre Channel Switched Fabric is the upper layer protocols that can be transported on the two technologies.
Ethernet can carry a wide variety of network protocols TCP/IP, 802.2, 802.3, NetBIOS, NetBEUI, IPX/SPX, among others.
Fibre Channel can carry the network protocol IP and the channel protocols HIPPI, IPI, IBM’s SBCON (FICON), and SCSI. The ability of Fibre Channel to carry FICON and SCSI traffic is what will lead the explosion of Fibre Channel and the proliferation of the Storage Area Network.
Switch Architecture
ASIC technology is widely used in Ethernet switches and Fibre Channel switches to process packets.
ASICs (Application Specific Integrated Circuit) are custom designed to handle specific operations: all of the functionality is "cast" in hardware. This means that if any changes are needed, manufacturing must be done to rework the silicon. No easy software upgrades are available.
ASICs can perform cut-through forwarding of frames based on MAC destination or Fibre Channel fabric addresses. A crosspoint switch matrix is a single ASIC that creates dedicated physical paths between any input port and the destination output port. It scales well and does not require the buffering of store-and-forward.
Cut-Through and Store-and-Forward
Two architectures determine switching applications and performance: cut-through and store-and-forward.
Cut-through switching starts sending packets as soon as they enter a switch and their destination address is read (within the first 20-30 bytes of the frame). The entire frame is not received before a switch begins forwarding it to the destination port. This reduces transmission latency between ports, but it can propagate bad packets and broadcast storms to the destination port.
Store-and-forward switching, a function traditionally performed by Ethernet bridges and routers, buffers incoming packets in memory until they are fully received and a cyclic redundancy check (CRC) is run. Buffered memory adds latency to the processing time and increases in proportion to the frame size. This latency reduces bad packets and collisions that can adversely effect the overall performance of the segment.
Some switches perform on both levels. They begin with cut-through switching, and through CRCs they monitor the number of errors that occur. When that number reaches a certain point, a threshold, they become store-and-forward switches. They remain so until the number of errors declines, then they change back to cut-through. This type of switching is called threshold detection or adaptive switching.
Locality of Reference
Switched Ethernet LANs and WANs consist of many clients that access shared network devices such as files and printers. Virtually all communications travel between a relatively low powered client and a high powered server. The locality of reference is many to one.
Storage area networks have a different reference. Traffic exists between high powered servers and high powered storage ports. The locality of reference is many to many.
Price and Availability
The prices of Switched Ethernet products are lower than that of Fibre Channel products. Ethernet switches can be obtained for around $1000 per port. Fibre Channel switches are more expensive. Ethernet has been deployed for about twenty-five years, thus infrastructure costs are lower than Fibre Channel. Trained personnel to run the Ethernet networks are easier to hire because the technology has been around longer.
WAN Access
Access to WAN’s and the Internet is difficult at this time for Fibre Channel. The edge path connections or gateways from a Fibre Channel SAN to the outside world are not presently available. Ethernet comes pre-wired to the outside world.
Interoperability
One can purchase an Ethernet card for a particular server or client from computer stores, dealers, catalogues, or web sites. This not only drives down the cost, but normally, one can assume that the Ethernet HBA that you buy, will interoperate with the rest of the network.
Not so with Fibre Channel. Every component of a SAN must be rigorously tested and verified prior to its deployment. One must be very careful with firmware, hardware, and software version when deploying a Fibre Channel network.
Summary
Ethernet is and will continue to be the transport of choice for most client/server applications. The infrastructure is in place and the price is low.
However, because of the ability of Fibre Channel to carry SCSI and FICON protocols and the explosion in the amount of servers and data storage, Fibre Channel is the technology of choice for SAN’s. The FC product cost will continue to drop, Fibre Channel IP drivers will be developed, and Fibre Channel cabling plants will be installed. At some point in the future, because of greater reliability, distance, and speed of Fibre Channel over Switched Ethernet, Fibre Channel will challenge Ethernet for the network client/server applications.
bibliography
Oppenheimer, Priscilla. Top-Down Network Design. Macmillan Technical Publishing, 1999
Benner, Alan F. Fibre Channel - Gigabit Communications and I/O for Computer Networks. McGraw-Hill, 1996 |
