To: Greg Hull who wrote (25976 ) 2/3/2000 12:56:00 PM From: Smart_Asset Respond to of 29386
Some light reading on coming attractions.is.pennwellnet.com Article Date: January, 2000 Magazine Volume: 4 Issue: 1 -------------------------------------------------------------------------------- CONNECTIVITY: Prepare for the shift to switched fabrics The road from PCI's shared-bus architecture to InfiniBand switched fabric architecture is through PCI-X. By Thomas Heil Switched fabrics are a revolutionary advance in server architecture. While new to certain markets, this architecture has existed in mainstream enterprise computing for many years. The concepts of switched fabrics are embodied in the current InfiniBand (formerly System I/O) initiative, which reflects the best of the Future I/O (FIO) and Next Generation I/O (NGIO) standards. The proposed standard promises to make systems and I/O attachments simpler, cheaper, more reliable, and easier to expand and service. However, the transition from shared-bus architectures to switched fabrics will take time. PCI is the dominant shared-bus standard and is well established. For switched fabrics to emerge, the industry must allow a graceful transition, implying a potentially long period during which buses and fabrics will coexist. PCI-X-an enhancement to the PCI bus that is expected to emerge next year-ensures the ongoing viability of the PCI bus architecture and provides a non-disruptive path from bus-based systems to fabric-based systems. What is switched fabric I/O? In contrast to shared buses such as PCI, switched fabrics connect processor and I/O nodes via an "any-to-any" network of switches and point-to-point links (see figure). Nodes communicate using intelligent messaging and protected remote DMA (RDMA) techniques, in contrast to bus-based operations in and out of a shared memory. Click here to enlarge image Switched fabric I/O standards Switched fabrics are not new. Proprietary fabrics have existed in mainframe, massively parallel, and Unix midrange markets for years. What is new is the intent to define industry standards that enable the fabric to eventually replace the bus as the I/O interface in high-volume markets such as Intel architecture systems. Today, Fibre Channel (switched fabric and arbitrated loop topologies) is emerging as the standard interconnect for storage area networks (SANs). Ultimately, however, a system area network standard-one that can handle all I/O functions-is needed. Switched fabrics offer the following promises: A "unified" system area network, in which a common switched fabric handles all I/O functions, including cluster inter-processor communication (IPC), network, and storage. A limited distance focus emphasizing "intra-chassis" or "same room" connectivity. (Fibre Channel and LAN/WAN technologies will handle long distance applications.) Incorporation into host chipsets to eventually eliminate PCI slots. Definition of a new "fabric I/O adapter" that plugs into a switched backplane instead of a shared bus. Fabric I/O adapters will be hot-pluggable and externally accessible. Bus-to-fabric transition The transition from bus-only architectures to fabrics will probably be gradual (see figure on next page). The Bus-Only curve represents the transition starting point-i.e., servers with PCI slots only. A Bus-Only server will connect to a fabric with a PCI-SIO host adapter. The Bus-Fabric Hybrids curve represents servers with PCI slots and native (integrated) fabric channels. The Fabric-Only curve represents the transition end-point-i.e., servers with no PCI slots where all I/O occurs over native fabric channels. Bus-Only systems will dominate the early years, while switched fabrics (implemented via PCI adapters) are introduced, debugged, and proven in production environments in all the major operating systems. During this time, a peripheral base and support infrastructure will emerge. Although some early-adopter OEMs may provide native fabric channels sooner, the broader market is typically reluctant to integrate technology that is not thoroughly proven or is not widely available. PCI slots are a low-risk proving ground. Once fabric technology is proven and a reasonable peripheral base exists, OEMs can increasingly ship Bus-Fabric Hybrids. Over a period of several years, OEMs will shift the mix from "mostly PCI" to "mostly fabric" as the fabric peripheral base grows (similar to the way OEMs used PCI-ISA hybrids to move the ISA adapter base to PCI). This process will continue until PCI connectivity is no longer required. Click here to enlarge image PCI-X provides a bridge PCI-X is a significant step in the evolution of PCI. It increases bandwidth with higher frequencies and protocol enhancements as well as scalability with electrical improvements that enable OEMs to provide more high-frequency slots. PCI-X and other initiatives like hot-plug and alternative form factors will significantly extend the useful life of the bus architecture while maintaining legacy compatibility. Switched fabrics are compelling, but it's clear that bus architectures will play a critical role well into the next century, and PCI-X provides an evolutionary path. As switched fabrics emerge, the role of Fibre Channel and SCSI may change somewhat. Today's PCI-SCSI and PCI-FC adapters will gradually give way to fabric adapters. These adapters will enable server vendors to replace PCI slots with native fabric channels, while providing connectivity to Fibre Channel SANs and SCSI or Fibre Channel peripherals. Switched fabric I/O is the architecture of the future, with potential to deliver significant value to end users. It is, however, a fundamental architectural shift that will unfold over many years. Just as important to end users is a non-disruptive path to this new architecture, which implies a period of bus/fabric coexistence. Thomas Heil is a senior systems architect at LSI Logic (www.lsilogic.com), in Milpitas, CA. PRINT SCREEN