Silicon Investor (SI) -- The First Internet Community

STOCKTALK

We've detected that you're using an ad content blocking browser plug-in or feature. Ads provide a critical source of revenue to the continued operation of Silicon Investor. We ask that you disable ad blocking while on Silicon Investor in the best interests of our community. If you are not using an ad blocker but are still receiving this message, make sure your browser's tracking protection is set to the 'standard' level.

Technology Stocks : From Here to InfiniBand -- Ignore unavailable to you. Want to Upgrade?

To: J Fieb who wrote (14)	10/26/2000 1:59:47 AM
From: higashii	Read Replies (1) \| Respond to of 57

Try this article for some background reading J. It is more technical than most and might help answer questions about signal loss and why analog engineers are required. The advantage in IB development would go to those companies already well stoked. glenn eoenabled.com Advanced I/O still comes up short By Deepal Mehta, Business Unit Manager, Computer Servers, William Lau, Senior Manager, CoreWare Development & Applications, Internet Computing Division, LSI Logic, Milpitas, Calif. EE Times (05/22/00, 1:00 p.m. EST) The Internet and burgeoning e-commerce applications are placing major demands on server I/O performance. The PCI and PCI-X buses are expected to meet today's I/O requirements. Infiniband, which is next-generation I/O technology, promises higher bandwidth and scalability to meet emerging I/O needs. It also offers better reliability and quality-of-service (QoS) capabilities. Yet even this advanced I/O technology comes with its own set of system design issues and challenges. Against this backdrop, there's also the problem that I/O performance isn't keeping pace with CPU performance. Server OEMs are focusing on I/O and memory subsystem designs to differentiate their server platforms. Consider that designers still face signal-integrity and timing issues designing PCI I/O at the 66-MHz level. The PCI-X specification permits the I/O design to move to a maximum of 133-MHz, 64-bit performance. That is double the bandwidth of the 66-MHz, 64-bit PCI bus, resulting in 1-Gbyte/second performance. Still, issues involving the areas of timing performance and signal integrity continue to arise. Here, the I/O interface is pushed to the limit. For example, the clock to Q output propagation delay is 6 nanoseconds. The PCI-X spec calls for a 3.8-ns path delay with basically the same logic element. Setup time is 3 ns for the PCI 66 and only 1.2 ns for PCI-X. Also, there is a 7.5-ns cycle time between input and output registers for the core logic to propagate. That core can easily have 15 to 20 levels of logic. So core logic timing is more relaxed due to the PCI-X protocol. But the I/O is much tighter. The PCI-X 3.8-ns clock to Q output delay means the PCI clock comes into the ASIC device through the input receiver, to a phase-locked loop (PLL), clock driver, on to the clock input of the output latch. That output flip-flop then goes through a JTAG mux and finally through a PCI-X driver. This path not only includes the delay of each element, but it also includes PLL jitter, PLL offset and clock skew. All these must be achieved within 3.8 ns to meet the PCI-X spec. Critical timing Setup time is the other critical timing area. Data enters the ASIC, goes through the input receiver, through an input JTAG mux and on to the input register for clock arrival setup. These data transactions must be achieved in only 1.2 ns, and must also account for PLL jitter and clock skew. Perhaps the most critical issue is the design of the PCI-X I/O circuit's input/output receiver path. The driver definitely cannot be too slow. Otherwise, the designer cannot meet timing closure. But the tricky part is that it cannot be too fast either, because in a high-speed system the designer must pay close attention to signal integrity and noise problems. The PCI-X bus by design is a noisy bus since the signaling is based on the reflection of the transmission line wave. Plus, it is a long unterminated bus. When designing the driver, the design engineer must consider the signal integrity and overall timing budget of the system, as well as the ASIC. For instance, let's consider the ASIC design and the driver part of the delay path. If the driver operations consume 3 ns, only 0.8 ns remain for the receiver. So the burning issue is how do you budget the delay for the driver and for the receiver? The design engineer has to consider that issue carefully. On the other hand, the designer can make the buffer fast by reducing the propagation delay. However, lower propagation delay typically results in a higher slew rate, which makes the buffer noisy. The designer needs a balance between propagation delay and slew rate. Signal-integrity issues, therefore, must be taken into consideration to make sure the buffer falls within the PCI and PCI-X specs and not be too noisy. At times, a buffer design can fully meet the PCI-X spec, but can still be very noisy. It can still have high current due to the pre-driver stages. In that regard, it is important for the designer to have sufficient previous PCI design experience to execute a well-balanced circuit. Infiniband uses point-to-point serial interfaces. That approach provides several benefits at the physical layer over parallel multidrop technologies that have been used in I/O buses traditionally (e.g., PCI and PCI-X): In a point-to-point signaling environment there is only one driver (at one end of the wire) and one receiver (at the other end of the wire). This results in a clean transmission path where signals may propagate down the wire at high speeds with minimal distortion and degradation. Multidrop buses, on the other hand, have multiple drivers and receivers hanging off the same wire at different points along the wire. Each driver and receiver introduces transmission-path imperfections (such as parasitic inductance and capacitance). Those imperfections result in signal reflections, distortion and attenuation. Consequently, the maximum speed at which you can drive a signal in a multidrop environment is lower than in a point-to-point environment. Infiniband employs serial connections with embedded clocks and lane de-skew (for x4 and x12 Infiniband) on the receive side. With this architecture, you don't need to worry about clock-to-data skew because the clock is actually embedded in the data. With lane de-skew you don't need to worry about skew between data lines (as long as you keep that skew within what is specified in the Infiniband standard) because any skew that has been introduced is eliminated on the receive side through realignment techniques. These two features let you go longer distances because you don't have to carefully manage the skew between clock and data, and between data lines. Server designs From another front, the designer must factor in the effects next-generation process technology will have on server I/O designs. PCI and PCI-X are parallel buses that require a number of I/O pins. The PCI-X device requires 64 data and address pins and another 16 command pins for 1-Gbyte/second bandwidth. As semiconductor technology moves to 0.18 micron and below, the number of pins required by PCI-X may make the design pad limited. This limits the die-size reduction achievable through next-generation process technology. The Infiniband standard, on the other hand, is a serial technology and thus offers considerably more bandwidth with a lower number of pins. Essentially, Infiniband is a switch fabric-based architecture. It decouples the I/O subsystem from memory by using channel-based point-to-point connections rather than a shared bus, load and store configuration. The interconnect uses a 2.5-Gbit/s wire-speed connection with 1, 4 or 12 wire link widths. One wire provides theoretical bandwidth of 2.5 Gbits/s, four wires provide 10 Gbits/s and 12 wires provide 30 Gbits/s bandwidth. Hence, Infiniband provides system designers scalable performance through multiline connections, as well as a host of interoperable link bandwidths. In an Infiniband server node, there is an Infiniband I/O bridge rather than the PCI I/O bridge. Also known as an Infiniband host-channel adapter, the Infiniband bridge generates Infiniband links, which are then connected to the Infiniband network, which may consist of an inter-processor communication (IPC) network, or storage area network (SAN) or local-area/wide-area network (LAN/WAN) (Fig. 1). Various I/O subsystems like Ethernet, Fibre Channel, SCSI or even interprocessor communication communicate through the Infiniband network switch fabric. Migrating a PCI-X server I/O design to Infiniband involves several considerations. A key decision is whether to embed the serial transceiver in the ASIC or keep it external as a separate PHY. Of prime importance is a power analysis of the device since an embedded transceiver adds power consumption. On the other hand, embedding the serial transceiver in the ASIC requires fewer pins, which may lead to smaller die-size and package option and hence lower cost. So here the designer is dealing with cost/power trade-offs. The designer also has to evaluate power consumption in terms of packaging and carefully determine the type of package best suited to the transceiver, whether it be a standalone or embedded device. Another consideration is the number of wire implementations the design requires. Is it 1, 4 or 12? A big part of this consideration comes from matching bandwidth with memory bandwidth on the server node, as well as with the performance the designer is attempting to achieve. Interoperability is key to any new technology. Infiniband is no exception and poses certain challenges to the designer to make sure the server I/O Infiniband subsystem interoperates with other subsystems in the enterprise. End-to-end interoperability from host-channel adapter to storage or networking element through the Infiniband fabric is very important. It is also important that the selected operating system support Infiniband. Infiniband is still in its infancy as a new technology, and the spec is still evolving. So, rather than hardwiring a device, the designer may choose to have an embedded microcontroller within the chip to gain extra programmable intelligence. Spec evolves Otherwise, with a state machine implementation, the system designer must keep changing it as the spec evolves. But with the programmability a MIPS or ARM microC core provides, a designer can be more flexible with implementing the Infiniband protocol. Embedded intelligence in the target devices also provides the traditional function of minimizing the interrupts, reducing bus traffic and offering a better error-recovery mechanism. Signal integrity is another area the designer must evaluate. Since Infiniband touts a 2.5-Gbit/s wire speed, signal integrity is paramount. Serializer/deserializer (Serdes) that operate at these speeds often use expensive bipolar, GaAs or BiCMOS process technologies. Few chipmakers offer Serdes using standard CMOS process technology, and even fewer can embed this Infiniband technology in a large ASIC and do it cleanly with good signal integrity. In addition, having a robust receiver that can receive signals without errors that have been attenuated or distorted is important. Some design camps want to completely switch to Infiniband. Other OEMs don't believe PCI-X will go away soon and believe that PCI-X and Infiniband will co-exist until the end of this decade. Hence, there will be system engineers who will opt for a hybrid system that supports both PCI and Infiniband. This calls for a system architecture that supports PCI-X for legacy functionality, as well as Infiniband. See related chart img.cmpnet.com

To: J Fieb who wrote (14)	10/26/2000 2:21:13 AM
From: higashii	Read Replies (1) \| Respond to of 57

Here is an article on InfiniBand switching's central role. Some more details, but not as techie. InfiniBand scales as a network switch By Rob Davis, Vice President, Advanced Engineering, QLogic Corp., Aliso Viejo, Calif. EE Times (09/18/00, 12:38 p.m. EST) eetimes.com InfiniBand is a new interconnect architecture designed to significantly boost data transfers between servers and peripherals. InfiniBand abandons the shared-bus concepts of the PCI bus, shifting I/O control from processors to a channel-based, switched-fabric, point-to-point, full-duplex interconnect architecture. This shift to intelligent I/O engines provides a scalable solution that promises new levels of performance, scalability, and reliability in networking and attached storage. To explore the current state of bus-based system architecture, the place to start is PCI. The conventional 32-bit, 33-MHz PCI local bus has become ubiquitous in servers and desktop computers for I/O devices. There are several reasons for this popularity, including PCI's processor independence, low-pin-count interface, and scalability up to 64-bit I/O performance. The technology has also benefited from evolutionary improvements. Despite PCI's evolution, all PCI-based architectures still force devices to share the total available bandwidth. While this approach was acceptable in nearly all environments when PCI was introduced years ago, there is an increasing number of distributed applications where a technology that can scale without impacting performance would be more appropriate, such as in e-commerce applications running in server cluster environments. Pushed to the limit Moreover, PCI is also subject to signal-integrity and timing constraints that push the I/O interface to the limit. In some environments, these constraints can make it challenging to consistently deliver all data bits on a bus to a destination at the same time. Widening the bus would not help, because that would increase the likelihood that data bits will not be synchronized with each other, making design more difficult and devices more expensive. Multivendor interoperability InfiniBand, backed by the Infiniband Trade Association (IBTA), is designed from the ground up with an eye to the future. Boasting a well-layered, standardized architecture, InfiniBand supports multivendor interoperability from day one. This approach enables the architecture to evolve at the rate of technology. InfiniBand supports a 2.5-Gbit/s wire-speed connection, with one (1X), four (4X), or 12 (12X) wire-link widths. Data throughput ranges from 2.5 Gbits/s per link (one link) to 30 Gbits/s per link (12 links), resulting in lower latency and easier and faster sharing of data. By comparison, PCI's maximum speed is 1 Gbit/s across all PCI slots. Even high-end PC servers with 64-bit, 66-MHz buses can only reach 4 Gbits/s of shared bandwidth. InfiniBand also increases the amount of usable bandwidth. Data is sent serially using fiber optic and copper cables. To minimize data errors, data is encoded with redundant information. In addition, the four-wire link provides four signaling elements in parallel for 10 billion bits/s, while the 12-wire link delivers 30 billion bits/s. Each InfiniBand link is full duplex. Further enhancing performance, InfiniBand employs a switched fabric that creates a direct, high-speed, virtual, dedicated channel between a server, other servers, and I/O units. InfiniBand addresses scalability in several ways. First, InfiniBand offers scalable performance through multiple wire-link width. Second, the I/O fabric itself is designed to scale without experiencing the latencies of shared-bus I/O architectures as workload increases. This is explained in more detail below. Third, InfiniBand's physical modularity obviates the need for customers to buy excess capacity up front in anticipation of future growth; instead, customers can buy what they need now and add capacity as their requirements grow, without impacting existing operations. InfiniBand's new form factor makes it easier to add, remove, and upgrade than today's shared-bus I/O cards. Instead of installing and removing cards from a bus, InfiniBand provides a single interface for multiple types of I/O. In this way, InfiniBand effectively takes I/O expansion outside the box, eliminating slot limits and bandwidth limitations, while freeing the CPU to work on applications. Redundant paths On the reliability front, InfiniBand creates multiple redundant paths between nodes, reducing the hardware that needs to be purchased. It also sheds the load-and-store-based communications methods used by shared local bus I/O to a more reliable message-passing paradigm. At the heart of the InfiniBand architecture is a bi-directional link with dedicated physical lanes sending and receiving data simultaneously. InfiniBand increases bandwidth by boosting the number of physical lanes per link-the InfiniBand specification calls for two, eight and 24 physical lanes. Half of the physical lanes in each link send data, while the other half receive data. Each physical lane has a theoretical signal rate of 2.5 Gbits/s. Therefore, two-, eight- and 24-channel bidirectional physical lanes have theoretical bandwidths of 5, 20 and 60 Gbits/s, respectively. InfiniBand defines four types of devices: host channel adapters (HCA), target channel adapters (TCA), switches and routers. The HCA is installed in the server and connects to one or more switches. I/O devices connect to the switches through a TCA, thereby creating a subnet with up to 64,000 nodes. The HCA can communicate with one or more TCAs either directly or through one or more switches. A router interconnects several subnets. Together, these components transform the system bus into a universal, dynamically configured, scalable interconnection mechanism that enables computers, peripherals and networks to work in concert to form one enormous, hybrid system. While the InfiniBand specification allows a TCA to connect directly to an HCA via a serial link, the real power of InfiniBand comes from having a switch between the TCA and HCA. Connecting a TCA to an HCA through a switch enables devices on an InfiniBand network to be connected to several hosts. Switches interconnect multiple links by passing data packets between ports within a subnet of up to 64,000 nodes. Similar to Internet Protocol, each packet has address and error-detection data, along with a payload. Each time a switch receives a packet, it reads the destination address at the beginning of the packet, and sends the packet to an outgoing port based on an internal table. One or more switches can link multiple HCAs with multiple TCAs to provide high availability, higher aggregate bandwidth, load balancing, or data copying to a backup storage site. All InfiniBand devices are thus connected in a fabric. The idea behind InfiniBand is to create a switched-fabric, serial point-to-point link I/O architecture that meets the requirements for cost-effective I/O and expands and simplifies connectivity between devices, while improving reliability, scalability, and performance. InfiniBand achieves this goal by using a unified fabric to connect elements of computer systems. Multiple stages A switched fabric is simply an interconnection architecture that uses multiple stages of switches to route transactions between an initiator and a target. Each connection is a point-to-point link. This inherently provides better electrical characteristics by allowing higher frequencies, while delivering greater throughput than bus architectures. The use of multistage switch architectures maximizes the flexibility and scalability of the interconnect. The concept of switched-fabric, point-to-point interconnects is not a new one. The latest example of such interconnects can be found in Fibre Channel-based storage-area networks (SANs), in which shared-bandwidth hubs never achieved broad acceptance and switches quickly became the norm. The concept of the InfiniBand switch fabric thus draws on other mature and proven technologies and network architectures, utilizing the collective knowledge of switched-fabric implementations to deliver the best and most cost-effective I/O solutions. The InfiniBand switched fabric brings increased bandwidth, and the abilities to aggregate bandwidth as connections are added, easily isolate a fault to a single connection, and upgrade connections and their bandwidth one at a time as new technologies are developed. We also anticipate an opportunity for enhanced integration with Fibre Channel-based SANs. By the time InfiniBand gains momentum in the marketplace, perhaps in 2002, a tremendous installed base of Fibre Channel and SAN products will be in place, particularly in the enterprise-class environments where InfiniBand makes its presence known first. Line-speed bridging between InfiniBand and Fibre Channel will be a critical requirement, and several vendors are actively addressing this challenge, including QLogic. The processing demands of distributed e-commerce applications have led to increased use of clusters. As microprocessors become faster and less expensive, clustering becomes increasingly viable. Clustering presents at least a couple of challenges, however. To work effectively, clustering activities require high-speed interprocessor communications, with virtually no latency. Many clustering applications also require significant amounts of I/O, such as that required for disk or network access. InfiniBand addresses both challenges head on. Multiple processors using HCAs can communicate with each other at up to 24 billion bits/s, and multiple I/O devices using TCAs can support the I/O needs of the cluster. Because the switches pass data on all ports at full bandwidth, all devices can work in concert. Infiniband building blocks are designed from the ground up to support high availability, as demonstrated in the following example. Assume that two servers need access to two different WANs. An HCA at each computer is used to access a TCA for each WAN through a switch. Normally, each server can leverage both WANs, thereby providing twice as much capacity as a single WAN. However, if WAN 1 goes down or is congested, the InfiniBand switch will direct all data flow to WAN 2. Similarly, if server 2 fails, the switch can direct all WAN data to server 1. To eliminate the switch as a single point of failure, an additional switch should be added to this configuration. See chart img.cmpnet.com QLogic has been working closely with the InfiniBand Trade Association to promote and develop InfiniBand. As one of the foremost proponents of InfiniBand, QLogic believes its endorsement will increase the architecture's adoption among industry leaders. Draft spec As InfiniBand evolves, QLogic intends to put it into practice by leveraging our core expertise in high-performance HBAs and switching to bring high-bandwidth solutions to the industry-standard space. The draft specification for InfiniBand is in preparation, and a final release is tentatively scheduled for this year. The industry's first InfiniBand products should start appearing by the first half of 2001, mostly in the form of storage and server connections. The first switches and routers should be introduced between then and the first half of 2002. In addition to QLogic, companies that have announced significant product development efforts have included Agilent Technologies, IBM, Intel, Sun Microsystems and many other major vendors. Some early products will be demonstrated this fall at the Comdex trade show in Las Vegas. QLogic's first InfiniBand-compliant products, including a switch, should also appear during 2001. QLogic demonstrated a prototype switch at the Intel Developer Forum in San Jose, Calif., last month.

To: J Fieb who wrote (14)	10/26/2000 2:53:51 AM
From: higashii	Respond to of 57

Thanks for accepting the invitation J. Having been an appreciative lurker for so long, I wanted to return the favors of many posters like yourself. I don't mind if it gets slow here bc this thread really exists so public idiot savants like Jim Seymour who misinform the investing public on InfiniBand can no longer claim in their articles that "you read it here first." I don't know where he read about it (and Gigabit Ethernet for SANs) first but it wasn't on SI's A Fibre Channel Future thread. His imperfect knowledge wouldn't be so delusive if he had. thestreet.com emailed to Jim - Mr. Seymour, This is to inform you of the InfiniBand thread on Silicon Investor. Subject 37223 Please feel free to avail yourself of a little more ersatz expertise on the subject. Sincerely, Glenn Higashi