Server I/O 2000: Tsunami Alert
If the IfiniBand Trade Association (IBTA) is not making history by bringing together for the first time Server, Storage, and Networking companies to hammer out the specifications for a new I/O architecture, they certainly fielded a darn good show down in Monterey. Over 100 companies sent 300 representatives to get a leg up on the future of computing that will see the death of Client/Server computing, bandwidth becoming 'free', 100GB Ethernet, cell phone PCs, a Global Grid Network, asset-vaulting SANs, and hosts of far-reaching e-technology ramifications that will change the world we live, work, invest, and spell in, forever.
Well, that's the idea, anyway. The Vision looked pretty convincing in PowerPoint, but nobody is laboring under the illusion that mere drag and drop graphics are going to make the future appear with a click. Server, Storage, and Networking Engineers from the heavy hitters have banded together in key work groups to sweat out the pivotal InfiniBand (IB) spec. The initial IB specification was completed last Q, and the Upgrade Spec will be RFC this Q, with the complete release slated for next Q. Considering that the NGIO/FIO wedding was back in August, they must have had a pretty smoochy honeymoon to be expecting late spring. I think they want this baby badly, like yesterday, and for good reasons.
Old MacDonald had a server farm, I/O, I/O, I/O
Let's face it, 10,000 servers, 1,000 switches, 500 SANs, and 15,000,000 daily hits on a host farm will deposit a huge pile of I/O. CPUs need to be doing much more important stuff (like computing) than shoveling I/O and stacking bales of protocols. The I2O specification has been successful in off-loading peripheral I/O from the CPU, and the VIA (Virtual Interface Architecture) clustering specification is handy at steering herds of CPUs, but...
What if with the IB spec we could then
!) Shrink servers to smaller rack form factors and cram them with CPU clusters
2) House banks of battery-backed nonvolatile memory fronting servers and storage
3) Distance storage and peripherals from the servers
4) Switch the fabrics in a chip
5) Embed IP stacks at the edges
6) Not abandon PCI/PCI-X
7) Manage it all somehow
It's not quite sub-space communications, but one could go through the alphabet a hundred times over to list all the acronyms the new IB architecture would aid and rescue. The AOL-TWX-SUNW grouping is one that immediately comes to mind, not to leave out ISP-ASP-CSP-SSP-ESP etc. You get the picture.
What about Fibre Channel?
That turned out to be quite a fractious question during the Masterminds Panel discussion period. Reading the recent quarterlies from QLGC and EMLX, one would think the Fibre Channel market is beginning to rocket. The count down is proceeding in the shadow of a thunderhead that is threatening the blast-off. The lack of FC switch interoperability between Brocade and Ancor was a frequent bone the Distinguished Panelists gnawed on time and again. Echoing Piper Jaffray's Ashok Kumar, concerns that the competetive posturing of the switch vendors could seriously impair the development of the Fibre Channel market were voiced. The work group leaders vowed to learn from the Fibre Channel's industry's mistakes in formulating the IB spec.
The main problem with the Fibre Channel spec lies within its history. After the ANSII standard was set, FC existed in IT 'islands' that never or rarely bridged over to other islands. This fostered a preponderance of idiosyncratic standards implementations. The arrival of Open Systems and heterogeneous networks to the FC world demands tighter specs or tighter compliance. The advent of IB will require the FC specs to be up to par in implementation across all vendors. If Ancor and Brocade can't play doubles, maybe the ball will be tossed into the FCIA's and IEETF's court. Apparently, neither company views switch interoperability as being critical to sales right now, or maybe ever?
Fibre Channel interconnects will exist in the IB architecture as a subnet transport, primarily as Storage Area Networks and some Cluster clustering. IB is not yet a distance protocol (250 meters, with a 10k spec in the works) so the multi-protocol properties of FC should keep it alive and kicking for awhile yet and may just let it survive and evolve. Justin Rattner of Intel gave FC five years, the others thought FC should have a healthy market for the next ten years. By then FC could be another lingering legacy from the last century, or a horse of a different stripe.
The Cisco Kid
Andy Bechtolsheim from Cisco turned a few heads when his presentation depicted embedded IP and 100Gb Ethernet as an interconnect for IB and Storage Area Networks. I wonder if he and George Gilder are drinking buddies. Justin Rattner of Intel replied that 100Gb and IP was fine at the edge and beyond, but they had no place as an IB transport. You see, I20, VIA, and IB are messaging protocols (if you know what that means, you're a better man than I) while ethernet and IP are communication protocols so there are too many hoops in encapsulation, I think. There are still a few wrinkles to be ironed out yet.
Gory Details, NOT
The IB tutorials were meant to update the attending engineering crowd and were replete with terms like Verbs, Queued Pairs, Atomics, LIDs, Virtual Lanes, and Flow Control that must have meant something to them but went through one ear and out the other for me. I knew it was important stuff, but most of the time I was thinking about lunch and the raven-haired reporter in the back row, so I'm sorry to disappoint any engineers out there, but this stuff is beyond me without further study.
The following links will take you to more information, and any questions are welcome.
InfiniBand - infinibandta.org
VIA - viarch.org
I2O - i2osig.org
FCIA - fclc.org
Ready for some football?
Douglas |
