To: George Dawson who wrote (22188 ) 5/30/1999 3:04:00 AM From: George Dawson Respond to of 29386
Technical Note on the InfoStor article: I thought I would post some notes from Thomas Ruwart's article entitled "Performance characteristics of large FC-AL loops, part 1." From this month's InfoStor. The structure of the experiment was interesting since it used a Silicon Graphics Origin 2000 with 8 - 195Mhz R10000 processors, 2GB memory, and an SGI HBA connected to 96 (8 sets of 12 disks) Seagate Barracuda 9 Fibre Channel Disk drives. The Seagate drives have a sustainable bandwidth of 11.2 MB/sec to the outer cylinder and 1500 IOS (I/O operations per second). A long loop was configured through 3 Vixel Rapport 1000 hubs that were ported either to loop sections, disk arrays, or the HBA. A Finisar Fibre Channel Analyzer was set up between the hub and the HBA. He described a very long loop of 126 devices separated by 10 km between each devices to show how even if possible would result in dramatically decreased performance of the disks due to speed of light limitations. He showed for example the 1500 IOS, would be reduced to 17.6 IOS due to the extended period of time for a trip around the loop. He went on to examine the following scenarios: 1. Read/Write IOS and bandwidth as a function of devices on the loop at loop lengths of 50m, 7.5km, 15km, and 30km. 2. Read/Write IOS and bandwidth as a function of the request size used for the same loop lengths as in 1. Above. In the above experiments loop size has a significant impact on performance with substantial drops as the loop lengthens. Read bandwidths of 92MB/sec were possible with 8 - 12 disks on a 50m loop. For the longer loops there is a point where additional disks don't add much because the bandwidth drops off to a multiple of the disk bandwidth (e.g. 21.97MB/sec for the 30km loop). This limitation appears to be significant even at a distance of 7.5km. In terms of performance as a function of bandwidth, a size of 256Kbtes appears to be optimal with less throughput below this number and above it for the 50m loop. I think the experiment says something about some of the configurations we are seeing commercially and also for the viability of the switch market. Here are a few thoughts: 1. FCAL was thought to be limiting in terms of shared bandwidth by the number of devices, but the physical parameters of the loop may also be critical as indicated above. 2. The FCAL linking FC drives would appear to operate optimally at the shortest possible lengths. Optimal placement of switches would be at points to optimize connectivity of the disks on short FCAL loops and could be external or in close approximation to the disks. 3. Bandwidth at the disk level would still appear to be the performance-limiting factor, but Ruwart makes the point that caches and buffers in the receiving device are critical to performance. I am assuming that the next article will describe the performance of arrays of greater than 12 disks. There is also a tie in of sorts with Ancor's latest comparison of switches and hubs - see "the hidden cost of hubs":ancor.com George D.