HWP doesn't sound like they are in an FC hurry.
They going to let everyone beat them to market?
SPECIAL REPORT - SEPTEMBER 1999
Graphics' Demands on Mass Storage
The foundation for the next generation of graphics will be storage technology
David Morgenstern
George Lucas's latest film adventures wowed audiences this summer-for the first and last time. No, we haven't forgotten the forthcoming installments in Lucas's latest trilogy-they just won't be shot on film.
According to Hollywood insiders, the next Star Wars episodes will be shot on "digital film" using a new camera developed by Sony Electronics Inc. and Panavision Inc. The films may also be the first all-digital commercial movies: they will be imaged with digital cameras, be edited on a variety of workstation platforms, feature extensive use of 3-D character animation and cinematic effects, and arrive in theaters displayed with next-generation high-definition projectors.
The foundation for this all-digital workflow will be storage. Lots of it. Each digital movie is estimated to total as much as three terabytes; and this total will be needed for each stage of the production workflow. After the "film" is transmitted to theaters via fiber (or high-speed satellite link), each screen of a neighborhood multiplex will also require this capacity.
So it's no surprise that hard-drive manufacturers and RAID vendors are readying a variety of high-performance technologies to address the specific requirements of digital film and other streaming-content applications such as broadcasting and prepress.
Tom Ellsworth
A Shifting Standard
Most innovation in hard-drive mechanisms has been focused on the larger enterprise- and consumer-computing markets. The performance checklist for these applications is for a random workload: speedy access times and the delivery of numerous small files.
However, bandwidth-oriented applications have much different requirements than online transaction processing (OLTP) environments. For example, a digital-video editing suite operator is more concerned with the maximum sustainable data-transfer rate for long sequential data. A streaming application involves few random elements. In addition, the drive aimed at streaming content may include different settings to manage housekeeping tasks, such as head degaussing and recalibration, which can eventually interfere with the reliability of the flow of data.
Of course, all applications gain when drives come fitted with new, faster interfaces, or incorporate platters with greater areal density (which boosts capacity and data-transfer rates by placing more data under the read/write head). However, the storage equation for professional streaming-media production recently shifted, resetting the bar for entry- and mid-level performance.
"The holy grail up to this time has been uncompressed NTSC video running at 27MB per second," said Randy Cohen, Atlas product manager at Quantum Corp. He said the performance of single drives is approaching that rate-good news for so-called digital-video "prosumers," or professional consumers.
"The problem facing the industry is that high-definition television is coming and the data-rate performance factors [then] go up by a factor of ten-on the order of 300MB per second," he said. "That's far and away above any level that anyone will be able to attain with a single drive any time soon."
The trend for television broadcast, digital-film, and multimedia production is to author content with the highest-quality method possible-using uncompressed data-and then down-sample the files via compression to lower-resolution formats and different media-delivery methods. This flexible scheme maintains the quality of the archived content for future uses, such as print publishing, in a higher-resolution generation of DVD, or delivered to pay-per-view customers over the Internet.
Attacking vibration
A leader in the AV storage market is Seagate Technology Inc., due to its pioneering of the 100MB/sec Fibre Channel (FC) interface. Until a year ago, the company was the sole provider of FC drives. The company claims that its Cheeta LP series and Barracuda 50 drives incorporate several technologies aimed at improving performance in high-bandwidth, high-capacity RAID systems.
The 10,000-rpm Cheeta LP is offered in capacities of 9GB, 18GB, and 36GB, and sports a sustained 18MB-28MB/sec data-transfer rate. The high-capacity Barracuda 50 is a 7,200-rpm drive with a sustained throughput from 14.9MB to 25.7MB/sec, the company said.
"The real important differentiator for AV drives will be in a multidrive setup, not the usual two- to four-drive striped set," said Michael Walton, product marketing manager at Seagate. "How well drives play with other drives, the elegance of the SCSI code to reorder commands, cache efficiency, and servo codes-these are the important factors to the user of an AV drive."
According to Walton, as platters become denser and spindle speeds increase past 7,200 rpm, small engineering problems become magnified. "I used to think the issue in storage was size and speed; now I discover that it's mechanical engineering," he said.
He pointed out that the rotational vibration found in 10,000-rpm drives can be tolerated in small RAID systems, but becomes a problem when an array is scaled to a cabinet holding 60 drives. The Seagate drives employ a just-in-time (JIT) seek technology that dampens a "heavy seek," the effect produced when many drives in a cabinet move their actuator arms simultaneously. The vibration in one part of the array can affect nearby drives in the cabinet, causing the head to miss a track, thus slipping a revolution and delaying a data transfer, an unacceptable condition for graphics-intensive applications.
The JIT seek algorithms monitor the commands in the queue, and employ one of four seek profiles that let the drive's actuator arm make the shortest, gentlest, and most power-efficient seek over the platter to find the correct logical block. The result is less vibration, improved power efficiency, and quieter operation, Walton said.
The new drives also use Seagate's Advanced SCSI Architecture 2 technology, which offers an optional Caching Analysis setting. The routines can examine commands in real time and let the buffer dynamically assign cache segment numbers between 2 and 16. Walton said this can alleviate the need for manual settings for AV optimization (see "Old Drives, New Tweaks," p. 55). "This firmware is dedicated to AV," he said.
Take the cache
According to Patricia Wadkins, an engineer at IBM's Rochester, MN Systems Integration Laboratory, "the term 'AV drive' is far too vague to be meaningful," she said, pointing to the widening list of performance specifications for products that are now in the category, including video servers, DV and audio editing, and even set-top boxes (see "DV Recorders").
However, Wadkins and the team from IBM's Hard Disk Drive Firmware Development and Application Marketing group said cache and multiple segmentation are two factors for sustained AV performance in professional editing systems, especially for the latest packages that handle dual streams of uncompressed video. The company's latest mechanisms support up to 8MB of cache.
The drives' Mode Page parameters can be set for 1 to 64 segments, Wadkins said, offering flexibility for different AV marketplaces. An AV server might need many streams, each using more cache segments that are large enough to hold the entire request. Meanwhile, a video-editing application might require only one or two streams.
Showing the cache and segmentation in action, IBM demonstrated a single 10,000-rpm Ultrastar 9LZX, delivering 22 separate 6Mbit MPEG2 video streams, at April's National Association of Broadcasting '99 conference in Las Vegas. The family of drives includes capacities of 9GB, 18GB, and 36.7GB, and a sustained data-transfer rate from 15.2MB/sec to 29.5MB/sec.
Quantum optimizations
Quantum made a foray into the AV arena this summer with ramped-up production of its Atlas 10K and Atlas IV, two new drive families that incorporate technologies to boost performance and reliability for streaming applications.
The Atlas 10K is the company's first mechanism with a spindle speed of 10,000 rpm. It is offered in capacities of 9GB, 18GB, and 36GB. According to the company, the drives offer a sustained data-transfer rate between 18MB/sec and 26MB/sec and an average access time of 5 milliseconds (ms).
The Atlas IV is a 7,200-rpm drive with similar capacities to the 10K. The drive has an average access time of 6.9ms and a sustained data-transfer rate from 12MB/sec to 21MB/sec.
The drives are the first to use the Ultra 160/m SCSI interface, which provides a maximum data-transfer rate of 160MB/sec, about half again the throughput of FC. Quantum said it will offer the drives with a FC-AL interface later in the year.
In addition to a faster interface, the Atlas drives use new technology to mitigate latencies. According to Quantum's Cohen, the company's previous high-end product, the Atlas III line, required workarounds to handle the demands of some broadcast applications. For example, he pointed to the digital broadcast-advertising-insertion systems designed by EVS Broadcast Equipment and Quantel. For this system, if an advertisement is not broadcast perfectly, the advertiser doesn't have to pay-even for a simple missed frame or two.
These vendors discovered that when a drive developed more than ten bad blocks, it couldn't always be relied upon to cleanly insert the advertising data. Defective blocks will crop up over the life of a drive and all hard drives will remap data on a bad block to another place on the drive if possible. This spare can be just another empty block somewhere else on the drive, away from its original position. With streaming content, however, this usual scheme can increase latency by forcing the drive to seek away from the rest of the data-which is enough to concern the system vendor about error-free delivery of that content. To cope with the problem, the systems made their RAID controllers poll the status of bad blocks and automatically send a service call just before trouble could occur.
To improve latency performance, Quantum's new Atlas drives handle defective blocks differently-they use an expanded cache architecture that gives speedy access to grown defects, the company said. When the drive first spins up, it reads a list of defects and places each entry into the cache. The Atlas 10K's cache provides room for 50 blocks.
In addition, the 10K drive uses a new zone-based sparing for the defects. Instead of replacing the data in the next available spot on the drive, the controller puts the data near its original location. Every platter is divided into 24 zones, with each zone containing a track's worth of spare locations (about 250 to 300 blocks). This minimizes the distance the read/write head must seek to find a remapped data block and reduces latency when the grown defects cache is full.
Like other drive manufacturers, Quantum developed software for OEM customers that lets them customize Mode Page settings to boost the performance for multimedia performance. However, Cohen said the new anti-latency technologies remove the need for AV tweaks.
Matching array and application
While manufacturers seek to improve drive mechanisms through a range of new technologies, managers can boost streaming performance by optimizing RAID configurations.
"How you tune and how you architect the array, that's important for bandwidth," said Kevin Liebl, vice president of marketing at MTI Technology Corp. "Often customers migrate to the lowest cost per megabyte, but they may suffer a performance penalty. Often you want more read-write heads."
For example, a striped or RAID-3 array built with 16 or 20 9GB drives will be faster for streaming content than one derived from four 36GB drives. This may be counterintuitive to managers more familiar with transactional-processing applications than large-file streaming tasks.
A careful analysis of tasks may also prove useful to RAID managers when choosing a controller. Some vendors, such as MTI, offer different engines for streaming and OLTP applications.
A case in point is an e-commerce Web server. Instead of placing everything on a single server, managers should consider two systems, each tuned to the appropriate function. The first would handle the financial transactions and pointers to the large-file-content catalog of digital photos, moving pictures, or 3-D images, which would be housed on the second array.
"Today, if you're bandwidth oriented, that's a good move, because there are devices that specialize in each type of I/O," said Liebl. He said this was due to time-to-market considerations by the vendors. "But as the storage market matures, more intelligence will be built into a single RAID controller for both bandwidth and OLTP."
Interface considerations
According to RAID vendors and analysts, there is little performance difference between FC and the latest flavors of parallel SCSI.
Ultra 160/m is an obvious value for standalone workstations with its relatively low cost for host adapter cards (see "UltraSCSI Fights Back," March 1999, p. 44). A system would need a few Ultra2 cards or two FC-AL adapters to gain a similar throughput.
Meanwhile, the SCSI Trade Association last spring announced Ultra4, the next version of the parallel interface standard. Due in 2000, the interface will support maximum data-transfer speeds up to 320MB/sec.
However, proponents of FC point to its performance and workflow advantages. Although the current standard is 100MB/sec (and versions scaling up to 400MB/sec are in the plan), many content-creation sites pack two to four cards in a workstation to get the highest performance posssible with a multiple-channel server solution. The FC standard also supports storage area networks that offer improved management and clustered storage (see "An Introduction To Fibre Channel," March 1999, p. 38).
But industry insiders say the SAN market is still in its infancy. Managers should budget extra time to test the reliability and performance of FC setups.
"The state of the art in [FC] compatibility today is a hell of a lot better than it was, but it's still an enormous problem," said James Wolff, vice president of Hewlett-Packard's Enterprise Storage Business Unit, formerly known as Transoft Networks Inc. He said the company found certain combinations of switches, hubs, and cards work successfully together, while others are better at dealing with legacy SCSI-based storage.
Let a thousand drives stream
Over the next couple of years, drive capacities will continue to rise and so will support for streaming data. Analysts say single drives will provide the sustained data-transfer rate necessary for uncompressed NTSC video.
As manufacturers move to beef up support for professional streaming applications in high-end drives, the vendors will also migrate these technologies into drives aimed at the enterprise market. IT managers are the expected movers, and need the extra capacity and performance to deal with the increasing use of color graphics and multimedia in the enterprise.
-------------------------------------------------------------------------------- |
