Power-Hungry Computers Put Data Centers in Bind
____________________________________________________________
Newer Hardware Guzzles Electricity and Spews Heat, Requiring Costly Alterations
By DON CLARK
Staff Reporter of THE WALL STREET JOURNAL
November 14, 2005; Page A1
The University at Buffalo installed a $2.3 million Dell Inc. supercomputer last summer, hoping to bolster its image as a research institution. Instead, the big machine came to symbolize an increasingly vexing problem for data centers world-wide.
Once the machine was delivered, university officials discovered they had only enough electrical power to switch on two-thirds of the system. They have temporarily responded by throttling back use of an older supercomputer, but a $20,000 electrical-system upgrade will be needed to run both systems at full capacity.
"The calculations that were done fell slightly short," says Bruce Holm, a senior vice provost at the school, which is part of the State University of New York. "The bottom line was that they missed."
More such misses are likely. That's because, in its long-running race to boost performance, the computer industry has hit a major hurdle: The newest hardware -- particularly the servers that run most business programs and Web sites -- draws too much electricity and generates too much heat.
The power-hungry machines, along with rising energy prices, are generating enormous utility bills and forcing changes on Silicon Valley technology suppliers that are akin to Detroit's struggle to improve gas mileage. (See related article.) Though more-energy-efficient computers are on the way, it could be years before companies replace the systems they have already purchased.
In the meantime, bringing in more electricity and cooling is expensive and difficult in some data-center buildings. Organizations face unpleasant choices that include building new facilities, putting off server purchases or leaving costly space in computer rooms unoccupied to avoid overwhelming their air-conditioning systems.
Facilities planners at the University at Buffalo, for example, originally erred because they thought an older supercomputer would no longer be needed by the time their new machine arrived, Mr. Holm says. The need for both systems caused the university to consider spending as much as $150,000 to upgrade the current data center's air conditioning, just as the university was on the verge of moving the systems to a more modern computer room. "It's that kind of juggling act," Mr. Holm says.
If planners miscalculate, servers overheat, damaging circuitry or causing shutdowns that disrupt operations. The Uptime Institute, an organization that represents data-center managers, predicts that power-related problems this year will cause four of the 20 major failures typically experienced by members annually, up from two of 20 last year. "We are headed into a territory where there is no precedent," says Kenneth Brill, the group's executive director.
For years, no one worried much about power consumption. Chip makers relentlessly shrank transistors, creating chips that operate at a higher frequency and consequently draw more electricity. Besides those speed increases, measured in gigahertz, power consumption increased because the tiniest new circuitry tended to leak current when switched off, like a faucet that won't turn off all the way.
No one pushed miniaturization harder than Intel Corp., the world's biggest chip maker. During the 1990s, faster chips helped reduce servers from refrigerator-size to models smaller than a pizza box, which companies can stack by the hundreds in racks in large data centers.
Along the way, power consumption for servers surged, approaching 3,800 watts per square foot this year for the most compact systems from 250 watts per square foot in 1992, according to the American Society of Heating Refrigerating and Air-Conditioning Engineers. That's as much as 38 standard light bulbs, or more than half the power required by many homes.
Now, hardware makers have changed course. Intel, under pressure from rival Advanced Micro Devices Inc., is striving to improve performance by using multiple electronic brains on its chips -- each of which operate at a lower gigahertz speed to save energy. Paul Otellini, Intel's chief executive, predicted in August that new chips his company is developing could save $1 billion in energy costs each year for every 100 million units sold.
The new pressures are transforming chip marketing and development plans. AMD, for example, is quick to note that its Opteron chip draws as much as 95 watts, compared with 150 watts for Intel's latest Xeon chips. Sun Microsystems Inc. today is announcing a new chip, code-named Niagara, that has eight processors but draws only about 70 watts.
New, more energy-efficient machines can't come fast enough for Denis Weber, executive director for information technology at Verizon Wireless. The company, a joint venture of Verizon Communications Inc. and Vodafone PLC, was forced to upgrade a data center in Ohio to bring in two megawatts of power, a nearly sevenfold increase from 1998, he says. The monthly power bill for the center rose to $40,000 over the same period from about $10,000.
Rackspace Ltd., a San Antonio service that manages servers for clients, has seen its power needs swell to eight megawatts from three megawatts in the past three years -- sending its monthly utility bill up roughly fivefold to nearly $300,000, says Paul Froutan, vice president of product engineering.
Those who run their own data centers have to do more than write big checks. To provide adequate power for its supercomputer center in Oakland, Calif., the Department of Energy's National Energy Research Scientific Computing Center had to dig up a parking lot and knock a hole in a basement wall to bring in locomotive-sized power supplies and air-conditioning units. Adding more capacity would require digging up the parking lot and knocking the wall down again.
One contributor to power problems is the fact that the people who buy computers often aren't the people who have to manage them. Some 59% of data-center managers say their biggest worry is the purchase of computing equipment without adequate concern for power and cooling, according to a survey completed early this year by market researcher InterUnity Group for AFCOM, a data-center managers association.
To keep servers from overheating, data centers typically have air-conditioning units that push cool air up through holes in floor tiles. That air is sucked in through vents in the front of servers, pushed by fans through the circuitry and out the back of the systems. Ceiling intakes remove the hot air and recirculate it.
Many things can go wrong. Servers at the top of racks get hot faster, and break down more often. Air flow in cooling tiles may get obstructed. If the power goes out, backup batteries and generators aren't always enough to keep the air conditioning running. "If you have any type of cooling failure, you have no time to react," says Gary Light, chief technology officer of Concord Hospital, a Concord, N.H., facility that recently installed advanced cooling gear from American Power Conversion Corp.
Consider the experience of Pomona Valley Medical Center. The hospital east of Los Angeles quickly grew to 70 servers from 30, says Kent Hoyos, its chief information officer. The heat they generated overwhelmed the computer room's air-conditioning system and a backup unit that was pressed into service.
With temperatures in the room averaging 92 degrees, machines began behaving erratically, Mr. Hoyos says. In late 2003, an air-conditioning unit broke down, sending the temperature over 100 degrees. The event caused a temporary shutdown of systems serving the hospital's laboratory, $40,000 in damage to servers and hard drives, and prompted a $500,000 retrofitting of the cooling system using equipment from Emerson Electric Co.'s Liebert unit, Mr. Hoyos says.
Nowhere is cooling a hotter topic than at Lawrence Livermore National Laboratory, where two of the most powerful computers in the world are used to simulate nuclear reactions. One system that uses conventional International Business Machines chips, dubbed ASCI Purple, draws more than seven megawatts of power. The first time it was switched on, the local utility called to ask what happened, lab officials say.
The second machine, dubbed BlueGene/L, is more than twice as fast for some applications. But it was designed with special low-powered IBM chips, so the system requires only 1.5 megawatts. Instead of drawing air from the room through side intakes, cold air is pushed up directly through the machine from the floor.
Even so, the lab's electrical bill for computers has swelled to more than $10 million a year from about $2 million several years ago, says Mike McCoy, its deputy associate director for computation. Without the radical design of the new system, the bill would be "way over $20 million, and that is completely hopeless," he says. |
