Internet & Technology
Monday, November 6, 2000
Information Exploding By The Petabyte
By Brian Deagon
Investor's Business Daily
A recent study puts the computer age into this perspective: More information will be produced in the next three years than has been produced in all time.
Try to digest that. Consider that the study says the amount of all the information in the world today, if stored on floppy diskettes, would create a stack 24 million miles high.
The world produced about 1.5 billion gigabytes of information last year. That’s 250 megabytes for each man, woman and child, says a study from the University of California at Berkeley. That number - which excludes copies - is likely to double each year, the study says.
Whoa, that’s a lot of data, and a lot of . . . responsibility.
"The challenge will be in analyzing and visualizing the data, finding relevant stuff to make the right decision," said Hal Varian, dean of UC-Berkeley’s School of Information Management & Systems. "Our ability to store and communicate information is quickly outpacing our ability to search, retrieve and present it."
Can Technology Keep Pace?
So far, technology has kept pace with the data explosion. Disk storage capacity doubles each year. So does Internet traffic and transmission capacity.
------------------------------------------------------------
But soon there will be a real problem in how to find, organize and present the data, says Paul Borrill, chief technology officer of Veritas Software Corp.
Quoting Nobel Prize economist Herbert Simon, Borrill says the growing wealth of information is creating a poverty of attention. Veritas, a maker of storage-management software, thinks the attention issue comes down to how fast data can be made available.
"What Veritas is doing is looking at this poverty of attention. It’s not storage management (that’s the issue), it’s data availability," he said.
Humans hate to wait. Web users, says Borrill, who’s studied the issue, get bored if information can’t be produced within eight seconds. The faster data come, the more likely that a customer will buy your product.
"Companies that do well will know how to value this scarcity of human attention and not abuse it," Borrill said.
Internet Use Triples
That’s especially true with Internet users. The Internet is growing so fast that analysts can only guesstimate its size. There are 100 million U.S. Internet users, five times the number of three years ago. A report this year said there are more than 1 billion unique Web pages - triple the number of just two years earlier.
As a result, the average U.S. household is spending less time reading newspapers, books and magazines than it did eight years ago. TV viewing has held steady at about four hours daily. But time spent on the Internet is up 2,000% to about 45 hours a year, the Berkeley report says. That figure does not include Internet use at work.
Organization Is The Key
Jim Rothnie, chief technology officer at EMC Corp., says the key to managing the information explosion is data organization. EMC, a maker of high-end storage systems, has focused on developing products that make it easier to automate the collection and flow of data. It also sees the need for systems that do a better job of storing and sharing information for households.
Rothnie predicts households will someday be storing more than 1,000 gigabytes, or one terabyte, of personal information. This includes medical records, financial documents, photographs and video. Much of the data, he says, will be stored at central repositories.
"People won’t want to store a terabyte of information on a PC," he said. "With this huge deluge of data, a lot comes down to manageability. Data that’s scattered everywhere is often wasted and never exploited."
The information age still is in its infancy, Rothnie says. He sees an increase in data storage volume to 10,000 "petabytes" in 2005 from 200 petabytes this year. One petabyte is equal to 1,000 terabytes. One terabyte is equal to 1,000 gigabytes.
By 2005, Rothnie says, the data storage market will grow to $100 billion a year - bigger than the market for servers. The cost of storage will drop to $1 a gigabyte from $10 today. Rothnie says data storage and communication bandwidth will be so plentiful and inexpensive as to seem "free and infinite."
Net Under-Hyped
One reason for the data explosion is the ease of creating documents. People not only have access to huge amounts of data, they can use those data to create gigabytes of data themselves and publish it via the Internet.
Denise Shiffman, marketing vice president of network storage at Sun Microsystems Inc., says there’s no question that the Internet is driving the data explosion.
"The Internet," she said, "actually is under-hyped. It’s mind-boggling to think we could have a billion cell-phones connected to the Net."
The trick to managing the data, she said, is "horizontal scaling," having lots of servers and storage systems that scale (grouped for larger computing power) infinitively to keep pace with demand.
"If I’m a service provider of a data center," she said, "I’m concerned with how my server-storage environment grows with the requests coming in." |
