October 29, 2001
NEW ECONOMY
Page by Page History of the Web
By JOHN SCHWARTZ
heb I was a little boy growing up in Galveston, Tex., there was no more amazing place to me than the Rosenberg Library, a haven where a young reader could imagine spending many happy years of discovery. It seemed vast, but also personal: in front of the building, a child could clamber up the side of the seated statue of Henry Rosenberg, the library's patron, and sit in his bronze lap.
Brewster Kahle wants to create within the Internet a library that is as vast as the infinite library of my dreams, and as personal as old Mr. Rosenberg's lap. And at the same time, he is teaching us a parable about what we used to call the new economy.
Mr. Kahle never has thought small.
Having become wealthy through a variety of Internet ventures over the years, Mr. Kahle, in 1996, took up collecting: Web pages — at last count, more than 10 billion of them. His Internet Archive project takes regular snapshots of millions of pages and, until recently, stored them like photos in the attic. Last week, the 41-year-old computer scientist celebrated the archive's fifth anniversary by unveiling the Wayback Machine, a free service that makes those old pages available to anyone who can get to the World Wide Web. (It is at web.archive.org, although it has been straining under the traffic.)
E-Commerce Report: More Web Spending, With a Focus (October 8, 2001)
New Economy: Hurdles to Wireless Priority Access (October 22, 2001)
New Economy: Using Humans as a Computer Model (October 15, 2001)
Get Stock Quotes
Look Up Symbols
Portfolio | Company Research
U.S. Markets | Int. Markets
Mutual Funds | Bank Rates
Commodities & Currencies
Using the pages is as simple as typing an Internet address into a search box and selecting a date. And it's big. The archive computers currently hold some 100 terabytes of data — compared with an estimated 20 terabytes of information in the entire Library of Congress. The archive grows by 10 terabytes a month.
But what is it good for? After all, if the science fiction author Theodore Sturgeon was right when he said that "90 percent of everything is crud," what is the percentage for the blather-prone pages of the Internet? "On the Web, it's probably more," Mr. Kahle admits. But even if you up Sturgeon's estimate to 99 percent, that leaves 1,000 gigabytes of solid gold — and every user's definition of gold will be different.
People searching the Internet, Mr. Kahle said, "have very specific interests," and when they find the topic that interests them, "they want it in extreme detail, like steam locomotives in Georgia in the 1840's." The wonder of the Web, he said, is that it has information about steam locomotives in 1840.
Mr. Kahle introduced his brainchild — named, yes, for the time machine used by the pedantic dog, Mr. Peabody, and his boy, Sherman, from the "Rocky and Bullwinkle" cartoons — with a flourish at the Bancroft Library at the University of California at Berkeley on Wednesday. He demonstrated it with a certified stunner: he pulled up a Web page from the White House Web site from Sept. 10, 1996, with a press release about President Clinton proclaiming the prevention of hijacking and terrorist attacks in the air a priority. Mr. Kahle said he had also used the system to read Web pages created by the Heaven's Gate suicide cult and to find a manual for a computer part that had been taken off of a company's Web site in 1998.
He doesn't want to stop with Web pages. Mr. Kahle (pronounced "Kale") is inviting copyright holders for books, movies, music and more to add their creations to the mix. Ultimately, he hopes to finally deliver the kind of library that the ancients tried to create in Alexandria. "We have the technology to make that huge collection again. More than that, we have the technology to give people access from anywhere in the world."
Mr. Kahle has talked this way before — specifically, when he kicked off one of his Internet companies, Alexa, now a subsidiary of Amazon.com. That company also tried to archive the Net. In fact, its technology was used to build the Internet Archive. But attempts to profit from the venture were unsuccessful, Mr. Kahle admitted; even Amazon (news/quote) has stopped putting money into it. "This year, Amazon doesn't have any spare money to do services like Alexa," he said.
But there is a big difference between having a good idea and being able to make money on it, and the fact that you can't make something pay does not necessarily mean it is dumb.
Building free libraries is a noble effort, Mr. Kahle says, citing the largesse of Andrew Carnegie in improving the nation's literacy through a system of libraries. "People ask, `How are you going to profit from this?' " he said. "We're not. It's a library. It's worth it to spend millions of dollars to build a library that doesn't cost users a penny."
People who make libraries and archives their lives are excited about the new collection. "Isn't it the coolest thing around," said Christopher A. Lee, chairman of the Electronic Records Section of the Society of American Archivists. He suggested that social historians of the future might use the archive to focus on things that today seem mundane or even inane. "A lot of social historians would say a Web site that says, `Here's a picture of me, here's a little about my cat' tells us so many important things about how people were using the Internet at a particular point in time."
The project has spurred a kind of enthusiasm that hasn't been seen in a while in the downhearted tech world. Lawrence Lessig, a Stanford University law professor who seeks to explain the interplay of technology and society, was uncharacteristically ebullient. "My brand is pessimism," he said. "This is not something to be pessimistic about. Brewster is my hero."
Mr. Lessig not only admires the project for the knowledge it will preserve. He said he also thinks it can shift the balance in the debate over copyrights and access to intellectual property like books, music and movies.
Holders of copyrights will eventually drag Mr. Kahle into court, Mr. Lessig predicted. So far the battle over copyright has been fought chiefly by copyright owners and their lawyers on the one side, and college professors and computer technicians on the other. Mr. Lessig says that will change if people use the Wayback Machine. "We finally have a clear and tangible example of what's at stake," he said. "Brewster is defining the public domain." Users will see "how easy and important this technology would be in keeping us sane and honest about where we've been and where we're going."
Mr. Lessig has a new book coming out on copyright this month, but he says he would have rewritten the whole thing if he had seen the Wayback Machine first.
Mr. Kahle said he is pleased that his new project is getting attention. "You probably couldn't get an article written about a library in 1999; in 1999, all of the stories were about how to sell dog food over the Net."
For myself, I can't help but think that old Mr. Rosenberg would be pleased.
Home | Back to Technology | Search | Help Back to Top |
