Lycos uses 8i -
zdnet.com
----------
Lycos slices data to smooth operations
Web portal provider engineers replacement for its flat-file system with Oracle8i as main ingredient
By Mark Hammond, PC Week Online
May 24, 1999 9:00 AM ET
The chefs at Lycos Inc., in cooking up billions of bytes of Web data and dishing them out to hungry consumers, are using an unwieldy 100-pound sack of flour for the main ingredient.
That's how a top Lycos executive characterizes the Web portal provider's reliance on the ever-growing collection of binary flat files that make up its data catalogs. These files allow consumers to, for example, run Web searches, access bulletin boards and see stock quotes. However, Lycos officials are having trouble managing the data they need to identify Web-surfing trends that would enable them to offer more services to consumers.
"It's very cumbersome, it's very difficult to manage, and it's failure-prone," said Ron Rainville, director of operations for the Waltham, Mass., company, referring to the binary flat-file system. "When you're trying to distribute and maneuver and maintain data and grow statistics and measurements on 200GB of flat files, it's next to impossible".
The only way for the company to get a handle on all that information was to break up the data into smaller, more manageable portions. But first, it added a secret ingredient to the mix: Oracle Corp.'s new database, Oracle8i.
The data management structure inherent in Oracle8i will enable Lycos to zero in with qualitative analysis on what's hot and what's not among Web surfers and searchers, Rainville said. Also with Oracle8i, Lycos can replace its existing overnight batch processing of data distributed to flat files with real-time updates, which means that data will stay fresh. Lycos officials said the company will ease into using the database to serve up Web content.
Smaller portions are healthy
Lycos is currently testing Oracle8i on Compaq Computer Corp. AlphaServer 8400 systems with Digital Unix; the company also plans to run the database on Intel Corp. chip-based servers running Linux. Over the next two months, Lycos will put Oracle8i into production for its smaller catalogs, such as message boards, which store up to 5 million records, as well as for news/media, membership and clubs.
The Oracle8i data management structure would minimize the burden on the Lycos staffers responsible for building flat-file Web catalogs.
"We've developed our own tools and management processes, and it dawned on us we were essentially rewriting Oracle," Rainville said. Oracle8i is expected to streamline the development process and help Lycos gain a clear understanding of its customers' needs. "We need to measure what people are looking at and what their interests are and go out and collect more of that information. Right now, we have 100 million URLs or thereabouts, and we're not really monitoring the results of what they're clicking on," he said.
The question is whether Oracle8i, handling smaller data stores and eventually the 200GB main catalog, will be able to match the speedy performance that Lycos has achieved with its home-grown flat-file system. "It's still to be determined because we haven't run it in production to see what really happens," said Delana Sandy, Lycos' manager of production data services. "That's the idea behind using the message boards as a guinea pig--if all goes well with that, we'll try other small catalogs and work our way up".
An earlier attempt to serve data directly from Oracle8i's predecessor, Oracle 8.05, fell short of the performance of flat files, which are cached into memory and are thus faster than trying to pull data from a disk-based system, Sandy said. "With 8.05, we saw decent performance at a certain level, but it didn't scale all that well," she said. "When it hit about 10 million URLs inside the database, that was the maximum".
Enhancements to Oracle8i, which was released in March, could give Lycos the performance boost it needs because the database automatically distributes loads across partitions and organizes partition tables by index. Another way to turbocharge performance may be to equip Oracle8i with a main-memory database as a front end so that data is retrieved from memory rather than from disk. Lycos is exploring main-memory databases from a pair of Silicon Valley developers, Angara Database Systems Inc. and TimesTen Performance Software Inc., said Bill Cohen, a Lycos engineering manager.
Separate from the Web system, Lycos has another job for Oracle8i. Within two months, the database will host Lycos advertising information.
The company is currently using Oracle 8.05 on Sun Microsystems Inc. servers to house advertising information that is accessed by advertisers over an extranet, but the system has suffered in performance as Lycos has grown. A crush of advertisers running simultaneous reports will slow down the system, Sandy said. More speed is expected through another capability in Oracle8i, materialized views, which improves performance by serving as "containers" for precomputed data aggregations.
Lycos would like to standardize on Oracle8i as its advertising database as well as a way to manage its Web catalogs. But before that can happen, the company must make sure it is fast and stable.
Lycos officials hope that with Oracle8i, they can find the right mix of ingredients to turn the flour into bread for its Web business. "We're looking at Oracle as an overall product enhancement to our existing technology," Rainville said. "It'll open up another window for the user on the Internet for us and make sure that every bit of data we store is making us money".
Case File
Company: Lycos Inc.
Location: Waltham, Mass.
The need: Lycos required better management and analysis of data generated by Web traffic and searches. The company also needed to improve performance of a data store accessed by its advertisers over an extranet.
The solution: The company will gradually move its Web catalogs from flat files into Oracle8i databases to improve data manageability and freshness. It is also upgrading its advertising system from Oracle 8.05 to Oracle8i to improve performance.
What's next: Successful implementation of smaller catalogs in Oracle8i databases will pave the way to migrate the main catalog from flat files to a database.
Toolbox
Oracle8i to be deployed on a mix of Compaq 8400 AlphaServers running Digital Unix and Intel boxes running Linux; advertising system on Oracle8/8i; Sun UltraEnterprise servers running Solaris. |
