and another one for the techies.
very nice.
-----
Internet Week
April 12, 1999, Issue: 760
Section: Internet Storage
Trading Gigabytes Online
Trader Online houses one of the Web's largest used vehicle databases. The Auto Trader site had 1.5 million visitors in January 1999. These Web customers performed 5.5 million car searches from 550,000 ads, 99 percent of which are less than three weeks old, and perused some 15 million page views. Storage capacity to run the online operation is at 126 GB and expanding daily. The parent company, Trader Publishing, prints more than four million mazines each week, including Auto Trader and Boat/Yacht Trader.
Joe Fuller, Trader Online's director of operations, keeps the ads fresh and the searches simple. Each day, more than 100,000 ads are deleted and another 100,000-plus are posted. Freelance writer James E. Gaskin recently talked with Fuller about the storage system needed for an e-commerce site. Gaskin is a Dallas-based writer who can be reached at james@gaskin.com.
Q. What was the original Web setup?
A: A single CPU Sun box that held the Web server and the database all in one box.
Q. And the storage for that single Sun box?
A: A couple of 2-GB drives. We still have the box.
Q. But it was nothing special?
A: No, nothing special. I think the database was 100 MB to begin with. It's 2 GB now.
Q. You now have 15 Web servers?
A: Right. Sun Ultra 5s.
Q. Do you use any of the storage in the Web servers?
A: The Web servers each use their own disk to boot, and that's about all they're used for. Boot disk, maintenance files, and system error logs.
Q. How do you share the legacy data and new ad files?
A: Each of our 43 production centers peels off an ASCII copy of the ad text from their Progress database and sends it to the Web server operation at headquarters. In that text is the name of a photo, if it has one. The production centers also send digital versions of the photos to the headquarter Web servers.
Q. When was the last major upgrade, and what do you have now for storage?
A: The big decision we made about upgrading storage was in late 1997 when we decided to add the Network Appliance 520. That coincided with a significant architecture change on our part when we added an extra network segment to better isolate the Web site from the corporate network.
Q. Why the change in network storage?
A: With the 15 Web servers, and the frequency with which we change the up to 2,000 different pages we offer, the maintenance would be a nightmare if we had to take and apply that to 15 Web servers. We store all the source code for all the Web pages in the Network Appliance device. We change it once, and it's automatically propagated among all the Web servers. All the developers, no matter what they are doing, point to the same file system, make their changes in that central place, and they're applied across all the Web servers.
Q. Why Network Appliance?
A: At the time we investigated adding this architecture, we looked many companies and narrowed it to two. We are a Sun house for the most part, and the two options were Sun and Network Appliance. With the built-in RAID 4 and RAID 5 redundancy and built-in network interfaces, Network Appliance was the most cost-effective option.
Q. Were performance differences considered?
A: We evaluated the specifications and determined they were well within the range of what we needed. Many devices were, but Network Appliance was the most cost effective. We narrowed down our selection process until cost per megabyte became the deciding factor. Network Appliance was substantially cheaper.
Q. So you started with a single Network Appliance 520?
A: That's right. And that's what we have now.
Q. How much storage?
A: Something like 126 GB from 14, 9-GB drives using RAID 5.
Q. And the 520 has the software and hardware?
A: It's got it all built in.
Q. All 15 Web servers hit that one 520 box, no problem?
A: That's right.
Q. What are you waiting for your storage system to deliver you can't quite get yet?
A: At this point, we're fine with the architecture and what it's delivering. We're getting to the point where we're going to have to add some more spindles. We're close to maximizing the number of processes it can handle simultaneously. There are no features that we're waiting for. We're kind of a down-and-dirty user. We stress it in terms of the number of processes we do, but it's a pretty basic application that keeps on running. It's not complicated, but it's heavy duty.
Q. What will your storage setup look like next year?
A: I can see us adding additional disk space to the existing device, and we may also need to add an additional network storage device if even after adding drives we're still taxing the number of processes it can handle. First, disk additions to this one and purchase of another one. We just had a meeting where we decided to use the Network Appliance box for online backup, meaning we will expand in the next three to six months.
Q. Any reason to move to an official "SAN" (storage area network) when the standards are set?
A: That's not necessary, because we're happy with the features we have now.
Q. Are you doing caching?
A:: Virtually all our pages are dynamic, and we serve up banners unique to where the person is and what they're looking for. We don't really take advantage of caching. We use CGI processes, and they don't lend themselves to caching.
Q. No static pages, except for company information, right?
A: Right, and even on those, there's a banner at the top of the page that will rotate.
Q. I assume the Web site is doing well for you.
A: Yes, it's going great. The last two years, the print publications have seen healthy growth, double digit percentages in revenue and in circulation. We've also seen significant increases in our viewership and revenue. We're not as important to the company from a revenue standpoint as we are from a promotion and positioning for the future standpoint.
Q. What's do you consider the best part of your storage architecture?
A: From an architecture standpoint, the time savings by not having our developers have to implement changes on multiple servers is tremendous. I can't even imagine how we would operate if we weren't storing the Web pages for those 15 Web servers in one central place. From a technology standpoint, another benefit is that we don't just use it to store the source code, we also use it for backups, archiving, and whatever other storage needs we have. We have a lot of flexibility in that we can be on any server, be it a development server or production server, and access the same file system.
Q. What's the worst part of the current setup?
A: The only thing I fear is that there is a single point of failure, however, the machine has redundant sytems and has redundant cards. We also have a spares kit with all of the important components standing by, so we feel we have covered that risk.
Joe Fuller grew the operation from a single Sun box to 15 Web servers and a NetApp 520. |
