Here's the SD Mag's Benchmark you were all waiting for....
Java Five-Oh Shoot Out
September 1998
POET Software's Universal Server 5.0 and Object Design's ObjectStore 5.0 are ODBMSs with strong support for Java. A peek under the hood reveals two very different products that can handle Java objects with amazing aplomb.
by Rob Wehrli
With the explosive growth of the Internet, Java, and electronic commerce, object databases are important tools for managing increasingly complex web data such as images, audio, and video. Java, a natural for multitiered, object-oriented architectures, is quickly becoming a staple in the web developer's diet. Object-oriented database management systems (ODBMSs), responsible for maintaining a given state of an object or collection of objects, play a vital role in managing the multitude of objects in an enterprise-sized web. Two strong performers from leading ODBMS makers, Universal Server from
POET Software and ObjectStore from Object Design Inc., were selected for this review because of their well-known support for Java.
The Tests
I created an online ordering system of some 800,000 products represented by various Product classes of source code files from EMS Software's CD series of C, C++, Visual Basic, Ada, Pascal, and others.
Each file was a Product with filename, size, date, and type attributes and corresponding accessor and mutator methods. Each Product also had a transient Price object in its attribute list, which represented a
variety of prices Customers might pay relative to their particular state. Actual prices were stored in a RDBMS (Microsoft SQL Server 6.5), and when an object was asked to displayPrice(), a call to the
RDBMS was made to assign values to the Price object so I could test legacy database integration.
I used automated scripting to simulate a field of Customers making purchases for several hours. Because each product would likely have an image file associated with it, I opened the jewel case of "Swimsuits"
and proceeded to attach images to each source code Product to complete the simulation. Serving web clients on the LAN may not accurately depict a realistic test environment, but it does tend to load the
ODBMS when keeping up with requests.
I set up 12 machines to constantly request random Products from the database, which I figured is like serving perhaps as many as 300,000 typical modem-sized (less than 120K) requests per hour. Because the web server ran on the same hardware as the ODBMS, the 486 and low-end Pentium boxes slowed substantially at peak load. Under the dual-processor machines, I found the client boxes were peaking the
network long before the server load peaked.
I loosely extrapolated all of this into more meaningful detail and found that segmenting the ODBMS from the web server is probably a good idea. Network speed is critical for moving large images, especially in
local areas. Even if properly segmented and supported by a single Pentium machine, a T1 connection will likely max-out long before the server expires. Web sites centered around the ODBMS will likely need
more horsepower to provide truly dynamic content from a busy site.
Because Netscape Communicator and Microsoft Internet Explorer browser clients were on four different hardware and software platforms, accurate timing between both products was not possible-partly because my script starting procedure was different between platforms and I was jumping around the room to get things going. I did find that each server answered thousands of requests sufficiently. This scenario, left running for several hours, approximated a reasonably busy site-though I suspect you'd need a higher degree of application code and server environment optimization for extremely busy sites such as Yahoo! or Microsoft. I focused on obtaining a reasonable degree of
certainty that I could perform business objectives for as many clients as a typical enterprise might see in a day of Internet traffic without any special optimizations. Both products completed these tests-with
ObjectStore perhaps feeling more sound, though Universal Server didn't skip a beat.
Universal Server
POET Software markets Universal Server to the enterprise as a departmental object server. The package arrived in a nicely printed box with manuals and media. The Microsoft Windows NT Server installation performed flawlessly. I had to do a simple, though manual, editing of plain text files to activate the product as licensed.
Once installed and configured by default to run as a Windows NT service, Universal Server consumed about 1.5MB of memory resources-not bad for such a powerful product. Once an administrator ensures the CLASSPATH variable sees the POET classes, you're up and running. The included documentation is in HTML and Microsoft Word document files in addition to the printed manuals and a few PDF files.
The tutorial and example source code files are straightforward and easily followed. Through trivial examples, the sample code demonstrates implementing the product's major features. I especially liked the "toons" database, which features many of the Warner Bros. cartoon characters as objects. It helped me identify with the POET developers and appreciate their humor.
Tools include the POET Administrator for administering databases and the POET Developer's Workbench for browsing and querying databases. These tools, while simple to use and easy to get your hands around, are surprisingly feature-rich and capable. I especially liked the class and extent browsers.
They gave me a much closer, hands-on look under the hood. Browsing object memory and disk requirements gave me direction in optimizing code for maximum performance. POET Developer's online Windows help works well and complements its PDF and HTML file documentation. One annoying issue in Developer's Object Query Language (OQL) window is that you must use the right-button mouse menu or backspace to delete code. Hitting the DEL key does nothing.
The POET Administrator is an important tool that seems less thought out than Developer. "Normal" administration actions like creating, recovering, and re-indexing databases were a bit unnerving when I
realized the dialog box prompting for the database required me to explicitly reenter the database name-even though it was already entered into a text field. Enabling user administration on a new
database, especially the first time, is another area where the comfort level could be increased through better online help. The dialog prompts the user with, "Are you sure you want to enable user
administration?" Clicking Help doesn't provide any hints. If enabling user administration can be a bad thing that permanently changes a database, I think a bit more online help should be available to answer
the pending question.
I encourage you to read the HTML in favor of the Windows help file, though I think the printed manual is exceptional. It is organized to move someone who understands Java to the ODBMS "how to" finish line. Be prepared to spend about a week using the manual, online documents, sample databases, HTML, and PDF files to become fully intimate with POET. Just be sure your boss isn't hovering over you while you're having so much fun!
Even with its undocumented features and other subtleties, Universal Server is easily used to do real business work. I installed it on a pair of test platforms to exercise its strengths and learn of any run-time weaknesses. I found it ran smoothly for what I'd call enterprise-level requirements on just about any typical hardware. Throwing more hardware at POET seems to help most in databases with many indices, and any time full-text searches are expected. For serving my plentiful, multimedia web objects, even a lowly 486 ran strong.
My first attempts at using Universal Server with Microsoft's Internet Information Server caused considerable grief when it just plain wouldn't run right. The problem had to do with Java class and servlet
compatibility and the Microsoft Java Virtual Machine. I hit Live Software's page (www.livesoftware.com) for a free copy of JRUN, which made all the difference in the world. Universal Server was immediately available to my web servlets. In all of my tests, such as serving multimedia objects and RDBMS-data wrapped objects, it ran strong.
The one test where things seemed to slow down a bit was when I built a Hashtable and stored 1,649 Strings in it. Then I stored 164,900 of those Hashtables in the database. The database ate it all up nicely
without a single skip in performance-in fact, data input went quickly. While the database was only about 64MB in size, the idea that someone would do such a thing with such "expensive" objects is a bit ludicrous. The slowdown didn't occur until I scrolled through the "Extents" using the Developer tool.
Even then, it only paused twice, for a couple seconds each time, until I reached the end of the report.
After that, scrolling the entire extent was smoothly cached. To me, this demonstrates Universal Server's robustness. It can handle even bizarre demands with hardly any slowdown. I tested it on different
hardware combinations to see what it really wanted. On a dual Pentium Pro 200, it smoked! The same 64MB database flew across the screen faster than the eye could see. With the price of hardware today, the ease of use and outright usability of an ODBMS makes good web sense to me.
Universal Server gets three stars for being rigorous and capable. For its invigorating performance I'd rate it higher, but untrapped exceptions are a big no-no in production software, even if data integrity is not violated. This product has the makings of being a four- or even five-star product once POET works out its interface issues.
Another thing I can say about POET's software is that it focuses more on performance and making the database work well than on fixing every last bug in a user interface. The people at POET are committed to their product, and their warm attitudes and professionalism support that feeling from start to finish. I pushed dozens of questions and technical issues at their team, and their responses were prompt. Thus, I give Universal Server another half-star for its talented and motivated support people-bringing the total to 3.5 stars.
At $5,000 for unlimited users, the package is attractively priced and hardy. Its ability to run well on even "performance-challenged" hardware is testament to its design. Given the role of middle-tier, enterprise Internet and intranet requirements where objects and Java often make a good fit, I'd recommend a close look at POET's free download and evaluation period for Universal Server.
I recommend Universal Server for all enterprises but those requiring the most stalwart user interfaces. I especially recommend it for companies that need a budget-minded solution for web integration of
objects. Interestingly, Universal Server's lack of native interoperability with Microsoft IIS and Java Virtual Machine was fixed in a recent patch that suggests to me that many enterprises will now realize its benefit.
ObjectStore
Object Design's ObjectStore 5.0 server is heralded as the market share and performance leader for enterprise-sized object repository needs. It installs easily under Windows NTS/IIS. For a low-end assessment of its capabilities, I initially chose to install it on a P5-66 with 64MB of RAM. The server's footprint and object manager combined was less than 800KB. I immediately noticed how fast everything ran on this low-end Pentium box. The surprise subsided after I noted the hardware requirements call for at least an IBM or Intel PC, but recommend a 486 or Pentium processor.
I love PDF files, and ObjectStore's plentiful amount simply swept me off my feet. I counted 20 in all, including installation PDFs for Digital, HP UNIX, Solaris, AIX, OS/2, and Windows NT. A 16-page
glossary is also included. Of greater interest to anyone considering ObjectStore is a 406-page mo.pdf, which, in case the name threw you, stands for Managing ObjectStore. I'd definitely ask Object Design
for a copy of all its PDFs, including mo. They are high-quality publications that present real information in an electronically friendly way. They weigh the pros and cons of issues and give the reader the information necessary to decide the right approach for a particular enterprise need. The 404-page ObjectStore Java API User's Guide got me moving along rather quickly. Object Design strongly recommends prospective customers attend its classroom training to make the most of a purchase.
ObjectStore uses a post-processor on your Java class files to prepare objects for the database. Its documentation thoroughly explains using this necessary utility to enable persistence in your classes.
Defining data and application classes often requires special persistence classes. For example, an ObjectStore class substitution for java.util.Hashtable is required to avoid troubles such as infinite loops.
Reworking existing classes to make them ready for ObjectStore may impose time and design challenges.
Other potential issues with using ObjectStore are addressed in the Miscellaneous Information section and should be reviewed before investing in the product. Whether or not any of these issues present a
problem in your development process, I feel it's noteworthy that Object Design makes every effort to shed light on the pros and cons of its products. Also included are reasons why you may or may not want to use its proprietary file system. Anyone who has experienced the prevalent "Use this 'cause it's faster and better" marketing phenomena will appreciate Object Design's take on openness and honesty. This level of confidence and commitment is part of what it takes to be enterprise-read. I rate ObjectStore's thorough documents at nearly five stars.
Learning to use Object Design's ObjectStore Inspector utility to browse and edit databases is straightforward. Inspector provides the functionality to view and modify databases wrapped up in a user-friendly interface. The interface is intuitive and simple to navigate. It also appears to have properly trapped every possible exception, as I had no annoying run-time errors or outright crashes. The tool is powerful and has all conceivable database management functionality integrated into it. I especially liked the spreadsheet interface for viewing object members. The database schema pane provides several different overviews of the selected database in Booch, OMT, Coad/Yourdon, and Hierarchical Tree notations. My favorite and probably the preferred notation standard, UML, is missing in that list.
Object Design's intention to present the user with multiple choices for viewing database schema also includes an additional object state view chart. I found using Inspector to browse aggregated class member data such as audio, video, and collections of classes and images exciting and simple. Inspector's "F1" context-sensitive, online help really is helpful. The View Instance Window shows a detailed description of the instance, including data member variables, their values, and instance relations. Set Instance Format lets you define how the instances of the selected class are displayed in Instance Views and View Instance Windows. I commend ObjectStore for its online help, which is devoid of filler and packed with good information.
Something that sets Inspector apart from the crowd is its ability to format the way data is displayed in the views. A toolbar button quickly takes you to a Format dialog that allows such things as managing member data layout, the icon associated with a class, and whether or not memory addresses of classes or members are displayed.
ObjectStore and its related tools embody a pillar of strength in ODBMS fare. Being extremely hardy and fast, the Object Design product line embraces the enterprise. While testing image objects of more than 20MB and literally hundreds of smaller objects of various types, I was unable to crash any component of the product short of tossing the power switch off. ObjectStore ran crisply through its sample applications and databases featuring multimedia pieces that lend well to the "webness" of this package. From installation to deployment, feature-rich ObjectStore with Inspector was molded to fit enterprise needs. The product is meticulously detailed and crafted with no apparent flaws.
Some digging produced some points of interest and perhaps even a couple weaknesses, but nothing that doesn't fit into the scope of process engineering and perfecting the product. I found that using the hot keys CTRL+ "+" and CTRL+ "-" in the schema pane to zoom in and out didn't work properly unless I used the "+" and "-" keys from the numeric keypad. The corresponding number-row keys just sat there,
staring dumbly at me during attempted zoom operations.
ObjectStore's Java support is well-documented and complete. The thoroughness of the supplied material suggests an avant-garde approach that only enterprise muscle can afford. There is nothing mom-and-pop
about this product-from pricing, to support, or even to the time it takes to learn it. ObjectStore implies an investment in technology that goes far beyond "Ain't this cool stuff? Can we have one, please?"
Diving in is as intense as it is rewarding. While all of the ramp-up pieces are in place, I strongly recommend nothing less than a full month of book pounding and firm hands-on experience to come away
with a peek under the hood. Of course, once bitten by the power of this particular bug, you're likely to find yourself thinking of every conceivable and fascinating way to put your new perpetual motion
machine to work. Java samples are simple to follow and implement.
The closest "natural" comparison I can give ObjectStore is that it is to ODBMS as Oracle is to RDBMS. Neither is easily learned in a week or two of even intense training and both are
enterprise-ready and scalable. In my opinion, all of this puts ObjectStore's Java support into a special category. The beauties of Java are that it is easy to learn and use, designed for the Internet, and freely available. This makes using Java with ObjectStore a natural for success in enterprise web applications. It pushes the developer in me to consider the multitude of other uses it offers to those who can step up to the plate with a full team of players. I'll confess there is too much to this product to offer more than a glimpse in any document less than, well, 20 or so PDF files. Object Design requires, or at least strongly recommends, companies entertaining a trip down ObjectStore lane to partake in its educational course offerings. I give ObjectStore four full stars, though my numbers in Table 1 show only a quarter-star difference between it and Universal Server.
Table 1 shows a mixture of objective and subjective categories I used to give ratings to these two products. The numbers give ObjectStore a slight edge over POET in some pertinent areas, but I think that POET's numbers speak for themselves when basic admission price is considered. My quick ratings explanation follow.
In performance, I gave ObjectStore the nod with its own file system (remember RAWFS?) and considerable speed in handling multimedia-though both products were hot performers in my "seat-of-the-pants" acceleration tests. Support means a lot of different things to just about everybody. I rated these characters on how quickly and enthusiastically they addressed problems, issues, areas of interest, customer questions, and just about everything else I could think of to throw at them.
Universal Server gets my vote by half a star for POET's bend-over-backward-to-help efforts. I didn't find any particular area where either product clearly won in the features department, but I'm sure both companies' marketing groups would argue that point. Once you get past the performance and features fluff, database stability is important, especially at the enterprise level. While Universal Server performed strongly and never dropped a bit of data, it didn't feel as stable as ObjectStore. I attest most of this to the user interfaces, but untrapped exceptions from various fronts suggests a full 1.5 point difference is required.
Management concerns such as cost and the learning curve required to staff such a product's maintenance, development, and implementation schedules are big factors in any purchasing decision. POET gets my vote on both these counts, but I'll be the first to admit that these numbers contrast complexity with cost and implementation vs. flexibility and scalability.
Universal Server is easier to pick up and get going. ObjectStore is more robust for a diverse, busy enterprise-level object platform. Universal Server is better suited for mid-sized businesses or enterprise departmental-sized needs. The distinction grays even more when the kaleidoscope of colorful and subjective ratings such as fun, cool, and awesome enters the foray. Universal Server is fun and cool. It is fun and cool in the way Java is fun and cool. ObjectStore is awesome in the way Java is awesome.
Perhaps it may appear a bit bewildering when jumping into reams of documents, but it is at least awe-inspiring nonetheless. Universal Server makes me feel warm and fuzzy like Levi's and T-shirts, while
ObjectStore makes me feel calculating and confident like lab coats and academic achievements of excellence. They both, however, do objects under Java with amazing aplomb. My only wish is that both would interface with Rational Rose for Java. Both products support Rose for C++. My personal favorite is clearly Universal Server, but if my job counted on selecting an ODBMS for a large enterprise web, I think I'd lean toward ObjectStore. I sincerely think each product aims at targeted markets and roles where it shines best. I encourage anyone interested in a web-based ODBMS to research both products
before making a choice.
Category
................Universal Server.......ObjectStore
Performance...........4...................4.5
Support..............3.5..................3
Features..............3...................3
Stability.............3...................4.5
Learning Curve........4...................2.5
Cost..................4...................3.5
Documentation.........4...................4.5
Scalability...........3...................4
User Interfaces.......2.5.................4
Nice People...........4...................3.5
Fun...................3.5.................3
Cool..................3.5.................3
Awesome...............3...................4
Totals.............3.46................3.62
Universal Server 5.0
POET Software Corp.
999 Baker Way, Suite 100
San Mateo, Calif. 94404
Tel: (650) 286-4640
Online:http://www.poet.com
Software Requirements:
Windows NT 4.0 or Windows 95
Hardware Requirements:
486 or higher x86 CPU, 32MB RAM, Pentium or higher suggested
RATING: ***1/2
Pros:
1. It's easy to learn and use effectively.
2. The total cost of ownership is low.
3. Its evaluation download is free.
Cons:
1. It has buggy GUIs.
2. Some of the online help is not very informative.
3. It doesn't support Rational Rose for Java.
ObjectStore 5.0
Object Design Inc.
25 Mall Rd.
Burlington, Mass. 01803
Tel: (800) 962-9620
Online:http://www.odi.com
Software Requirements:
Windows NT 4.0
Hardware Requirements:
486 or higher x86 CPU, 32MB RAM, Pentium or higher suggested
RATING: ****
Pros:
1. It has intuitive and attractive GUIs.
2. Its documentation is excellent.
3. It is scalable and rugged.
Cons:
1. There is a high total cost of ownership.
2. It is difficult to learn quickly.
3. It doesn't support Rational Rose for Java.
|
