Persistence frameworks

(Please note: “Persistence tools / frameworks” used throughout this entry refers to Hibernate, JDO, iBATIS, SDO, etc. I’m not considering JDBC to be a persistence tool for this entry.)

The risk associated with the persistence tools is very high given the state of flux that the industry is in (especially with EJB 3.0 coming out in the “near” future). Remember that choosing a technology isn’t just based on the cost to implement it today. It is based on the total cost which also includes all of the facets of maintenance and change.

If I had a project that had a persistence component and I knew that I was going to be actively developing and maintaining it for at least 2 years, I would likely use JDBC. Let’s look at some of the factors involved in my decision:

  • The persistence frameworks are all going to change significantly over the next two years especially with EJB 3.0 coming out. The same goes for the various query languages that they expose.
  • New persistence technologies may come out that replace the incumbents redering them effectively dead. (I’m concerned less about this with Hibernate than I am with, say, JDO. But that’s just a gut feeling based on the number of people using it, exposure, etc.) If IBM does what I think it will do with SDO, for example, then the space will change dramatically.
  • The learning curve with most of these persistence tools is very high. Some may poo-poo this, but if you do anything beyond “just the basics” then you’re investing a LOT of time learning, debugging, and posting on forums. *grin*
  • The maintenance efforts and risks for the persistence tools are not well understood. Given that the tools themselves are still in flux, the techniques and skills needed to maintain the tools and associated code are also changing.
  • JDBC, for the most part will remain unchanged. It is a known quantity with known risks and the maintenance efforts and associated risks are well understood.
  • There is no “specialized training” associated with JDBC. I don’t have the “hit by a bus” problem with JDBC that I have with the other persistence tools.

My job as a development manager is to produce software with the lowest total cost and the lowest risk. To me, JDBC is still the clear winner for the cases that I stated earlier.

I should point out that if you’re prototyping an application, working on certain types of one-off application, or working on a application with a severely limited lifetime then the RAD aspects of the persistence tools may work to your benefit.

If you disagree, let’s hear it!

Advertisements

8 comments

  1. The problems of a persistence are not new, and each of the technologies you discuss addresses the problem in a slightly different way – save JDBC. It doesn’t solve the underlying ORM problem, it just leaves it in your face to deal with. Yes it’s straight forward, but it’s also messy and prone to breakage. For any kind of complex use you’re still going to end up writing a TON of glue code to abstract the DB out of the situation, and now you have a custom framework to train developers to maintain – no forums to post to and only the docs they develop. I don’t know how your developers are, but under pressure to get the job done the fact of the matter is docs don’t get done well very frequently. Add into this the problems that frameworks and app servers solve – caching, automatic retrieval of an entire object graph…
    Yes there are problems with frameworks and with EJB. Lord knows there are problems with EJB and a really unpleasant learning curve if you don’t have developers conceptually up to speed on the ORM concepts. Many of these problems can be at least mitigated by other tools… its not as circular an argument as it sounds. One of the most useful tools I’ve seen or used in quite some time is XDOclet. Is it open source – yes. In flux – sure. Not documented – false. There is an excellent book on the subject, and having read only a little of it you can do wonderfully complex things – with EJB or with Hibernate. Contrary to the reality that open source doesn’t mean easy to fix, the templating language used by the XDoclet 1.x series is easy to understand and extend. I’ve don’t it with massive productivity benefits. And after careful evaluation at my company, we decided it wasn’t even a “big bet”. Actually no bet at all. We have the source, understand it (actually have it under our own source control for minor tweaks we make), and it works great, removing much of the learning curve and error proneness that hamper both EJB 2.0 CMP and hibernate.
    But wait, XDoclet requires Ant… and that’s a tool that requires a learning curve and is under change. So we should instead use batch files and scripts for all of our builds… Cruise control falls into the same category, so we should write our own continuous build system….
    I understand how you could see any of the ORM technologies as too big a bet, but to what degree of lost productivity will you suffer your developers to mitigate it?
    Actually, the best solution I’ve seen to this to date was in a recent JDJ article about a product called Metaboss. I won’t pitch for them, but the iteas put forth are the same ones we have for basic OO design (hide the implementation) are demonstrated to be applicable to all remoting technologies as well as persistence techniques.
    I suppose you make good arguments for using straight JDBC, but I can’t believe that all of the ORM tools are just hype that are going to fall away 6 months down the road, or that they are going to break backwards compatibility yet introduce so many new “must have” features that I’ll be mad I used hibernate 2.x rather than waiting for the 3.x series to leave beta. And while there is always some hype around ORM tools, you have to admit that not EVERYONE has drunk some nasty industry kool-aid and has gone ORM crazy. enterprise persistence is a real problem, and there are a good number of ways to solve it. I don’t think any of the mitigating factors make the existing tools such a bit bet that I’d torture my developers with writing their own framework to escape the risks.

  2. Thank you for your comments Mr. Greene. You bring up the excellent points that I was hoping that someone would mention: what about the bets surrounding the infrastructure necessary to support JDBC? Doesn’t the creation of your own custom persistence framework using JDBC have the same problems and bets as with the “COTS” (consumer, off the self — in this case meaning Hibernate, JDO, etc) persistence frameworks? The answer is a resounding YES! And you even brought up the other important point of there being no forums to go for help on when using your own tools.
    The current set of persistence frameworks didn’t form out of a vacuum. There was a well defined need and these tools have grow out of that need. So given this, why do I still say use JDBC for long term projects? I simply believe that the bet associated with the state of flux is too large to mitigate in most cases. For example, the recent move of Mr. King to JBoss scared the snot out of me. It seems to be working well, but that is one of those things that could have stagnated or forked Hibernate into a nasty state. I’m still waiting to see how it all is going to work out. Even more recently, the hullabullo with JDO 2.0 gave me the willies.
    As to your question “but to what degree of lost productivity will you suffer your developers to mitigate it?”, I don’t have a canned answer. As part of the design phase we would need to look at all factors involved. If, for example, I have a few people on my team with experience in delivering successful Hibernate projects then obviously a number of the factors in the Hibernate bet diminish greatly and an appropriate decision needs to be made. If we’re all persistence framework virgins then I would look at mitigating the JDBC bet with careful architecture choices. For example, if you’re using a more mature database, you may be able to simplify some of your persistence concerns using views or stored procedures. What about something like Tangosol’s Coherence to eliminte some of the unneeded data persistence?
    I should point out that not all frameworks / tools give leave me as unsettled as the persistence frameworks. You brought up a great number of cases: XDoclet, Ant, and Cruise Control.
    Don’t even get me going on EJB. That big fat hairy bet scares the snot out of me. But that is another topic for another time *grin*.

  3. Our company is now evaluating the value of using ORM tools, specifically Hibnernate. We have implemented a few simple data access services successfully with the tool. What we are finding out is that, as mentioned in the original comment, the learning curve is quite steep when going beyond just simple CRUD. We are also finding that the overhead is quite significant.
    I have been working in the area of interfacing applicaitons to data for more than 10 years. I remember evaluating ORM tools such as TopLink back in the mid 90’s. Though they have made some progress, specifically in providing more flexibility in allowing you to define your own object model, many of the problems that existed in the mid 90’s still exist today. The primary problems with ORM tools are:
    Performance
    Resource Utilization (memory and CPU)
    Lack of tuning flexibility
    Tool Complexity
    We have also faced some of the issues mentioned in regard to backward compatibility. We found that some of our “configuration” of our existing Hibernate 2.X implementation will have to change when we go to Hibernate 3.x.
    Some of the claims for benefits of ORM tools that I have heard include:
    Easier for developers
    Reduces need for SQL expertise
    Reduces framework code
    faster development
    less costly to maintain (e.g. keep up with DB changes)
    What we have discovered so far is that some of these things are true for simple CRUD, but not in general.
    Easier – I think people tend to over estimate the complexity of SQL and JDBC while underestimating the complexity of ORM tools.
    Need for SQL Experts – Since we are finding that Hibernate is not useful in more complex scenarios, we still need SQL Expertise on projects. Most developers, even those not experienced in SQL, can learn to write simple CRUD SQL statements quickly. SQL isn’t that hard!
    Also, in using an ORM tool, the developer needs to learn a proprietary query language, such as HQL for Hibernate. Is that really less complex than SQL? It certainly isn’t standard!
    Reduces Framework code – We use a DAO framework. The base framework only consists of about 3 classes, which are infrequently updated. The most significant savings is probably in the code written to process the result set. However, this is typically a simple for-loop consisting of a dozen or so lines of code. Not sure that justifies an ORM tool.
    Faster Development – It may be slightly faster if an experienced Hibernate developer does simple CRUD. Generally, we are not seeing the benefit here.
    Lower Maintenance costs – This argument is primarily around accomodating database changes (e.g. adding fields or changing table/column names, etc.) This may be true for some changes, where only the mapping XML and object have to change in ORM vs. having to SQL, result set code, and object in JDBC (DAO). When you look at it, the only real difference is updating the result set code. Not really that big of a difference.
    We still have some work to do to complete our evaluation. So far, we haven’t experienced the advertised benefits of ORM (Hibernate).

  4. Thank you Mr. Bellamy for that excellent write-up. You experience with O-R with CRUD and non-CRUD follows my experience and expectations. When you get more information I’d would love to hear it.
    Take care and thank you again!

  5. You wont here me say I’m the expert on object-relational mapping. But I have some experience doing both JDBC and ORM persistence.
    My guess is that the most important part of the design here is that your business layer should not be exposed to your persistence layer, or if you cannot help it only in a very abstract manner. I’ve seen project fail only because of the fact they exposed their SQL in their business layer (or even GUI layer). If you expose the SQL in your business layer even adding an attribute becomes a pain, especially if you have many reports extracting different kinds of data on the same type.
    Overall I would tend to go for an ORM solution now because it can take a lot of boring code out of your application and in most cases can do some quite decent caching. But if it would happen to be the case I can’t do it in a project I would definitely use something like Spring and JdbcTemplate to hide the overall persistence layer and still be able to swap in a real ORM solution if the lack of ‘object’ caching becomes a problem.
    But all in all I can understand your doubts with respect to the current state of ORM. Developer experience is required and you will need to do some tuning.

  6. Having worked on a few projects over the last few years including a few large ones, I still don’t get the hype about ORM tools even for large databases.
    I think the risks outweigh the benefits of ORM tools such as Hibernate, JDO, EJBs.
    Everyone knows DAOs and JDBC/SQL. That immediately makes it desirable because there is a learning curve to use any of the other OR tools and query languages – EJBQL, JDOQL etc.
    I think JDBC/SQL provides a lot of flexibility, simplicity and transparency. You will have to jump through hoops to achieve the same using an ORM tool.
    Admittedly you will be writing additional lines of persistence code with JDBC/SQL but it is fairly simple code that is easy to write/test/maintain.
    Of the risks on a large project, I feel technical issues are low risk items and of that, the additional lines of code written for JDBC/SQL persistence are insignificant.
    I think history has proven how successful OR tools are. If they were truly successful, we would have gravitated to one standard mechanism. I can name several standards and OR tools.

  7. Hi, are you still arguing against use of persistence frameworks?
    In my experience there is spent to little time to learn the frameworks and to shear the knowledge among the developer team members. Or rather neglect the fact that the developer need to spend some time to find out how to use the new “tool” in a proper manner.
    In particular frameworks like Hibernate and Spring build on a great number of design patterns which is often is not communicated to the new participators.

  8. Hello Ole! Thanks for the comment.
    I’m not against persistent frameworks per se. I’m about properly identifying and managing the risk associated with software development.
    My original posting was written over 3 years ago and as predicted the industry went through a period of large change. Things have certainly stabilized (or are still stabilizing) and one can more safely choose a persistence framework and be assurred that it will do what they need it to do (given that the right choice is made), have a large degree of stability and be around when they’re done with their project. Additionally, the maintenance models of these technologies are much better known.
    I completely agree with you on the learning curve, the time and effort *not* spent properly understanding the tool, and the communication problems that plague most projects. It’s an inherent risk that must be understood and mitigated early on.
    Take care!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s