Sup­pose you want to de­vel­op a high-vol­ume trans­ac­tion pro­cess­ing sys­tem in Java/J2EE. How would you do it? Most peo­ple would say: don't use JTA/XA trans­ac­tions be­cause they kill per­for­mance. Wrong. And they would also say: use an appserv­er to scale. Again, they couldn't be more wrong.

Here is the mag­ic recipe on how we build sys­tems with vir­tu­al­ly un­lim­it­ed scal­a­bil­i­ty at Atomikos:

• Kick out your appserv­er as soon as you can, as ex­plained here. J2EE is not lim­it­ed to an appserv­er. J2EE is a set of APIs. The appserv­er ties these APIs to a pro­gram­ming mod­el that al­most no­body needs. Con­clu­sion: drop the lat­ter.
• Use a per­sis­tent JMS queue to store trans­ac­tion re­quests. This al­lows easy load-bal­anc­ing and pro­vides crash re­silience for on­go­ing re­quests. It also de-cou­ples the clients from the trans­ac­tion pro­cess­ing sys­tem.
• Use Ex­tremeTrans­ac­tions to process the re­quests (stored in JMS). This al­lows for re­li­able, ex­act­ly-once mes­sage pro­cess­ing as out­lined here. Make sure to use the sup­plied JMS and JDBC dri­vers!
• To add more pow­er, just add a sec­ond VM (process) on a sep­a­rate CPU.
• Re­peat un­til per­for­mance is high enough.

You will reach the re­quired per­for­mance be­cause of the in­tra-VM na­ture of each process you add. The only po­ten­tial bot­tle­necks are your own data­base or JMS back­end. So scal­ing comes down to scal­ing your back­ends, which is much sim­pler than scal­ing your ap­pli­ca­tion it­self (which has al­ready been done in a nat­ur­al way as out­lined above).

So don't let any­body fool you: trans­ac­tions do scale - even with­out lim­its!.

Ex­tra: Elas­tic scal­ing

Our com­mer­cial prod­uct Ex­tremeTrans­ac­tions in­cludes elas­tic scal­ing fea­tures for cloud de­ploy­ments. You can ap­ply for a free tri­al be­low...

Try Ex­tremeTrans­ac­tions For Free

I have talked to a num­ber of peo­ple who claim to be do­ing SOA, when in the end all they do is loose­ly-cou­pled de­sign. Let me ex­plain what I mean by an ex­am­ple.

A team of en­ter­prise ar­chi­tects was de­sign­ing an SOA in­fra­struc­ture for a bank I know. The sys­tem they were build­ing would be based on in­ter­faces, so that it would be pos­si­ble to de­ploy parts of the sys­tem as sep­a­rate in­stances lat­er on. This was their no­tion of SOA...

The good thing about it is that there are in­ter­faces in their de­sign, mean­ing it is like­ly to be loose­ly-cou­pled. The bad news is that this is not SOA, at least not in my view: one of the biggest ad­van­tages of SOA - reuse in place - is nev­er re­al­ized in this way. So, where­as this ap­proach to 'SOA' may be loose­ly cou­pled in de­sign, it is not loose­ly cou­pled in de­ploy­ment (which is at least as im­por­tant).

The con­se­quence? When­ev­er a 'ser­vice' is up­grad­ed, they will need to up­grade all the de­pen­dent ser­vices and re­de­ploy them. This is be­cause each 'ser­vice' is re­al­ly an em­bed­ded mod­ule in­side oth­er parts of the sys­tem.

I guess this also holds for the de­bate on cloud vs grid com­put­ing: in my view, a cloud is more loose­ly cou­pled than a grid in its de­ploy­ment.

Is BPEL a good tool for im­ple­ment­ing com­pen­sa­tion? It re­al­ly de­pends, and you re­al­ly have to know what you are do­ing - which (with all re­spect) doesn't seem the case for most peo­ple (not even BPEL spe­cial­ists). So if not even those ex­perts know, how can we ex­pect the rest of us to know? Hence this blog en­try.

For in­stance, on re­peat­ed oc­ca­sions I have heard renowned BPEL and work­flow ex­perts men­tion that com­pen­sat­ing trans­ac­tions are "per­haps" best mod­eled at the busi­ness log­ic lev­el. This, by the way, in­cludes Bill Burke in the case of JBoss/jBPM - see here. Note that I em­pha­sized the word "per­haps": this in­di­cates the shade of mis­un­der­stand­ing usu­al­ly present in the ar­gu­ments.

I have been say­ing this here and there in the past (and in fine de­tail in this ar­ti­cle), but I want to re­peat it again: BPEL, nor work­flow nor WS-BA are ide­al for com­pen­sa­tion un­less the com­pen­sat­ing par­ty doesn't care whether it needs to com­pen­sate even­tu­al­ly. In oth­er words, if the com­pen­sa­tion is busi­ness as usu­al to the provider of the com­pen­sa­t­able ser­vice then BPEL might be OK (though cer­tain­ly not de­sir­able - see be­low).

Why is that? Put your­self in the place of a ser­vice that is asked to com­pen­sate by a BPEL en­gine some­where. Also sup­pose that you are in a B2B ecosys­tem where you don't nec­es­sar­i­ly trust the par­ty that owns the BPEL en­gine. Now what would you rather do: trust the BPEL to com­pen­sate - even­tu­al­ly (which might be nev­er!) or rather deal with com­pen­sa­tion your­self, say af­ter a time­out? I would def­i­nite­ly choose the lat­ter. I don't want some­one else to de­cide when I need to com­pen­sate. I want to de­cide for my­self, and the Atomikos TCC mod­el al­lows for that. BPEL and jBPM don't.

So BPEL is ruled out for me - at least as far as com­pen­sa­tion goes. What about WS-BA? It is a step in the right di­rec­tion, but un­for­tu­nate­ly it is a bloat­ed pro­to­col, very in­ef­fi­cient and loaded with ap­pli­ca­tion-lev­el mes­sages that pol­lute the com­pen­sat­ing part. Even worse, it also suf­fers in a large part from the lack of time­out and de­pends on the BPEL to at least trig­ger com­pen­sa­tion.

Also, WS-BA doesn't al­low for ap­pli­ca­tion log­ic on close - I won't go and both­er you with the en­tire spec de­tails but it is like a try..catch...fi­nal­ly where the ex­cep­tion is raised by the client (ugly!) and where the fi­nal­ly block can only be emp­ty! Again, Atomikos TCC is far su­pe­ri­or, more ef­fi­cient and more el­e­gant. It is also more nat­ur­al for com­pen­sa­tion than any BPEL en­gine will ever be.

One last note on BPEL and this sup­posed "mod­el­ing the com­pen­sa­tion in the busi­ness process": I was talk­ing to an IBM ar­chi­tect the oth­er day. He said that they were do­ing a large tel­co project with BPEL to co-or­di­nate things. One of the things he com­plained about was ex­act­ly this: they have to mod­el the com­pen­sa­tion and er­ror log­ic as ex­plic­it work­flow paths, and it was lit­er­al­ly over­load­ing every­thing with com­plex­i­ty. More­over, this com­plex­i­ty is hard to test. As he cor­rect­ly put it, they were im­ple­ment­ing a trans­ac­tion man­ag­er at the busi­ness log­ic (BPEL) lev­el, over and again in every process mod­el. In ad­di­tion, this was also hard to test he said and that it was vir­tu­al­ly killing the project - es­pe­cial­ly if there were change re­quests to con­sid­er. I be­lieve him:-) I gave him the URL to our TCC ar­ti­cle above.

Atomikos and TCC al­low you to fo­cus on the hap­py path of your work­flow mod­els. We take care of the rest. Now imag­ine what a re­duc­tion in com­plex­i­ty that is, and how much more re­li­able things get! So no, com­pen­sa­tion should NOT be mod­eled at the busi­ness lev­el. Ex­cept on rare oc­ca­sions maybe.

When­ev­er I see a pre­sen­ta­tion on REST I am im­pressed by its sim­plic­i­ty. With just four op­er­a­tions (GET, POST, PUT, DELETE) it seems to ac­com­plish a sim­ple mod­el for ser­vice-ori­ent­ed ar­chi­tec­tures, where every busi­ness re­source has a URL.

With this sim­plic­i­ty, REST also lever­ages the ubiq­ui­tous HTTP pro­to­col as the un­der­ly­ing mech­a­nism. More and more peo­ple seem to like this, in­clud­ing me.

How­ev­er, the big ques­tion for me is: how do you make this re­li­able? Imag­ine that you in­te­grate 4 sys­tems in a REST style. You would be us­ing HTTP and a syn­chro­nous in­vo­ca­tion mech­a­nism for each ser­vice. Now comes the ques­tion: how re­li­able is this? The an­swer: less than the least re­li­able sys­tem that you are us­ing! More pre­cise­ly, avail­abil­i­ty goes down quick­ly be­cause your ag­gre­gat­ed ser­vice fails as soon as one of the ser­vices fails...

With trans­ports like JMS you can im­prove re­li­a­bil­i­ty, but how do you do REST of JMS, giv­en its close re­la­tion­ship with HTTP and URLs? That is the prob­lem with REST for me.

When de­sign­ing a cor­po­rate SOA ar­chi­tec­ture you are of­ten faced with a tough choice: do you rely on a com­mon data­base (cen­tral­ized) or do you im­ple­ment repli­ca­tion in­stead?

Let me ex­plain what I mean. The idea in SOA is that you de­fine more or less in­de­pen­dent ser­vices that cor­re­spond (hope­ful­ly) to clear­ly de­fined and busi­ness-re­lat­ed ac­tiv­i­ties. For in­stance, you could have a cus­tomer man­age­ment ser­vice and a pay­ment/in­voic­ing ser­vice. The cus­tomer man­age­ment ser­vice be­longs to CRM, the in­voic­ing to the billing de­part­ment. How­ev­er, both of these ser­vices might need the same cus­tomer data. Now what do you do? Ba­si­cal­ly, you have the fol­low­ing op­tions:

1. Use the same cen­tral­ized cus­tomer data­base. This gives you the ben­e­fit of easy main­te­nance be­cause there is only one copy. How­ev­er, this also means that you are cou­pling your ser­vices into the same data­base schema, and up­dates to the schema are like­ly to af­fect more than one ser­vice.
2. Repli­cate the cus­tomer data­base, by iden­ti­fy­ing one mas­ter (the CRM?) that reg­u­lar­ly push­es or pub­lish­es up­dates (in an XML feed, for in­stance). While you lose the ben­e­fit of easy main­te­nance, this does give you loose cou­pling: as long as the XML for­mat is the same, you can change DBMS schemas as much as you like - with­out af­fect­ing oth­er ser­vices.
3. Merge the cus­tomer and in­voic­ing ser­vices into one. How­ev­er, this may not al­ways be pos­si­ble or de­sir­able, and may even de­feat the pur­pose of ser­vice-ori­ten­ta­tion al­to­geth­er.
4. Have the in­voic­ing query the cus­tomer ser­vice for each pay­ment. Thi seems to in­cur a lot of de­pen­den­cies and net­work traf­fic.

So what do you do? My pref­er­ence tends to go to the sec­ond op­tion. How­ev­er, it means that re­al­is­tic SOA ar­chi­tec­tures are like­ly to have an event-dri­ven na­ture.