Idem­po­tent Jms Con­sumer Pit­falls

A com­mon and per­ceived al­ter­na­tive to JTA/XA trans­ac­tions in mes­sag­ing is the so-called Idem­po­tent Con­sumer or Idem­po­tent Re­ceiv­er pat­tern.

Idem­po­tent Con­sumer Log­ic

The ba­sic flow goes like this:

1. The re­ceiv­er gets the next mes­sage from the queue
2. The re­ceiv­er process­es the mes­sage and up­dates the data­base ac­cord­ing­ly
3. The re­ceiv­er com­mits the DB changes
4. The re­ceiv­er re­turns any re­sponse (where ap­plic­a­ble)
5. The re­ceiv­er com­mits the queue (re­mov­ing the mes­sage)

Idem­po­tent Con­sumer Pit­falls

This works fine if there are no prob­lems. How­ev­er, a crash be­tween steps 3 and 5 will ef­fec­tive­ly lead to du­pli­cate mes­sage con­sump­tion (see Reli­able JMS with Trans­ac­tions for de­tails). A triv­ial so­lu­tion here is to use JTA/XA and, say, Trans­ac­tion­sEssen­tials -- and get im­proved per­for­mance at the same time.

How­ev­er, sur­pris­ing­ly many peo­ple choose to hack around as fol­lows:

1. The re­ceiv­er gets the next mes­sage from the queue
2. The re­ceiv­er checks if the mes­sage was al­ready processed
3. The re­ceiv­er process­es the mes­sage and up­dates the data­base ac­cord­ing­ly
4. The re­ceiv­er com­mits the DB changes
5. The re­ceiv­er re­turns any CACHED re­sponse (where ap­plic­a­ble)
6. The re­ceiv­er com­mits the queue (re­mov­ing the mes­sage)

The new step 2 here en­sures that mes­sages are nev­er processed twice, but it comes with a sig­nif­i­cant cost:

• a lot of com­plex­i­ty to hack around
• very hard to get right - re­quir­ing some kind of re­quest store and dirty reads with NO lock­ing to work in a clus­tered en­vi­ron­ment (by the way, how long are you go­ing to keep those re­quests stored in your data­base?)
• very hard to test be­cause of the above
• more im­por­tant: this is not scal­able

Step 5 re­quires you to store your re­spons­es in case the re­quest was processed be­fore, but the caller failed to re­ceive the an­swer. How are you go­ing to do that, and how long do you keep those re­spons­es around?

Also, scal­a­bil­i­ty is lim­it­ed be­cause all the mes­sages would prob­a­bly have to be tracked with in­serts in a data­base ta­ble of pri­or mes­sages. The im­pli­ca­tions are se­vere:

• This ta­ble is shared among all re­ceivers on the queue
• It be­comes a hot-spot that makes lin­ear scal­a­bil­i­ty hard

Per­for­mance

The idem­po­tent con­sumer pat­tern is slow. It may come as a sur­prise, but the idem­po­tent con­sumer tends to be much slow­er than XA - so it is hard­ly worth the ex­tra com­plex­i­ty. The only case where it may be use­ful is if you have an in­com­plete bro­ker that does not sup­port XA.

So­lu­tion

Want to avoid the has­sle of im­ple­ment­ing the idem­po­tent con­sumer and get high per­for­mance mes­sage con­sumers? Try this in­stead:

FREE down­load: Trans­ac­tion­sEssen­tials