Idempotent Jms Consumer Pitfalls

A common and perceived alternative to JTA/XA transactions in messaging is the so-called Idempotent Consumer or Idempotent Receiver pattern. The basic flow goes like this:

1. The receiver gets the next message from the queue
2. The receiver processes the message and updates the database accordingly
3. The receiver commits the DB changes
4. The receiver commits the queue (removing the message)

This works fine if there are no problems. However, a crash between steps 3 and 4 will effectively lead to duplicate message consumption (see Reliable JMS with Transactions for details). A trivial solution here is to use JTA/XA and, say, Atomikos TransactionsEssentials®.

However, surprisingly many people choose to hack around as follows:

1. The receiver gets the next message from the queue
2. The receiver checks if the message was already processed
3. The receiver processes the message and updates the database accordingly
4. The receiver commits the DB changes
5. The receiver commits the queue (removing the message)

The new step 2 here ensures that messages are never processed twice, but it comes with a huge cost:

• a lot of complexity to hack around
• more important: this is not scalable

Scalability is limited because all the messages would have to be tracked with inserts in a database table of prior messages. The implications are severe:

• This table is shared among all receivers on the queue
• It becomes a hot-spot that makes linear scalability hard

Ironically, most people will argue that XA is not scalable. This is wrong: it is linearly scalable with Atomikos.