ACID transactions across REST microservices

So how would you implement classical, ACID transactions across REST microservices?

Disclaimer

If you are a REST purist then you may not like the idea, but several pragmatist developers have been asking us how to implement classical ACID transactions for their REST systems. After all, splitting up that monolith is nice but there's still data involved…

Introduction

Some of you may know our TCC for compensation-based REST transactions for reservation-style interactions. It's nice and loosely coupled, but it not a perfect fit for scenarios that would like to use a classical rollback instead of compensation. This post presents an alternative design, one that we are currently working on here at Atomikos.

We'll refine the specification incrementally in the rest of this post, so the concepts are easier to digest.

Introducing the actors involved

We'll use the following actors and responsibilities:

REST application

Imagine a (composite) REST application that:

• Co-ordinates the overall workflow
• Calls a number of participant services
• Commits (or rolls back) all of the participant services' work via a coordinator service (as described below).

Participant service

A participant service is really a microservice that:

• Creates a local ACID transaction when it is invoked
• Performs local database work
• Keeps the transaction open at the end of the invocation for later commit by the REST application (as described below).

Coordinator service

The coordinator service is a specialised utility service that coordinates two-phase commit across several participant services. It also performs logging and recovery in case of failures and/or restart. More details are below…

The minimal use case: commit or rollback REST microservices together

The minimal use case is really the most basic thing one could do in this context: two or more microservice calls are committed or rolled back together. It goes like this:

1. The REST application goes about and calls any number of participant services.
2. Each participant service returns (as part of its response) AcidParticipantLink information: basically a URI where the transaction can be committed.
3. The REST application collects all the AcidParticipantLinks as it proceeds in the overall "transaction" logic.
4. The REST application then asks the coordinator service to commit (or, alternatively, rollback) ALL the relevant AcidParticipantLinks of its work.

More elaborate use case: repeated invocations can share the same locks

The previous scenario works, but repeated invocations of the same participant will use different local ACID transactions, meaning they cannot see/access the same data in the database. There is no lock sharing among subsequent invocations. In some cases this may be desirable, and this can be done as follows:

1. The REST application calls a participant service for the first time and gets an AcidParticipantLink back.
2. The REST application extracts the participant's transaction identifier (included in the AcidParticipantLink).
3. The REST application can now call the participant service a second time, with the transaction identifier as an argument of the call.
4. The participant service detects the existing transaction identifier and allows the call to "join" the existing transaction.

Everything else works like in the previous scenario.

Advanced use case: a service sharing locks of the caller in a shared database

In some cases it may be desirable for the participant service to join a transaction that exists at the caller side, notably if they access some shared database. The details of this will be worked out later, so keep checking back for more!