Best of both worlds: us­ing re­la­tion­al stor­age and the DDD event store to­geth­er

Read here how to com­bine DDD with the safe­ty and con­sis­ten­cy of re­la­tion­al stor­age so you can ex­per­i­ment with DDD with­out the risks...

The prob­lem

Re­la­tion­al stor­age is safe, and wide­ly un­der­stood. Do­main-dri­ven de­sign (DDD) and event stores are new and not so wide­ly un­der­stood. Ap­ply­ing DDD to your project can have dra­mat­ic ar­chi­tec­tur­al im­pact, mean­ing the cost of mak­ing a mis­take can be very high. Giv­en the lack of suf­fi­cient ex­per­tise among most teams, this is a sig­nif­i­cant project risk.

In the DDD world, purists will say that ag­gre­gate (root) ob­jects should be saved in an "event store". This means that the lat­est state of the ob­ject is re­con­struct­ed by se­quen­tial­ly re-ap­ply­ing all past events since the be­gin­ning of time (or since the last check­point). This ap­proach im­plies a cou­ple of ma­jor ar­chi­tec­tur­al de­ci­sions at the start of your project:

1. The scope of the ag­gre­gate root must be de­fined (along with the trans­ac­tion scope) so that the event store con­tains only sig­nif­i­cant events and no oth­er. Other­wise, re­con­struct­ing the state based on events be­comes cum­ber­some. The con­se­quence: if you make a mis­take here, you are fac­ing ar­chi­tec­tur­al re­design af­ter­wards - not a pleas­ant out­look. Yet, few peo­ple have good knowl­edge of DDD at this time and ag­ile prac­tices al­most for­bid a de­sign-up-front ap­proach.
2. The stor­age it­self must be cho­sen - pos­si­ble a non-re­la­tion­al data­base or even flat files of your own. Again, this is a ma­jor de­ci­sion in your project and it may or may not be com­pat­i­ble with the evo­lu­tion of your re­quire­ments.
3. Query­ing a non-re­la­tion­al data­base has its chal­lenges and lim­i­ta­tions. This means you would have to pro­gram query log­ic your­self.

To sum­marise: the de­ci­sion to take a DDD purist ap­proach is a sig­nif­i­cant one, and the cost of mak­ing a mis­take is very high. By de­f­i­n­i­tion, in the be­gin­ning of our project we do not know every­thing yet so we would pre­fer to be more flex­i­ble - i.e. not favour­ing one ap­proach over the oth­er and stay­ing com­pat­i­ble with fu­ture re­quire­ments. This strat­e­gy also al­lows us to mit­i­gate the risks due to lack of ex­per­tise with DDD in the project teams.

The so­lu­tion

Ar­chi­tec­ture Di­a­gram

Let's sup­pose your ap­pli­ca­tion is about or­der man­age­ment. Our pro­posed ar­chi­tec­ture looks like this:

Er­ror: im­age not found

Use both re­la­tion­al and DDD event stor­age to­geth­er. Have your de­vel­op­ers pro­gram in what they know and like best and com­bine the two ap­proach­es seam­less­ly in a loose­ly-cou­pled way, as ex­plained be­low.

Per­sist your do­main ob­jects the tra­di­tion­al way, pro­duce do­main events as pub­lish/sub­scribe

• Just like you al­ways did: use Hib­er­ate, JPA or JDBC (or any oth­er re­la­tion­al per­sis­tence map­ping) to store your ob­jects in the data­base.
• Within the same trans­ac­tion, pub­lish a do­main event on a JMS/XA des­ti­na­tion.
• Use ACID trans­ac­tions (like with Atomikos) to make both of the above hap­pen to­geth­er ex­act­ly once - or not at all.

This is shown in the ar­chi­tec­ture di­a­gram: the Order man­age­ment ap­pli­ca­tion stores or­ders in the re­la­tion­al data­base and posts an event mes­sage to the bro­ker.

Add any num­ber of event sub­scribers and/or event stores

The way events are pro­duced above al­lows any num­ber of sub­scribers to treat every sin­gle event, there­by stor­ing it, trans­form­ing it into a spe­cif­ic query mod­el or any­thing else for that mat­ter. This al­lows you to build your event store side-by-side with the tra­di­tion­al ap­pli­ca­tion ar­chi­tec­ture, so you don't have to make any sig­nif­i­cant high-risk de­ci­sion ear­ly on in your project. Your oth­er ap­pli­ca­tions can choose to ei­ther query the tra­di­tion­al data­base, or use the event store one way or an­oth­er. The point is: they don't have to make a choice right now.

The ar­chi­tec­ture di­a­gram shows how this can work for a No-DDD ap­pli­ca­tion, an event store and any oth­er ap­pli­ca­tion be­sides those.

Ad­di­tion­al ben­e­fits

Com­pat­i­bil­i­ty with non-DDD con­sumers / ap­pli­ca­tions

While it may be fea­si­ble for your teams to make the in­ter­nal de­sign choice to go for DDD, there will al­ways be ex­ter­nal or 3rd par­ty ap­pli­ca­tions that need to in­te­grate with your work and that were built with­out the DDD mind­set. In oth­er words: not all ap­pli­ca­tions were de­signed for query­ing an event store to re­trieve the lat­est events. For these ap­pli­ca­tions, the pub­lish/sub­scribe ap­proach of­fers a flex­i­ble and proven way of in­te­grat­ing with your projects.

Orders are saved faster

Im­ple­ment­ing an event store can be a chal­lenge, and it typ­i­cal­ly takes a lot of event records to store the lat­est state. Stor­ing a new or­der can also take a lot of in­serts, which are slow­er than re­la­tion­al up­dates.

Easy queries on the re­la­tion­al store

Query­ing an event store is not straight­for­ward. Be­cause the re­la­tion­al stor­age is con­served, SQL can come to the res­cue.

Want to try for your­self? Down­load and start us­ing Trans­ac­tion­sEssen­tials to­day - it's FREE!

The XA trans­ac­tions in this post are eas­i­ly pro­vid­ed with our free prod­uct - try it out for your­self by giv­ing your email ad­dress and we'll send you the down­load in­struc­tions plus doc­u­men­ta­tion:

Down­load Now

I have read and ac­cept the pri­va­cy pol­i­cy.

What about lost mes­sages / events?

If the event store uses XA trans­ac­tions to con­sume mes­sages then you can achieve ex­act­ly once de­liv­ery with zero mes­sage loss (for more in­for­ma­tion see this post: re­li­able JMS with trans­ac­tions).