Fea­ture185294

Add gRPC sup­port and ex­am­ples

We now of­fer re­mot­ing sup­port for gRPC so your trans­ac­tions can span gRPC calls.

Tech­ni­cal de­tails

Some­times you may want com­mit or roll­back to ex­tend across one more more out­go­ing gRPC calls. This is now pos­si­ble.

See the ex­am­ple mod­ule: ex­am­ples-jta-grpc in the down­load for a work­ing sam­ple.

Changes im­pact­ing client API

New in­ter­cep­tors for your gRPC ar­chi­tec­ture are in­clud­ed.

Fea­ture188756

Add spring boot starter

Gen­er­ous­ly do­nat­ed by Piv­otal's Spring Boot team, we were able to add the Spring Boot starter mod­ule to our code base.

Tech­ni­cal de­tails

In­stead of adding Spring's starter mod­ule, you should now add "our" starter mod­ule to your Spring Boot project's pom.

In par­tic­u­lar, in­stead of spec­i­fy­ing:

   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-jta-atomikos</artifactId>
   </dependency>

You should now spec­i­fy:

   <dependency>
      <groupId>com.atomikos</groupId>
      <artifactId>transactions-spring-boot-starter</artifactId>
      <version>5.0.9</version> <!-- or any later atomikos release that contains our starter module -->
   </dependency>

Ad­di­tion­al de­tails are here...

Changes im­pact­ing client API

No real API changes, only pom de­pen­den­cy changes.

Fea­ture189603

API ex­ten­sion to al­low trig­ger­ing re­cov­ery by the ap­pli­ca­tion

You can now ex­plic­it­ly trig­ger re­cov­ery in your ap­pli­ca­tion, via our API.

Tech­ni­cal de­tails

Re­cov­ery al­ready hap­pens pe­ri­od­i­cal­ly, in the back­ground. For big­ger clus­ters that con­nect to the same data­base (or oth­er shared re­source) this can cause a high load on the back­end, be­cause of many such back­ground threads hit­ting the back­end at the same time. This is es­pe­cial­ly true if most clus­ter nodes start up at the same time with the same con­fig­u­ra­tion for re­cov­ery, and are NOT us­ing LogCloud Doc­u­men­ta­tion. To al­le­vi­ate this, you can now have a bit more con­trol over when re­cov­ery hap­pens, like this:

import com.atomikos.icatch.RecoveryService;
import com.atomikos.icatch.config.Configuration;

boolean lax = true; //false to force recovery, true to allow intelligent mode
RecoveryService rs = Configuration.getRecoveryService();
rs.performRecovery(lax);

In or­der for this to work, make sure to set (in jta.prop­er­ties):

# set to Long.MAX_VALUE so background recovery is disabled
com.atomikos.icatch.recovery_delay=9223372036854775807L 

Changes im­pact­ing client API

We have added meth­ods on an ex­ist­ing API in­ter­face, which does not break ex­ist­ing clients.

Bug189264

Make sure timed out trans­ac­tions are cleaned up in the JVM

Sever­i­ty:	2
Af­fect­ed ver­sion(s):	5.0.x

De­scrip­tion

Timed out trans­ac­tions (in par­tic­u­lar heuris­tic haz­ard cas­es af­ter con­nec­tion is­sues) used to stay around in the JVM and kept on gen­er­at­ing time­out events. This has now been fixed.

Tech­ni­cal de­tails

Aban­doned trans­ac­tion in­stances now also stop any pend­ing threads that gen­er­ate time­out warn­ings. This pre­vents timed out trans­ac­tions from gen­er­at­ing end­less warn­ings that are no longer rel­e­vant.

Changes im­pact­ing client API

None.

Bug189263

JtaU­nawareThread­Lo­calCon­nec­tion: close should no­ti­fy pool for reuse

Sever­i­ty:	2
Af­fect­ed ver­sion(s):	5.0.x

De­scrip­tion

We op­ti­mised con­nec­tion pool ef­fi­cien­cy for non-JTA/XA aware use cas­es, so con­nec­tions are reused more ef­fi­cient­ly when wait­ing for busy con­nec­tions.

Tech­ni­cal de­tails

Pre­vi­ous­ly, wait­ing con­nec­tion re­quests were not no­ti­fied im­me­di­ate­ly when a busy con­nec­tion was be­ing closed (i.e., marked for reuse) by the ap­pli­ca­tion. This has now been fixed.

Changes im­pact­ing client API

None.

Bug188918

XaRe­sourceTrans­ac­tion: don't log ERROR when xare­source is null

Sever­i­ty:	2
Af­fect­ed ver­sion(s):	5.0

De­scrip­tion

In some rare cas­es the XARe­source used for com­mit­ting a trans­ac­tion af­ter pre­pare may be­come null (pre­sum­ably due to con­nec­tion er­rors, un­con­firmed though). In­stead of log­ging an er­ror (like we used to), we now trust back­ground-lev­el re­cov­ery to han­dle this - for which it was de­signed in the first place. This should avoid re­peat­ed the er­rors logged in such a case.

Tech­ni­cal de­tails

This "bug" would lead to mil­lisec­ond-lev­el re­peat­ed er­rors sim­i­lar to the fol­low­ing:

19/11/2020 15:20:43.467 [Atomikos:3321] ERROR com.atomikos.icatch.imp.CommitMessage - Unexpected error in commit
com.atomikos.icatch.HeurHazardException: XAResourceTransaction: 31302E3235342E3134362E3131302E746D313539353531353037383735393139373038:31302E3235342E3134362E3131302E746D373030333533: no XAResource to commit?
        at com.atomikos.datasource.xa.XAResourceTransaction.commit(XAResourceTransaction.java:529)
        at com.atomikos.icatch.imp.CommitMessage.send(CommitMessage.java:52)
        at com.atomikos.icatch.imp.CommitMessage.send(CommitMessage.java:23)
        at com.atomikos.icatch.imp.PropagationMessage.submit(PropagationMessage.java:67)
        at com.atomikos.icatch.imp.Propagator$PropagatorThread.run(Propagator.java:63)
        at com.atomikos.icatch.imp.Propagator.submitPropagationMessage(Propagator.java:42)
        at com.atomikos.icatch.imp.HeurHazardStateHandler.onTimeout(HeurHazardStateHandler.java:71)
        at com.atomikos.icatch.imp.CoordinatorImp.alarm(CoordinatorImp.java:650)
        at com.atomikos.timing.PooledAlarmTimer.notifyListeners(PooledAlarmTimer.java:95)
        at com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:82)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
19/11/2020 15:20:53.468 [Atomikos:3321] ERROR com.atomikos.datasource.xa.XAResourceTransaction - XAResourceTransaction: 31302E3235342E3134362E3131302E746D313539353531353037383735393139373038:31302E3235342E3134362E3131302E746D373030333533: no XAResource to commit?

Now we no longer log these as er­rors, since we de­signed re­cov­ery to take care of ex­act­ly those kinds of ex­cep­tions.

Changes im­pact­ing client API

None.

Bug185591

Bug in JDBC state­ment proxy: callable state­ment

Sever­i­ty:	4
Af­fect­ed ver­sion(s):	5.0.x

De­scrip­tion

Fixed a bug that would hap­pen in cer­tain class load­ing en­vi­ron­ments and pre­vent­ed Cal­lableS­tate­ments from be­ing cre­at­ed.

Tech­ni­cal de­tails

Fixed a bug that would hap­pen in cer­tain class load­ing en­vi­ron­ments and pre­vent­ed Cal­lableS­tate­ments from be­ing cre­at­ed. This would lead to er­rors like this:

java.lang.ClassCastException: com.sun.proxy.$Proxy364 cannot be cast to java.sql.CallableStatement
            at com.atomikos.jdbc.internal.AbstractJdbcConnectionProxy.prepareCall(AbstractJdbcConnectionProxy.java:73)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at com.atomikos.util.DynamicProxySupport.callProxiedMethod(DynamicProxySupport.java:162)
            at com.atomikos.util.DynamicProxySupport.invoke(DynamicProxySupport.java:116)
            at com.sun.proxy.$Proxy64.prepareCall(Unknown Source)

Changes im­pact­ing client API

None.

Bug183884

Re­mot­ing par­tic­i­pants do not re­spect com­mit or­der­ing

Sever­i­ty:	2
Af­fect­ed ver­sion(s):	5.0.x

De­scrip­tion

Re­mot­ing par­tic­i­pants now also re­spect com­mit or­der­ing so you can avoid is­sues with JMS no­ti­fi­ca­tions go­ing out be­fore the back­end data­bas­es are up to date.

Tech­ni­cal de­tails

Con­sid­er the fol­low­ing sce­nario for your mi­croser­vice:

1. re­ceive a JMS mes­sage 2. do a re­mote call to a dif­fer­ent mi­croser­vice that in turn up­dates a DB 3. send out a JMS "tick­et" mes­sage

Pre­vi­ous­ly, the JMS mes­sage (3) would be com­mit­ted first, when DB of the re­mote call in 2 still need­ed to com­mit. Re­cip­i­ents of (3) would some­times not see the up­dates in the DB when they re­ceive the mes­sage.

This has now been fixed, by in­cor­po­rat­ing the re­mote par­tic­i­pant of step 2 in the com­mit or­der.

Changes im­pact­ing client API

None.

Bug182578

NPE when check­ing class load­ers

Sever­i­ty:	4
Af­fect­ed ver­sion(s):	5.0.x

De­scrip­tion

We now check for null when adding class load­ers to cre­ate dy­nam­ic prox­ies.

Tech­ni­cal de­tails

At least one user has re­port­ed null is­sues when cre­at­ing a dy­nam­ic proxy, like this:

java.lang.NullPointerException: null
       at java.util.ArrayDeque.addLast(ArrayDeque.java:249) ~[?:1.8.0_251]
       at java.util.ArrayDeque.add(ArrayDeque.java:423) ~[?:1.8.0_251]
       at com.atomikos.util.DynamicProxySupport.getClassLoadersToTry(DynamicProxySupport.java:194) ~[?:?]
       at com.atomikos.util.DynamicProxySupport.createDynamicProxy(DynamicProxySupport.java:189) ~[?:?]
       at com.atomikos.jdbc.internal.AtomikosXAPooledConnection.doCreateConnectionProxy(AtomikosXAPooledConnection.java:119) ~[?:?]
       at com.atomikos.jdbc.internal.AtomikosXAPooledConnection.doCreateConnectionProxy(AtomikosXAPooledConnection.java:31) ~[?:?
       at com.atomikos.datasource.pool.AbstractXPooledConnection.createConnectionProxy(AbstractXPooledConnection.java:86) ~[?:?]
       at com.atomikos.datasource.pool.ConnectionPoolWithConcurrentValidation.concurrentlyTryToUse(ConnectionPoolWithConcurrentValidation.java:61) ~[?:?]
       at com.atomikos.datasource.pool.ConnectionPoolWithConcurrentValidation.retrieveFirstAvailableConnection(ConnectionPoolWithConcurrentValidation.java:43) ~[?:?]
       at com.atomikos.datasource.pool.ConnectionPool.retrieveFirstAvailableConnectionAndGrowPoolIfNecessary(ConnectionPool.java:140) ~[?:?]
       at com.atomikos.datasource.pool.ConnectionPool.findOrWaitForAnAvailableConnection(ConnectionPool.java:128) ~[?:?]
       at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:119) ~[?:?]
       at com.atomikos.jdbc.internal.AbstractDataSourceBean.getConnection(AbstractDataSourceBean.java:371) ~[?:?]

We now fixed this by sim­ply check­ing if a class loader is not null be­fore at­tempt­ing to use it.

Changes im­pact­ing client API

None.

Bug185558

Avoid pass­ing all re­mote par­tic­i­pants as di­rect par­tic­i­pants for re­mot­ing trans­ac­tions

Sever­i­ty

2

Af­fect­ed ver­sions

5.0.x

De­scrip­tion

We used to pass all re­mote par­tic­i­pants as di­rect par­tic­i­pants, mean­ing that two-phase com­mit would some­times be re­peat­ed and fail. This would par­tic­u­lar­ly be the case in "di­a­mond" calls - or also in deep­er call hi­er­ar­chies of more than one lev­el down.

Tech­ni­cal de­tails

Con­sid­er a re­mot­ing client ser­vice A call­ing an­oth­er re­mot­ing ser­vice B.

For re­mot­ing we dis­tin­guish be­tween di­rect par­tic­i­pants (i.e., end­points at B to be in­clud­ed for two-phase com­mit at A) and in­di­rect par­tic­i­pants (i.e., URIs added as meta­da­ta only for check­ing or­phaned calls).

The class De­fault­Im­port­ingTrans­ac­tionMan­ag­er in mod­ule trans­ac­tions-re­mot­ing used to pass all its re­mote par­tic­i­pants as di­rect par­tic­i­pants, i.e. par­tic­i­pants to be in­clud­ed in the re­mot­ing's two-phase com­mit set. In par­tic­u­lar for di­a­mond-shaped calls (A calls B and C, and both B and C in turn call D) this would give re­peat­ed two-phase com­mit calls to D, be­cause A would in­cor­rect­ly re­ceive D as a di­rect par­tic­i­pant via the call hi­er­ar­chies of both B and C.

We fixed this, and for this spe­cif­ic case A would now only re­ceive 2 di­rect par­tic­i­pants: B and C.

This fix also re­solves an is­sue with a sim­pler call stack: A → B → C, where C would also be called for two-phase com­mit by both A and B. The is no longer the case.

Changes im­pact­ing client API

None.

Spe­cial thanks

Thanks to @bean­ww for re­port­ing this on GitHub.

Bug185557

Avoid app-serv­er thread leak when de­ployed in servers like Tom­cat

Sever­i­ty

3

Af­fect­ed ver­sions

4.0.x, 5.0.x

De­scrip­tion

Serv­er-side (Tom­cat) in­te­gra­tion cre­at­ed threads for in­di­vid­ual trans­ac­tions (at web app lev­el) with the class loader of the web ap­pli­ca­tion. This would cause (Tom­cat) warn­ings when the web ap­pli­ca­tion is stopped be­cause those threads will hang around in the thread pool.

So­lu­tion: we now start new threads with the serv­er-lev­el class loader.

Tech­ni­cal de­tails

If you have the trans­ac­tion core at the serv­er lev­el (in the serv­er's class­path) then in­di­vid­ual trans­ac­tions start­ed in the web ap­pli­ca­tion will use the serv­er-lev­el thread pool of Atomikos (named com.atom­ikos.thread.TaskMan­ag­er).

This would typ­i­cal­ly give warn­ings (in Tom­cat) like this when the web ap­pli­ca­tion is be­ing shut down:

WARNING [http-nio-3030-exec-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [mywebapp] appears to have started a thread named [Atomikos:2] but has failed to stop it. This is very likely to create a memory leak.

That's be­cause the serv­er-side thread pool threads used to be cre­at­ed with the web ap­pli­ca­tion's class loader.

This also hap­pened for our built-in Tom­cat in­te­gra­tion (since it is con­fig­ured at the serv­er-lev­el).

We now cre­ate those threads with the thread pool's class­loader in­stead. When con­fig­ured at the serv­er lev­el, this will no longer be the ap­pli­ca­tion's class loader and that seems to pre­vent this prob­lem.

Changes im­pact­ing client API

None.

Spe­cial thanks

Spe­cial thanks to @jjim­jam for re­port­ing this on GitHub, and thanks also to the Jet­ty - Webtide team for ex­tra in­put.

Bug189921

Avoid that ex­cep­tions when writ­ing a check­point need­less­ly cor­rupt the trans­ac­tion log

Sever­i­ty

2

Af­fect­ed ver­sions

4.0.x, 5.0.x

De­scrip­tion

You now no longer get "Log cor­rupt­ed - restart JVM" ex­cep­tions af­ter you in­ter­rupt a thread that is writ­ing to the trans­ac­tion log file, or af­ter any oth­er ex­cep­tion that make a log check­point fail.

Tech­ni­cal de­tails

Any ex­cep­tions dur­ing a check­point (such as when a thread was in­ter­rupt­ed dur­ing trans­ac­tion log file I/O) would lead to a gener­ic ex­cep­tion han­dling block in our com.atom­ikos.re­cov­ery.fs.CachedRe­pos­i­to­ry class, leav­ing the in­stance in an in­valid state:

2021-03-01 16:15:56.662 ERROR 41669 --- [pool-1-thread-1] c.a.recovery.fs.FileSystemRepository     : Failed to write checkpoint

java.nio.channels.ClosedByInterruptException: null
   at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[na:1.8.0_192]
   at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:392) ~[na:1.8.0_192]
   at com.atomikos.recovery.fs.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:196) ~[transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.recovery.fs.CachedRepository.performCheckpoint(CachedRepository.java:84) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.recovery.fs.CachedRepository.put(CachedRepository.java:77) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.recovery.fs.OltpLogImp.write(OltpLogImp.java:46) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:284) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:273) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:587) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:268) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:550) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:682) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:279) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:168) [transactions-jta-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:428) [transactions-jta-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.jta.UserTransactionManager.commit(UserTransactionManager.java:160) [transactions-jta-5.0.9-SNAPSHOT.jar:na]
   at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1035) [spring-tx-5.2.5.RELEASE.jar:5.2.5.RELEASE]
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743) [spring-tx-5.2.5.RELEASE.jar:5.2.5.RELEASE]
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711) [spring-tx-5.2.5.RELEASE.jar:5.2.5.RELEASE]
   at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:152) [spring-tx-5.2.5.RELEASE.jar:5.2.5.RELEASE]
   at com.example.atomikos.AtomikosApplicationTests.lambda$4(AtomikosApplicationTests.java:78) [test-classes/:na]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_192]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_192]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_192]
   at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_192]

Later re­quests try­ing to read from the trans­ac­tion logs would get sys­tem­at­ic cor­rup­tion er­rors like this:

com.atomikos.recovery.LogReadException: Log corrupted - restart JVM
   at com.atomikos.recovery.fs.CachedRepository.assertNotCorrupted(CachedRepository.java:137) ~[transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.recovery.fs.CachedRepository.findAllCommittingCoordinatorLogEntries(CachedRepository.java:145) ~[transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.recovery.fs.RecoveryLogImp.getExpiredPendingCommittingTransactionRecordsAt(RecoveryLogImp.java:52) ~[transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.RecoveryDomainService.performRecovery(RecoveryDomainService.java:76) ~[transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.icatch.imp.RecoveryDomainService$1.alarm(RecoveryDomainService.java:55) [transactions-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.timing.PooledAlarmTimer.notifyListeners(PooledAlarmTimer.java:101) [atomikos-util-5.0.9-SNAPSHOT.jar:na]
   at com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:88) [atomikos-util-5.0.9-SNAPSHOT.jar:na]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_192]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_192]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_192]

This has now been fixed.

Changes im­pact­ing client API

None.

About Sever­i­ty

The sever­i­ty lev­els we use are de­fined in our sup­port terms and con­di­tions.

Re­lease notes for 5.0.8

Re­lease notes for Trans­ac­tion­sEssen­tials 5.0.7

Re­lease notes for Trans­ac­tion­sEssen­tials 5.0.6

Re­lease notes for Trans­ac­tion­sEssen­tials 5.0.5