What is it?

Available as of ExtremeTransactions 5.0, the LogCloud is our answer to the needs of transaction processing projects that need to move into the cloud. Cloud applications often cannot take for granted that there will be persistent storage available. Cloud applications also need self-healing infrastructure.

The LogCloud is a cloud-enabled transaction logging and recovery service so your cloud application nodes themselves do not need any persistent storage. All transaction logging and recovery is taken care of by the LogCloud, so application nodes can come and go without affecting transaction recovery.

This enables truly elastic OLTP in the cloud, with the recovery service in the background - taking care of cleaning up any pending transactions.

Self-healing characteristics eliminate the need for administration. It was never easier to do distributed transactions!

How does it work?

A typical LogCloud deployment is shown in the figure below:

LogCloud Architecture

Let's discuss the important actors and how they collaborate:

Transactional Application Node(s)

One or more transactional application nodes use the LogCloud for maintaining (and recovering) their transaction logs, because in cloud environments application-level disk access may not be available. These application nodes could either be mirrored application deployments in a cluster, or a set of collaborating transactional REST microservices. In the former case each application node probably has the same configuration, in the latter case it concerns different microservice application nodes each with their own configuration.

Application nodes interact with the LogCloud via a shared DBMS. This means that application nodes are available even if no recovery service is running. All that is needed is availability of the DBMS where the log data is written.

LogCloud Service

Recovery service nodes connect to the LogCloud DBMS for recovery purposes. In order to work correctly, this service needs DBMS access (for log storage) and access to the same XA resources that are used by the application nodes.

XA Resource Backend(s)

The backend resources used by the application nodes are the ones that need to be recovered by the LogCloud. So it's logical that the LogCloud needs access to those. In the config sections below we'll clarify how this access is achieved.

LogCloud recovery domain

All of the above concerns one recovery domain - i.e. the set of transactions and resources that need recovery by the same LogCloud. A transaction belongs to a recovery domain if it is started by an application node that uses the LogCloud instance in that domain. By design, XA transactions have to stay within the same recovery domain, meaning that for each application node the following holds:

• all the transactions it starts are within the same recovery domain
• all resources it accesses must be made accessible to the LogCloud instance of that domain
• all other application nodes that are called within a transaction must also belong to the same recovery domain

Create a system account for the LogCloud database

1. Choose your DBMS (it needs to support JDBC)
2. Create a server instance
3. Create a user account for recovery.

This is needed for both the recovery instance and the application nodes, since both use the DBMS as shared interface. Both the recovery instance AND the application nodes need to have read/write access to all LogCloud tables, so it is best to use the same user/password combination for both. Make sure to setup this user account on your DBMS server first! The account also needs creation rights for tables and indexes (unless you want to setup the tables yourself first - not recommended).

Configuring the LogCloud recovery instance

Configuring access to the LogCloud database

This is a Spring Boot application. To configure it:

1. Create a subfolder named config/ in the folder where you want to start the app.
2. In the config folder, add a file named application.properties (we use an example with PostgreSQL below)

#JDBC driver class
spring.jta.atomikos.logcloud.driver-class-name=org.postgresql.Driver
#JDBC connect URL
spring.jta.atomikos.logcloud.url=jdbc:postgresql://localhost:5432/logcloud
#user to connect to DBMS
spring.jta.atomikos.logcloud.user=...
#password to connect to DBMS
spring.jta.atomikos.logcloud.password=...
spring.jta.atomikos.logcloud.max-pool-size=...

Start the recovery service

Start an instance of com.atomikos.logcloud.service.LogCloudRecoveryServiceApplication (a Spring Boot application):

java -jar logcloud-recovery-service-<RELEASE-VERSION>.jar 

This will create a default DBMS schema for the logging data along with default configuration settings.

Tuning your recovery configuration

Configuring recovery is done via SQL updates in the table "LOG_CLOUD_STARTUP_CONFIG" of the DBMS you selected:

• You SHOULD update the property value of com.atomikos.icatch.tm_unique_name to be unique for this installation, or recovery will interfere among different LogClouds!
• Recovery will terminate pending transactions only after com.atomikos.icatch.max_timeout milliseconds, so you should set this parameter as low as acceptable to avoid pending locks in resources.

As application nodes start and initialise their connection pools, these connections will be mirrored on the LogCloud. So there is no immediate need to configure the user/password for recovery: this will be sent over by the application nodes.

Configuring the application nodes

Each application node must be configured to log to the LogCloud instance, by doing all of the following:

• adding the transactions-logcloud-client-simple jar to the application's classpath
• setting com.atomikos.icatch.tm_unique_name to the same value of the (shared) LogCloud instance's com.atomikos.icatch.tm_unique_name value (in table "LOG_CLOUD_STARTUP_CONFIG")
• making sure that the application's com.atomikos.icatch.max_timeout setting does not exceed the same setting in the LogCloud instance

In addition, the application needs to find the LogCloud DBMS. You can use a similar mechanism as outlined above for the recovery service, or if your application does not use Spring Boot, you can configure access to the database as follows:

• Create an instance of com.atomikos.jdbc.AtomikosNonXADataSourceBean with uniqueResourceName=logCloudDS
• Initialise it before starting the transaction core

Configuring the backend resources

The backend resources need to be configured for XA transactions, but that is outside the scope of the LogCloud documentation (since that has to be done for all XA transaction uses).

Resources (i.e., Atomikos driver configurations) used by the application nodes are registered automatically in the LogCloud DBMS so the recovery service can recover them. These driver configurations are managed (inserted/updated) based on their uniqueResourceName property as the primary key identifier.

Resources are not deleted automatically, so if you ever need to delete a resource then you have to do a manual delete via the SQL interface in the LogCloud DBMS store. You should only do so if you are sure that there are no pending (prepared) XIDs in the resource. The easiest way of ensuring this is as follows:

• Remove the resource from each application node's configuration.
• Redeploy each application node.
• Wait for at least com.atomikos.icatch.max_timeout milliseconds to make sure all pending transactions are terminated in the resource being removed.
• Update the 'RESOURCES' table and delete the configuration rows for the uniqueResourceName in question.

Supported XA backends

The recovery service needs the drivers to connect to each XA backend. By default, the recovery service has the following drivers in its classpath:

• PostgreSQL
• MySQL
• ActiveMQ
• SolaceMQ

We reuse the driver versions from Spring Boot - so the concrete version will depend on what Spring Boot version used.