Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .github/workflows/development.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,6 @@ jobs:
- name: Build with Maven
run: ./mvnw clean verify -U -B -ntp -T4

# itest
- name: Run itest
run: ./mvnw integration-test failsafe:verify -Pitest -ntp -U -B -T4


# - name: Upload coverage to Codecov
# if: github.event_name == 'push' && github.actor != 'dependabot[bot]'
# uses: codecov/codecov-action@v1.0.2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:

# Publish release
- name: Deploy a new release version to Maven Central
run: ./mvnw clean deploy -B -ntp -DskipTests -DskipExamples -Prelease -Dgpg.keyname="${{ secrets.GPG_KEYNAME }}"
run: ./mvnw clean deploy -B -ntp -DskipTests -DskipITests -DskipExamples -Prelease -Dgpg.keyname="${{ secrets.GPG_KEYNAME }}"
env:
OSS_CENTRAL_USERNAME: ${{ secrets.SONATYPE_USERNAME }}
OSS_CENTRAL_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
Expand Down
98 changes: 98 additions & 0 deletions docs/idempotency.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
title: Idempotency Registry
---

The Idempotency Registry is a feature designed to prevent duplicate worker invocations. It ensures that if a task is processed multiple times (e.g., due to network issues or retries in the process engine), the worker logic is only executed once, and the previous result is returned for subsequent calls.

## How it Works

When a worker is triggered, the process engine worker:
1. Checks the `IdempotencyRegistry` if a result already exists for the given `taskId`.
2. If a result exists, it skips the worker execution and returns the stored result.
3. If no result exists, it executes the worker.
4. After successful execution, it registers the result in the `IdempotencyRegistry`.

## Implementations

There are three available implementations of the `IdempotencyRegistry`:

| Implementation | Description | Recommended Use |
| --- | --- | --- |
| `NoOpIdempotencyRegistry` | Does nothing. No results are stored or retrieved. | Default, use if idempotency is handled elsewhere. |
| `InMemoryIdempotencyRegistry` | Stores results in a local `ConcurrentHashMap`. | Testing or non-clustered environments. |
| `JpaIdempotencyRegistry` | Stores results in a database using JPA. | Production, clustered environments. |

## Setup Procedures

### In-Memory Registry

To use the in-memory registry, you need to provide a bean of type `IdempotencyRegistry` in your Spring configuration:

```kotlin
@Configuration
class IdempotencyConfiguration {

@Bean
fun idempotencyRegistry(): IdempotencyRegistry = InMemoryIdempotencyRegistry()

}
```

> **Warning:** The `InMemoryIdempotencyRegistry` is not suitable for clustered environments as the state is not shared between nodes.

### JPA-based Registry

The JPA-based registry is suitable for production environments. It persists the results in the database, allowing multiple instances of the worker to share the same idempotency state.

#### 1. Add Dependency

Add the following dependency to your `pom.xml`:

```xml
<dependency>
<groupId>dev.bpm-crafters.process-engine-worker</groupId>
<artifactId>process-engine-worker-spring-boot-idempotency-registry-jpa</artifactId>
<version>${process-engine-worker.version}</version>
</dependency>
```

The `JpaIdempotencyAutoConfiguration` will automatically register the `JpaIdempotencyRegistry` if an `EntityManager` is present and no other `IdempotencyRegistry` bean is defined.

#### 2. Database Schema (Liquibase)

The JPA registry requires a table named `task_log_entry_`. You can use the following Liquibase changeSet to create it:

```yaml
databaseChangeLog:
- changeSet:
id: create-idempotency-table
author: bpm-crafters
changes:
- createTable:
tableName: task_log_entry_
columns:
- column:
name: task_id_
type: varchar(100)
constraints:
nullable: false
primaryKey: true
primaryKeyName: task_log_entry_pk_
- column:
name: process_instance_id_
type: varchar(100)
constraints:
nullable: false
- column:
name: created_at_
type: timestamp
constraints:
nullable: false
- column:
name: result_
type: blob # or bytea for PostgreSQL
constraints:
nullable: false
```

> **Note:** The `result_` column type should be suitable for storing binary data (e.g., `blob` for most databases, `bytea` for PostgreSQL).
9 changes: 9 additions & 0 deletions docs/process-engine-worker.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,15 @@ public class MySmartWorker {
}
```

The `@ProcessEngineWorker` annotation supports the following properties:

| Property | Type | Default | Description |
|----------------|------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `topic` | `String` | `""` | Topic name to subscribe this worker for. Alias for `value`. |
| `autoComplete` | `boolean` | `true` | Flag indicating if the task should be automatically completed after the worker execution. If the return type is `Map<String, Any>`, it will overrule this setting and auto-complete with the returned payload. |
| `completion` | `enum` | `DEFAULT` | Configures when the worker completes a task if `autoComplete` is active. Possible values are `DEFAULT`, `BEFORE_COMMIT`, and `AFTER_COMMIT`. Has no effect if the worker is not transactional. |
| `lockDuration` | `long` | `-1` | Optional lock duration in seconds for this worker. If set to `-1` (default), the global configuration of the process engine adapter will be used. (Available since `0.8.0`) |

## Method parameter resolution

Parameter resolution of the method annotated with `ProcessEngineWorker` is based on a set of strategies
Expand Down
43 changes: 43 additions & 0 deletions itest/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>dev.bpm-crafters.process-engine-worker</groupId>
<artifactId>process-engine-worker-root</artifactId>
<version>0.7.2-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>process-engine-worker-itest</artifactId>
<packaging>pom</packaging>
<description>ITest: Parent for the integration testing.</description>

<properties>
<!-- Never deploy integration testing -->
<deploy.skip>true</deploy.skip>
<gpg.skip>true</gpg.skip>
<dokka.skip>true</dokka.skip>
</properties>

<modules>
<module>spring-boot-integration-testing</module>
<module>spring-boot-starter-integration-test</module>
</modules>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>dev.bpm-crafters.process-engine-worker</groupId>
<artifactId>process-engine-worker-spring-boot-starter</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>dev.bpm-crafters.process-engine-worker</groupId>
<artifactId>process-engine-worker-spring-boot-idempotency-registry-jpa</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

</project>
149 changes: 149 additions & 0 deletions itest/spring-boot-integration-testing/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>dev.bpm-crafters.process-engine-worker</groupId>
<artifactId>process-engine-worker-itest</artifactId>
<version>0.7.2-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

<artifactId>process-engine-worker-integration-testing</artifactId>
<description>ITest: Testing utilities for integration testing of the library.</description>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-bom</artifactId>
<version>2.0.3</version>
<scope>import</scope>
<type>pom</type>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<scope>import</scope>
<type>pom</type>
</dependency>

<dependency>
<groupId>org.mockito.kotlin</groupId>
<artifactId>mockito-kotlin</artifactId>
<version>${mockito.version}</version>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>process-engine-worker-spring-boot-starter</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
<groupId>dev.bpm-crafters.process-engine-adapters</groupId>
<artifactId>process-engine-adapter-camunda-platform-c7-remote-spring-boot-starter</artifactId>
<version>${process-engine-adapters-c7.version}</version>
</dependency>
<dependency>
<groupId>io.holunda.c7</groupId>
<artifactId>c7-rest-client-spring-boot-starter-feign</artifactId>
<version>${c7.version}</version>
</dependency>
<dependency>
<groupId>org.camunda.bpm.springboot</groupId>
<artifactId>camunda-bpm-spring-boot-starter-external-task-client</artifactId>
<version>${camunda-bpm-spring-boot-starter-external-task-client.version}</version>
</dependency>

<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
</dependency>
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility-kotlin</artifactId>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-test-junit5</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
</dependency>
<dependency>
<groupId>org.mockito.kotlin</groupId>
<artifactId>mockito-kotlin</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>postgresql</artifactId>
</dependency>
</dependencies>

</project>
Loading