Introduce CompositeBackPressureHandler and related backpressure improvements

Author: loicrouchon

docs/src/main/asciidoc/sqs.adoc

@@ -806,7 +806,6 @@ NOTE: The same factory can be used to create both `single message` and `batch` c
 
 IMPORTANT: In case the same factory is shared by both delivery methods, any supplied `ErrorHandler`, `MessageInterceptor` or `MessageListener` should implement the proper methods.
 
-
 ==== Container Options
 
 Each `MessageListenerContainer` can have a different set of options.
@@ -1977,6 +1976,7 @@ If after the 5 seconds for `maxDelayBetweenPolls` 6 messages have been processed
 If the queue is depleted and a poll returns no messages, it'll enter `low throughput` mode again and perform only one poll at a time.
 
 ==== Configuring BackPressureMode
+The default `BackPressureHandler` can be configured to optimize the polling behavior based on the application's throughput requirements.
 The following `BackPressureMode` values can be set in `SqsContainerOptions` to configure polling behavior:
 
 * `AUTO` - The default mode, as described in the previous section.
@@ -1987,6 +1987,70 @@ Useful for really high throughput scenarios where the risk of making parallel po
 
 NOTE: The `AUTO` setting should be balanced for most use cases, including high throughput ones.
 
+==== Advanced Backpressure management
+
+Even though the default `BackPressureHandler` should be enough for most use cases, there are scenarios where more fine-grained control over message consumption is required not to overwhelm downstream systems or exceed resource limits.
+In such a case, it is necessary to replace the default `BackPressureHandler` with a custom one that implements the `BackPressureHandler` interface.
+A `backPressureHandlerFactory` can be set in  `SqsContainerOptions` to configure which `BackPressureHandler` to use.
+
+===== What is a BackPressureHandler?
+
+A `BackPressureHandler` is an interface that determines whether the container should apply backpressure (i.e., slow down or pause polling) based on the current state of the system.
+It is invoked before each poll to SQS and can prevent polling or poll for fewer messages if certain conditions are met, e.g., too many inflight messages, custom resource constraints, etc.
+
+===== Creating a custom BackPressureHandler
+
+To implement a custom backpressure logic, the `BackPressureHandler` interface must be implemented.
+
+A `SqsMessageListenerContainer` can be configured to use the desired  `BackPressureHandler` by setting the `backPressureHandlerFactory` on the `ContainerOptions`.
+
+```java
+SqsMessageListenerContainer container = SqsMessageListenerContainer.builder()
+    .configure(options -> options
+        .backPressureHandlerFactory(containerOptions -> new CustomBackPressureHandler())
+        // ... other options
+    )
+    // ... other container settings ...
+    .build();
+```
+
+===== Combining Multiple BackPressureHandlers
+
+If necessary, multiple `BackPressureHandler` can be combined by using the `CompositeBackPressureHandler`.
+Each of the `BackPressureHandler` (which we'll call delegates) are chained in the order they are provided.
+The first delegate will be requested the initial amount of permits and will return the number of permits it accepts to grant.
+The second delegate will get that potentially reduced number of permits as a request and might in turn reduce it further.
+The process continues until all delegates have been called or one of them returns 0, which will prevent the polling of messages from SQS.
+
+For example, to implement the `BackPressureMode.ALWAYS_POLL_MAX_MESSAGES` strategy, we can combine a concurrency limiter, an adaptative throughput handler, and a "full batch only" handler.
+The resulting `CompositeBackPressureHandler` looks like this:
+
+```java
+Duration maxIdleWaitTime = Duration.ofMillis(50L);
+List<BackPressureHandler> backPressureHandlers = List.of(
+    BackPressureHandlerFactories.concurrencyLimiterBackPressureHandler(options),
+    BackPressureHandlerFactories.throughputBackPressureHandler(options),
+    BackPressureHandlerFactories.fullBatchBackPressureHandler(options)
+);
+CompositeBackPressureHandler backPressureHandler = BackPressureHandlerFactories.compositeBackPressureHandler(
+    options, maxIdleWaitTime, backPressureHandlers);
+```
+
+===== Built-in BackPressureHandlers
+
+Spring Cloud AWS provides several built-in `BackPressureHandler` implementations:
+
+- `ConcurrencyLimiterBackPressureHandler`: Limits the number of messages being processed concurrently.
+- `ThroughputBackPressureHandler`: Switches between high and low throughput modes. In high throughput mode, multiple polls can be done in parallel.
+In low throughput mode, only one poll is done at a time.
+- `FullBatchBackPressureHandler`: Ensure polls will always be done with a full batch of messages, meaning that the number of messages polled will always be equal to `maxMessagesPerPoll` if possible or `0` if not possible.
+This `FullBatchBackPressureHandler` must always be the last in the chain for it to work properly.
+
+The `BackPressureHandlerFactories` class provides factory methods to create these handlers easily.
+These handlers can be used directly or combined with custom ones using the `CompositeBackPressureHandler` to fit the application's needs.
+
+Additionally, the `BackPressureHandlerFactories#adaptativeThroughputBackPressureHandler` factory method combines the `ConcurrencyLimiterBackPressureHandler`, `ThroughputBackPressureHandler`, and `FullBatchBackPressureHandler` as per the desired `BackPressureMode`.
+
 === Blocking and Non-Blocking (Async) Components
 
 The SQS integration leverages the `CompletableFuture`-based async capabilities of `AWS SDK 2.0` to deliver a fully non-blocking infrastructure.

spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractContainerOptions.java

@@ -32,6 +32,7 @@
  * Base implementation for {@link ContainerOptions}.
  *
  * @author Tomaz Fernandes
+ * @author Loïc Rouchon
  * @since 3.0
  */
 public abstract class AbstractContainerOptions<O extends ContainerOptions<O, B>, B extends ContainerOptionsBuilder<B, O>>
@@ -55,6 +56,8 @@ public abstract class AbstractContainerOptions<O extends ContainerOptions<O, B>,
 
 	private final BackPressureMode backPressureMode;
 
+	private final BackPressureHandlerFactory backPressureHandlerFactory;
+
 	private final ListenerMode listenerMode;
 
 	private final MessagingMessageConverter<?> messageConverter;
@@ -90,6 +93,7 @@ protected AbstractContainerOptions(Builder<?, ?> builder) {
 		this.listenerShutdownTimeout = builder.listenerShutdownTimeout;
 		this.acknowledgementShutdownTimeout = builder.acknowledgementShutdownTimeout;
 		this.backPressureMode = builder.backPressureMode;
+		this.backPressureHandlerFactory = builder.backPressureHandlerFactory;
 		this.listenerMode = builder.listenerMode;
 		this.messageConverter = builder.messageConverter;
 		this.acknowledgementMode = builder.acknowledgementMode;
@@ -162,6 +166,11 @@ public BackPressureMode getBackPressureMode() {
 		return this.backPressureMode;
 	}
 
+	@Override
+	public BackPressureHandlerFactory getBackPressureHandlerFactory() {
+		return this.backPressureHandlerFactory;
+	}
+
 	@Override
 	public ListenerMode getListenerMode() {
 		return this.listenerMode;
@@ -232,6 +241,8 @@ protected abstract static class Builder<B extends ContainerOptionsBuilder<B, O>,
 
 		private static final BackPressureMode DEFAULT_THROUGHPUT_CONFIGURATION = BackPressureMode.AUTO;
 
+		private static final BackPressureHandlerFactory DEFAULT_BACKPRESSURE_FACTORY = BackPressureHandlerFactories::semaphoreBackPressureHandler;
+
 		private static final ListenerMode DEFAULT_MESSAGE_DELIVERY_STRATEGY = ListenerMode.SINGLE_MESSAGE;
 
 		private static final MessagingMessageConverter<?> DEFAULT_MESSAGE_CONVERTER = new SqsMessagingMessageConverter();
@@ -254,6 +265,8 @@ protected abstract static class Builder<B extends ContainerOptionsBuilder<B, O>,
 
 		private BackPressureMode backPressureMode = DEFAULT_THROUGHPUT_CONFIGURATION;
 
+		private BackPressureHandlerFactory backPressureHandlerFactory = DEFAULT_BACKPRESSURE_FACTORY;
+
 		private Duration listenerShutdownTimeout = DEFAULT_LISTENER_SHUTDOWN_TIMEOUT;
 
 		private Duration acknowledgementShutdownTimeout = DEFAULT_ACKNOWLEDGEMENT_SHUTDOWN_TIMEOUT;
@@ -296,6 +309,7 @@ protected Builder(AbstractContainerOptions<?, ?> options) {
 			this.listenerShutdownTimeout = options.listenerShutdownTimeout;
 			this.acknowledgementShutdownTimeout = options.acknowledgementShutdownTimeout;
 			this.backPressureMode = options.backPressureMode;
+			this.backPressureHandlerFactory = options.backPressureHandlerFactory;
 			this.listenerMode = options.listenerMode;
 			this.messageConverter = options.messageConverter;
 			this.acknowledgementMode = options.acknowledgementMode;
@@ -390,6 +404,12 @@ public B backPressureMode(BackPressureMode backPressureMode) {
 			return self();
 		}
 
+		@Override
+		public B backPressureHandlerFactory(BackPressureHandlerFactory backPressureHandlerFactory) {
+			this.backPressureHandlerFactory = backPressureHandlerFactory;
+			return self();
+		}
+
 		@Override
 		public B acknowledgementInterval(Duration acknowledgementInterval) {
 			Assert.notNull(acknowledgementInterval, "acknowledgementInterval cannot be null");

spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/AbstractPipelineMessageListenerContainer.java

@@ -230,10 +230,9 @@ private TaskExecutor validateCustomExecutor(TaskExecutor taskExecutor) {
 	}
 
 	protected BackPressureHandler createBackPressureHandler() {
-		return SemaphoreBackPressureHandler.builder().batchSize(getContainerOptions().getMaxMessagesPerPoll())
-				.totalPermits(getContainerOptions().getMaxConcurrentMessages())
-				.acquireTimeout(getContainerOptions().getMaxDelayBetweenPolls())
-				.throughputConfiguration(getContainerOptions().getBackPressureMode()).build();
+		O containerOptions = getContainerOptions();
+		BackPressureHandlerFactory factory = containerOptions.getBackPressureHandlerFactory();
+		return factory.createBackPressureHandler(containerOptions);
 	}
 
 	protected TaskExecutor createSourcesTaskExecutor() {

spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/listener/BackPressureHandler.java

@@ -24,25 +24,55 @@
  * semaphore-based, rate limiter-based, a mix of both, or any other.
  *
  * @author Tomaz Fernandes
+ * @author Loïc Rouchon
  * @since 3.0
  */
 public interface BackPressureHandler {
 
 	/**
-	 * Request a number of permits. Each obtained permit allows the
+	 * Requests a number of permits. Each obtained permit allows the
 	 * {@link io.awspring.cloud.sqs.listener.source.MessageSource} to retrieve one message.
 	 * @param amount the amount of permits to request.
 	 * @return the amount of permits obtained.
 	 * @throws InterruptedException if the Thread is interrupted while waiting for permits.
 	 */
 	int request(int amount) throws InterruptedException;
 
+	/**
+	 * Releases the specified amount of permits for processed messages. Each message that has been processed should
+	 * release one permit, whether processing was successful or not.
+	 * <p>
+	 * This method can be called in the following use cases:
+	 * <ul>
+	 * <li>{@link ReleaseReason#LIMITED}: all/some permits were not used because another BackPressureHandler has a lower
+	 * permits limit and the difference in permits needs to be returned.</li>
+	 * <li>{@link ReleaseReason#NONE_FETCHED}: none of the permits were actually used because no messages were retrieved
+	 * from SQS. Permits need to be returned.</li>
+	 * <li>{@link ReleaseReason#PARTIAL_FETCH}: some of the permits were used (some messages were retrieved from SQS).
+	 * The unused ones need to be returned. The amount to be returned might be {@literal 0}, in which case it means all
+	 * the permits will be used as the same number of messages were fetched from SQS.</li>
+	 * <li>{@link ReleaseReason#PROCESSED}: a message processing finished, successfully or not.</li>
+	 * </ul>
+	 * @param amount the amount of permits to release.
+	 * @param reason the reason why the permits were released.
+	 */
+	default void release(int amount, ReleaseReason reason) {
+		release(amount);
+	}
+
 	/**
 	 * Release the specified amount of permits. Each message that has been processed should release one permit, whether
 	 * processing was successful or not.
 	 * @param amount the amount of permits to release.
+	 *
+	 * @deprecated This method is deprecated and will not be called by the Spring Cloud AWS SQS listener anymore.
+	 * Implement {@link #release(int, ReleaseReason)} instead.
 	 */
-	void release(int amount);
+	@Deprecated
+	default void release(int amount) {
+		// Do not implement this method. It is not called anymore outside of backward compatibility use cases.
+		// Implement `#release(int amount, ReleaseReason reason)` instead.
+	}
 
 	/**
 	 * Attempts to acquire all permits up to the specified timeout. If successful, means all permits were returned and
@@ -52,4 +82,24 @@ public interface BackPressureHandler {
 	 */
 	boolean drain(Duration timeout);
 
+	enum ReleaseReason {
+		/**
+		 * All/Some permits were not used because another BackPressureHandler has a lower permits limit and the permits
+		 * difference need to be aligned across all handlers.
+		 */
+		LIMITED,
+		/**
+		 * No messages were retrieved from SQS, so all permits need to be returned.
+		 */
+		NONE_FETCHED,
+		/**
+		 * Some messages were fetched from SQS. Unused permits if any need to be returned.
+		 */
+		PARTIAL_FETCH,
+		/**
+		 * The processing of one or more messages finished, successfully or not.
+		 */
+		PROCESSED;
+	}
+
 }