Skip to content

Commit 5e3ea66

Browse files
committed
Update documentation to include manual configuration instructions for SQS automatic request batching.
1 parent 9f043a3 commit 5e3ea66

File tree

2 files changed

+63
-112
lines changed

2 files changed

+63
-112
lines changed

docs/src/main/asciidoc/sqs.adoc

Lines changed: 51 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -812,124 +812,87 @@ Spring Cloud AWS supports automatic message batching using AWS SDK's `SqsAsyncBa
812812

813813
IMPORTANT: This is different from the <<Batch Processing,Batch Processing>> feature described above. Batch Processing refers to processing multiple messages in a single listener method call, while Automatic Batching refers to the AWS SDK automatically combining multiple SQS API calls into batched requests for efficiency.
814814

815-
===== Enabling Automatic Batching
815+
==== Automatic Request Batching with SqsAsyncBatchManager
816816

817-
To enable automatic batching, set the following property:
817+
Spring Cloud AWS allows you to leverage the AWS SDK's `SqsAsyncBatchManager` for automatic request batching. This feature can significantly improve performance and reduce costs by transparently combining multiple SQS API calls (`sendMessage`, `deleteMessage`, etc.) into single batch requests.
818818

819-
[source,properties]
820-
----
821-
spring.cloud.aws.sqs.batch.enabled=true
822-
----
823-
824-
When enabled, Spring Cloud AWS will automatically wrap the `SqsAsyncClient` with a `BatchingSqsClientAdapter` that uses `SqsAsyncBatchManager` internally.
825-
826-
===== Configuration Properties
827-
828-
The following properties can be used to configure the batching behavior:
829-
830-
[source,properties]
831-
----
832-
# Enable automatic batching (default: false)
833-
spring.cloud.aws.sqs.batch.enabled=true
834-
835-
# Maximum number of messages in a batch (default: AWS SDK default, max: 10)
836-
spring.cloud.aws.sqs.batch.max-number-of-messages=10
819+
IMPORTANT: This is different from the <<Batch Processing,Batch Processing>> feature for `@SqsListener`. Listener batch processing deals with handling multiple messages within a single listener invocation, whereas automatic request batching optimizes the underlying API calls to AWS.
837820

838-
# Frequency at which batched requests are sent (default: AWS SDK default)
839-
spring.cloud.aws.sqs.batch.send-batch-frequency=PT0.2S
821+
===== Manual Configuration of the Batching Client
840822

841-
# Visibility timeout for received messages (default: queue default)
842-
spring.cloud.aws.sqs.batch.visibility-timeout=PT30S
823+
Since automatic batching is a powerful feature with specific trade-offs, Spring Cloud AWS does not auto-configure it. You can enable it by creating your own `SqsAsyncClient` bean using the provided `BatchingSqsClientAdapter`.
843824

844-
# Wait time for receiveMessage requests (default: AWS SDK default)
845-
spring.cloud.aws.sqs.batch.wait-time-seconds=PT5S
825+
###### 1. Defining the Batching Client Bean
826+
The following example shows how to define a bean named `batchingSqsAsyncClient`. Notice the use of `@Qualifier("sqsAsyncClient")` in the method parameter. This is crucial to explicitly inject the standard, auto-configured `SqsAsyncClient` and avoid ambiguity.
846827

847-
# System attributes to request for receiveMessage calls
848-
spring.cloud.aws.sqs.batch.system-attribute-names=SentTimestamp,ApproximateReceiveCount
849-
850-
# Message attributes to request for receiveMessage calls
851-
spring.cloud.aws.sqs.batch.attribute-names=MessageGroupId,MessageDeduplicationId
828+
[source,java]
852829
----
830+
@Configuration
831+
public class SqsBatchingConfiguration {
853832
854-
===== Important Considerations
855-
856-
WARNING: When using automatic batching, operations are processed asynchronously by the AWS SDK. This means that a method call may return successfully, but the actual request to AWS SQS might fail later during the batching process. This can result in **false positives** where operations appear to succeed locally but fail during transmission.
857-
858-
Applications should:
859-
860-
- **Always handle the returned `CompletableFuture`** to detect actual transmission errors
861-
- **Implement appropriate error handling and monitoring** to detect delayed failures
862-
- **Consider retry mechanisms** for critical operations
863-
864-
IMPORTANT: **Batch Manager Bypass**: The AWS SDK batch manager will bypass batching and send regular asynchronous requests when `receiveMessage` is called with any of the following parameters:
833+
// Define a constant for the bean name to avoid typos
834+
public static final String BATCHING_SQS_ASYNC_CLIENT = "batchingSqsAsyncClient";
865835
866-
- `messageAttributeNames`
867-
- `messageSystemAttributeNames`
868-
- `messageSystemAttributeNamesWithStrings` (not used in Spring Cloud AWS `ReceiveMessageRequest`)
869-
- `overrideConfiguration` (not used in Spring Cloud AWS `ReceiveMessageRequest`)
836+
@Bean(name = BATCHING_SQS_ASYNC_CLIENT)
837+
public SqsAsyncClient batchingSqsAsyncClient(
838+
// Inject the standard, auto-configured client to wrap it
839+
@Qualifier("sqsAsyncClient") SqsAsyncClient standardSqsAsyncClient) {
870840
871-
When these parameters are used, the performance benefits of batching are lost for those specific requests.
841+
ScheduledExecutorService scheduledExecutor = Executors.newScheduledThreadPool(5);
872842
873-
**Note**: When using Spring Cloud AWS's automatic batching feature, `SqsTemplate` automatically excludes `messageAttributeNames` and `messageSystemAttributeNames` from individual `receiveMessage` requests to maintain batching efficiency. These attributes should be configured globally in the batch configuration instead:
843+
SqsAsyncBatchManager batchManager = SqsAsyncBatchManager.builder()
844+
.sqsAsyncClient(standardSqsAsyncClient)
845+
.scheduledExecutor(scheduledExecutor)
846+
.build();
874847
875-
[source,properties]
876-
----
877-
# Configure globally for batched requests
878-
spring.cloud.aws.sqs.batch.system-attribute-names=SentTimestamp,ApproximateReceiveCount
879-
spring.cloud.aws.sqs.batch.attribute-names=MessageGroupId,MessageDeduplicationId
848+
return new BatchingSqsClientAdapter(batchManager);
849+
}
850+
}
880851
----
881852

882-
If you need to use different attribute configurations per request, consider disabling automatic batching and using the standard `SqsAsyncClient` instead.
883-
884-
Example of proper error handling:
853+
###### 2. Using the Batching Client
854+
Now, use `@Qualifier` to inject your named bean. The most common use case is configuring a dedicated `SqsTemplate`.
885855

886856
[source,java]
887857
----
888858
@Service
889-
public class MessageService {
890-
891-
private final SqsTemplate sqsTemplate;
859+
public class MyBatchingMessageService {
892860
893-
public MessageService(SqsTemplate sqsTemplate) {
894-
this.sqsTemplate = sqsTemplate;
895-
}
861+
private final SqsTemplate batchingSqsTemplate;
896862
897-
public void sendMessage(String queueName, String message) {
898-
CompletableFuture<SendResult<String>> future = sqsTemplate.sendAsync(queueName, message);
899-
900-
future.whenComplete((result, throwable) -> {
901-
if (throwable != null) {
902-
// Handle actual transmission error
903-
log.error("Failed to send message to queue {}: {}", queueName, throwable.getMessage());
904-
// Implement retry or alternative handling logic
905-
} else {
906-
// Message sent successfully
907-
log.info("Message sent successfully with ID: {}", result.messageId());
908-
}
909-
});
863+
public MyBatchingMessageService(
864+
@Qualifier(SqsBatchingConfiguration.BATCHING_SQS_ASYNC_CLIENT) SqsAsyncClient batchingClient) {
865+
this.batchingSqsTemplate = SqsTemplate.builder()
866+
.sqsAsyncClient(batchingClient)
867+
.build();
910868
}
869+
// ... service methods using batchingSqsTemplate
911870
}
912871
----
913872

914-
===== Performance Benefits
873+
===== Important Considerations & Best Practices
915874

916-
Automatic batching provides several benefits:
875+
WARNING: **Asynchronous Operations and False Positives**. The batching client processes operations asynchronously. A call to `sqsTemplate.sendAsync(...)` might return a `CompletableFuture` that completes successfully before the message is actually sent to AWS. The actual transmission happens later in a background thread. This can lead to **false positives**. Always attach error handling to the `CompletableFuture` to detect and handle real transmission failures.
917876

918-
- **Reduced API calls**: Multiple operations are combined into single API calls
919-
- **Lower costs**: Fewer API calls result in reduced AWS charges
920-
- **Improved throughput**: Batching reduces network overhead and latency
921-
- **Better resource utilization**: More efficient use of network and AWS resources
877+
[source,java]
878+
----
879+
CompletableFuture<SendResult<String>> future = batchingSqsTemplate.sendAsync(queueName, message);
922880
923-
===== Compatibility
881+
future.whenComplete((result, ex) -> {
882+
if (ex != null) {
883+
// This is where you handle the actual transmission error
884+
log.error("Failed to send message to queue {}: {}", queueName, ex.getMessage());
885+
} else {
886+
log.info("Message acknowledged for batch sending with ID: {}", result.messageId());
887+
}
888+
});
889+
----
924890

925-
Automatic batching is compatible with:
891+
WARNING: **Not Recommended for `@SqsListener`**. While technically compatible, using this batching client with `@SqsListener` for receiving messages is **not recommended**. The `@SqsListener` infrastructure already performs efficient batch receiving and has a complex acknowledgment lifecycle. Adding another layer of asynchronous batching provides limited performance benefits while significantly increasing complexity. For listeners, it's best to rely on the default `SqsAsyncClient`.
926892

927-
- `SqsTemplate` for sending and receiving messages
928-
- `@SqsListener` methods for message processing
929-
- Both standard and FIFO queues
930-
- All message conversion and error handling features
893+
IMPORTANT: **Bean Injection Safety**. By using a named bean and `@Qualifier` as shown in the configuration examples, you ensure the batching client is only used where intended. This prevents it from accidentally being injected into `@SqsListener` infrastructure, which should use the default `SqsAsyncClient`.
931894

932-
The batching is transparent to application code - existing `SqsTemplate` and `@SqsListener` code continues to work without changes.
895+
IMPORTANT: **AWS SDK Batching Bypass**. The `SqsAsyncBatchManager` will bypass batching for `receiveMessage` calls if certain parameters like `messageAttributeNames` are set on a per-request basis. To ensure batching works effectively, these should be configured globally on the `SqsAsyncBatchManager` builder, not on individual `receiveMessage` calls. See the `BatchingSqsClientAdapter` Javadoc for more details.
933896

934897
==== Container Options
935898

spring-cloud-aws-sqs/src/main/java/io/awspring/cloud/sqs/operations/BatchingSqsClientAdapter.java

Lines changed: 12 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -43,33 +43,21 @@
4343
* </ul>
4444
*
4545
* <p>
46-
* <strong>Batch Optimization:</strong> The AWS SDK bypasses batching when {@code receiveMessage} is called with any of
47-
* the following parameters: {@code messageAttributeNames}, {@code messageSystemAttributeNames},
48-
* {@code messageSystemAttributeNamesWithStrings}, or {@code overrideConfiguration}. To maintain consistent batching
49-
* performance, Spring Cloud AWS handles these parameters as follows:
46+
* <strong>Batch Optimization:</strong> The underlying {@code SqsAsyncBatchManager} from the AWS SDK bypasses
47+
* batching for {@code receiveMessage} calls that include per-request configurations for certain parameters.
48+
* To ensure batching is not bypassed, it is recommended to configure these settings globally on the
49+
* {@code SqsAsyncBatchManager} builder instead of on each {@code ReceiveMessageRequest}.
50+
* The parameters that trigger this bypass are:
5051
* <ul>
51-
* <li>{@code messageAttributeNames} - excluded from per-request, configured globally via
52-
* {@code spring.cloud.aws.sqs.batch.attribute-names}</li>
53-
* <li>{@code messageSystemAttributeNames} - excluded from per-request, configured globally via
54-
* {@code spring.cloud.aws.sqs.batch.system-attribute-names}</li>
55-
* <li>{@code messageSystemAttributeNamesWithStrings} - not used in Spring Cloud AWS {@code ReceiveMessageRequest}</li>
56-
* <li>{@code overrideConfiguration} - not used in Spring Cloud AWS {@code ReceiveMessageRequest}</li>
52+
* <li>{@code messageAttributeNames}</li>
53+
* <li>{@code messageSystemAttributeNames}</li>
54+
* <li>{@code messageSystemAttributeNamesWithStrings}</li>
55+
* <li>{@code overrideConfiguration}</li>
5756
* </ul>
5857
* <p>
59-
* This design prevents batch bypass and ensures optimal performance. If per-request attribute configuration is
60-
* required, consider disabling automatic batching.
61-
*
62-
* <p>
63-
* This adapter is automatically configured by Spring Cloud AWS when automatic batching is enabled. Users do not need to
64-
* create instances directly - instead, enable batching through configuration:
65-
*
66-
* <pre>
67-
* spring.cloud.aws.sqs.batch.enabled = true
68-
* </pre>
69-
*
70-
* <p>
71-
* Once enabled, all {@code SqsTemplate} operations will automatically use batching transparently.
72-
*
58+
* By configuring these globally on the manager, you ensure consistent batching performance. If you require
59+
* per-request attribute configurations, using the standard {@code SqsAsyncClient} without this adapter may be
60+
* more appropriate.
7361
* @author Heechul Kang
7462
* @since 3.2
7563
* @see SqsAsyncBatchManager

0 commit comments

Comments
 (0)