Skip to content

Commit fe356c2

Browse files
abhu85claude
authored andcommitted
docs: remove unimplemented ItemBatcher documentation from map.md
Fixes aws#304 The ItemBatcher feature is not yet implemented and is reserved for future use. This commit removes all references to batching functionality from the map operations documentation to avoid confusing users. Changes: - Remove "Item batching" terminology definition - Remove "Batching support" from Key features - Remove "Batching items" section from Advanced patterns - Remove ItemBatcher from Configuration example and MapConfig parameters - Remove batching-related FAQ entry - Update Best practices and Performance tips to remove batching references - Update method signature to remove BatchedInput type hint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 46c3ffd commit fe356c2

File tree

1 file changed

+11
-62
lines changed

1 file changed

+11
-62
lines changed

docs/core/map.md

Lines changed: 11 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,6 @@
2828

2929
**Concurrency control** - Limiting how many items process simultaneously using `max_concurrency` in `MapConfig`.
3030

31-
**Item batching** - Grouping multiple items together for processing as a single unit to optimize efficiency.
32-
3331
**Completion criteria** - Rules that determine when a map operation succeeds or fails based on individual item results.
3432

3533
[↑ Back to top](#table-of-contents)
@@ -42,7 +40,7 @@ Use map operations to:
4240
- Transform collections with automatic checkpointing
4341
- Process lists of items in parallel
4442
- Handle large datasets with resilience
45-
- Control concurrency and batching behavior
43+
- Control concurrency behavior
4644
- Define custom success/failure criteria
4745

4846
Map operations use `context.map()` to process collections efficiently. Each item becomes an independent operation that executes in parallel with other items.
@@ -55,7 +53,6 @@ Map operations use `context.map()` to process collections efficiently. Each item
5553
- **Independent checkpointing** - Each item's result is saved separately
5654
- **Partial completion** - Completed items don't reprocess on replay
5755
- **Concurrency control** - Limit simultaneous processing with `max_concurrency`
58-
- **Batching support** - Group items for efficient processing
5956
- **Flexible completion** - Define custom success/failure criteria
6057
- **Result ordering** - Results maintain the same order as inputs
6158

@@ -102,7 +99,7 @@ If the function is interrupted after processing items 0-2, it resumes at item 3
10299
```python
103100
def map(
104101
inputs: Sequence[U],
105-
func: Callable[[DurableContext, U | BatchedInput[Any, U], int, Sequence[U]], T],
102+
func: Callable[[DurableContext, U, int, Sequence[U]], T],
106103
name: str | None = None,
107104
config: MapConfig | None = None,
108105
) -> BatchResult[T]
@@ -113,7 +110,7 @@ def map(
113110
- `inputs` - A sequence of items to process (list, tuple, or any sequence type).
114111
- `func` - A callable that processes each item. See [Map function signature](#map-function-signature) for details.
115112
- `name` (optional) - A name for the map operation, useful for debugging and testing.
116-
- `config` (optional) - A `MapConfig` object to configure concurrency, batching, and completion criteria.
113+
- `config` (optional) - A `MapConfig` object to configure concurrency and completion criteria.
117114

118115
**Returns:** A `BatchResult[T]` containing the results from processing all items.
119116

@@ -128,7 +125,7 @@ The map function receives four parameters:
128125
```python
129126
def process_item(
130127
context: DurableContext,
131-
item: U | BatchedInput[Any, U],
128+
item: U,
132129
index: int,
133130
items: Sequence[U]
134131
) -> T:
@@ -140,7 +137,7 @@ def process_item(
140137
**Parameters:**
141138

142139
- `context` - A `DurableContext` for the item's processing. Use this to call steps, waits, or other operations.
143-
- `item` - The current item being processed. Can be a `BatchedInput` if batching is configured.
140+
- `item` - The current item being processed.
144141
- `index` - The zero-based index of the item in the original collection.
145142
- `items` - The full collection of items being processed.
146143

@@ -186,7 +183,6 @@ from aws_durable_execution_sdk_python import (
186183
from aws_durable_execution_sdk_python.config import (
187184
MapConfig,
188185
CompletionConfig,
189-
ItemBatcher,
190186
)
191187

192188
def process_item(context: DurableContext, item: int, index: int, items: list[int]) -> dict:
@@ -196,14 +192,13 @@ def process_item(context: DurableContext, item: int, index: int, items: list[int
196192
@durable_execution
197193
def handler(event: dict, context: DurableContext) -> BatchResult[dict]:
198194
items = list(range(100))
199-
195+
200196
# Configure map operation
201197
config = MapConfig(
202198
max_concurrency=10, # Process 10 items at a time
203-
item_batcher=ItemBatcher(max_items_per_batch=5), # Batch 5 items together
204199
completion_config=CompletionConfig.all_successful(), # Require all to succeed
205200
)
206-
201+
207202
result = context.map(items, process_item, name="process_numbers", config=config)
208203
return result
209204
```
@@ -212,8 +207,6 @@ def handler(event: dict, context: DurableContext) -> BatchResult[dict]:
212207

213208
**max_concurrency** - Maximum number of items to process concurrently. If `None`, all items process in parallel. Use this to control resource usage.
214209

215-
**item_batcher** - Configuration for batching items together. Use `ItemBatcher(max_items_per_batch=N)` to group items.
216-
217210
**completion_config** - Defines when the map operation succeeds or fails:
218211
- `CompletionConfig()` - Default, allows any number of failures
219212
- `CompletionConfig.all_successful()` - Requires all items to succeed
@@ -254,36 +247,6 @@ def handler(event: dict, context: DurableContext) -> BatchResult[dict]:
254247
return result
255248
```
256249

257-
### Batching items
258-
259-
Group multiple items for efficient processing:
260-
261-
```python
262-
from aws_durable_execution_sdk_python.config import MapConfig, ItemBatcher, BatchedInput
263-
264-
def process_batch(
265-
context: DurableContext,
266-
batch: BatchedInput[None, int],
267-
index: int,
268-
items: list[int]
269-
) -> list[dict]:
270-
"""Process a batch of items together."""
271-
# Process all items in the batch together
272-
return [{"item": item, "squared": item * item} for item in batch.items]
273-
274-
@durable_execution
275-
def handler(event: dict, context: DurableContext) -> BatchResult[list[dict]]:
276-
items = list(range(100))
277-
278-
# Process items in batches of 10
279-
config = MapConfig(
280-
item_batcher=ItemBatcher(max_items_per_batch=10)
281-
)
282-
283-
result = context.map(items, process_batch, config=config)
284-
return result
285-
```
286-
287250
### Custom completion criteria
288251

289252
Define when the map operation should succeed or fail:
@@ -391,8 +354,6 @@ def handler(event: dict, context: DurableContext) -> list[str]:
391354

392355
**Control concurrency for external calls** - When calling external APIs, use `max_concurrency` to avoid rate limits.
393356

394-
**Batch for efficiency** - For small, fast operations, use `item_batcher` to reduce overhead.
395-
396357
**Define completion criteria** - Use `CompletionConfig` to specify when the operation should succeed or fail.
397358

398359
**Keep map functions focused** - Each map function should process one item. Don't mix collection iteration with item processing.
@@ -401,7 +362,7 @@ def handler(event: dict, context: DurableContext) -> list[str]:
401362

402363
**Handle errors gracefully** - Wrap error-prone code in try-except blocks or use completion criteria to tolerate failures.
403364

404-
**Consider collection size** - For very large collections (10,000+ items), consider batching or processing in chunks.
365+
**Consider collection size** - For very large collections (10,000+ items), consider processing in chunks.
405366

406367
**Monitor memory usage** - Large collections create many checkpoints. Monitor Lambda memory usage.
407368

@@ -415,23 +376,15 @@ def handler(event: dict, context: DurableContext) -> list[str]:
415376

416377
**Use max_concurrency wisely** - Too much concurrency can overwhelm external services or exhaust Lambda resources. Start conservative and increase as needed.
417378

418-
**Batch small operations** - If each item processes quickly (< 100ms), batching reduces overhead:
419-
420-
```python
421-
config = MapConfig(
422-
item_batcher=ItemBatcher(max_items_per_batch=10)
423-
)
424-
```
425-
426379
**Optimize map functions** - Keep map functions lightweight. Move heavy computation into steps within the map function.
427380

428381
**Use appropriate completion criteria** - Fail fast with `tolerated_failure_count` to avoid processing remaining items when many fail.
429382

430383
**Monitor checkpoint size** - Large result objects increase checkpoint size and Lambda memory usage. Return only necessary data.
431384

432-
**Consider memory limits** - Processing thousands of items creates many checkpoints. Monitor Lambda memory and adjust batch size or concurrency.
385+
**Consider memory limits** - Processing thousands of items creates many checkpoints. Monitor Lambda memory and adjust concurrency.
433386

434-
**Profile your workload** - Test with representative data to find optimal concurrency and batch settings.
387+
**Profile your workload** - Test with representative data to find optimal concurrency settings.
435388

436389
[↑ Back to top](#table-of-contents)
437390

@@ -443,7 +396,7 @@ A: Map operations process a collection of similar items using the same function.
443396

444397
**Q: How many items can I process?**
445398

446-
A: There's no hard limit, but consider Lambda's memory and timeout constraints. For very large collections (10,000+ items), use batching or process in chunks.
399+
A: There's no hard limit, but consider Lambda's memory and timeout constraints. For very large collections (10,000+ items), consider processing in chunks.
447400

448401
**Q: Do items process in order?**
449402

@@ -471,10 +424,6 @@ for item_result in batch_result.results:
471424

472425
A: Yes, you can call `context.map()` inside a map function to process nested collections.
473426

474-
**Q: How does batching work?**
475-
476-
A: When you configure `item_batcher`, multiple items are grouped together and passed as a `BatchedInput` to your map function. Process all items in `batch.items`.
477-
478427
**Q: What's the difference between serdes and item_serdes?**
479428

480429
A: `item_serdes` serializes individual item results as they complete. `serdes` serializes the entire `BatchResult` at the end. Use both for custom serialization at different levels.

0 commit comments

Comments
 (0)