Skip to content

Commit 51a7af7

Browse files
committed
docs: use env var for DDB table, no hardcode
1 parent 6632396 commit 51a7af7

23 files changed

+124
-64
lines changed

docs/utilities/idempotency.md

Lines changed: 37 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@ You can use `data_keyword_argument` to tell us the argument to extract an idempo
168168

169169
=== "Using Dataclasses"
170170

171-
```python title="working_with_idempotent_function_dataclass.py" hl_lines="3-7 11 26 39"
171+
```python title="working_with_idempotent_function_dataclass.py" hl_lines="4-8 12 28 41"
172172
--8<-- "examples/idempotency/src/working_with_idempotent_function_dataclass.py"
173173
```
174174

@@ -178,7 +178,7 @@ You can use `data_keyword_argument` to tell us the argument to extract an idempo
178178

179179
=== "Using Pydantic"
180180

181-
```python title="working_with_idempotent_function_pydantic.py" hl_lines="1-5 10 23 34"
181+
```python title="working_with_idempotent_function_pydantic.py" hl_lines="3-7 12 26 37"
182182
--8<-- "examples/idempotency/src/working_with_idempotent_function_pydantic.py"
183183
```
184184

@@ -196,7 +196,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
196196

197197
=== "Inferring via the return type"
198198

199-
```python hl_lines="6 24 25 32 36 45"
199+
```python hl_lines="8 27 35 38 48"
200200
--8<-- "examples/idempotency/src/working_with_pydantic_deduced_output_serializer.py"
201201
```
202202

@@ -206,7 +206,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
206206

207207
Alternatively, you can provide an explicit model as an input to `PydanticSerializer`.
208208

209-
```python hl_lines="6 24 25 32 35 44"
209+
```python hl_lines="8 27 35 35 47"
210210
--8<-- "examples/idempotency/src/working_with_pydantic_explicitly_output_serializer.py"
211211
```
212212

@@ -216,7 +216,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
216216

217217
=== "Inferring via the return type"
218218

219-
```python hl_lines="8 27-29 36 40 49"
219+
```python hl_lines="9 30 38 41 51"
220220
--8<-- "examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py"
221221
```
222222

@@ -226,7 +226,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
226226

227227
Alternatively, you can provide an explicit model as an input to `DataclassSerializer`.
228228

229-
```python hl_lines="8 27-29 36 39 48"
229+
```python hl_lines="8 30 38 40 50"
230230
--8<-- "examples/idempotency/src/working_with_dataclass_explicitly_output_serializer.py"
231231
```
232232

@@ -237,7 +237,7 @@ The output serializer supports any JSON serializable data, **Python Dataclasses*
237237
* **to_dict**. Function to convert any type to a JSON serializable dictionary before it saves into the persistent storage.
238238
* **from_dict**. Function to convert from a dictionary retrieved from persistent storage and serialize in its original form.
239239

240-
```python hl_lines="8 32 36 40 50 53"
240+
```python hl_lines="9 34 38 42 52 54 64"
241241
--8<-- "examples/idempotency/src/working_with_idempotent_function_custom_output_serializer.py"
242242
```
243243

@@ -255,7 +255,7 @@ By default, caching is disabled since we don't know how big your response could
255255

256256
=== "Enabling cache"
257257

258-
```python hl_lines="12"
258+
```python hl_lines="15"
259259
--8<-- "examples/idempotency/src/working_with_local_cache.py"
260260
```
261261

@@ -290,7 +290,7 @@ Imagine the function runs successfully, but the client never receives the respon
290290

291291
=== "Payment function"
292292

293-
```python hl_lines="5-9 16 30"
293+
```python hl_lines="6-10 18 31"
294294
--8<-- "examples/idempotency/src/working_with_payload_subset.py"
295295
```
296296

@@ -308,8 +308,14 @@ Imagine the function runs successfully, but the client never receives the respon
308308
To prevent against extended failed retries when a [Lambda function times out](https://aws.amazon.com/premiumsupport/knowledge-center/lambda-verify-invocation-timeouts/){target="_blank"},
309309
Powertools for AWS Lambda (Python) calculates and includes the remaining invocation available time as part of the idempotency record.
310310

311-
???+ example
312-
If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will execute the invocation again as if it was in the `EXPIRED` state (e.g, `expire_seconds` field elapsed).
311+
To prevent extended failures, use **`register_lambda_context`** function from your idempotency config to calculate and include the remaining invocation time in your idempotency record.
312+
313+
```python title="working_with_lambda_timeout.py" hl_lines="14 23"
314+
--8<-- "examples/idempotency/src/working_with_lambda_timeout.py"
315+
```
316+
317+
???+ example "Mechanics"
318+
If a second invocation happens **after** this timestamp, and the record is marked as `INPROGRESS`, we will run the invocation again as if it was in the `EXPIRED` state.
313319

314320
This means that if an invocation expired during execution, it will be quickly executed again on the next retry.
315321

@@ -344,7 +350,9 @@ If an exception is handled or raised **outside** your decorated function, then i
344350

345351
This persistence layer is built-in, allowing you to use an existing DynamoDB table or create a new one dedicated to idempotency state (recommended).
346352

347-
=== "Customizing DynamoDBPersistenceLayer to suit your table structure"
353+
```python title="customize_persistence_layer.py" hl_lines="10-18"
354+
--8<-- "examples/idempotency/src/customize_persistence_layer.py"
355+
```
348356

349357
```python hl_lines="7-15"
350358
--8<-- "examples/idempotency/src/customize_persistence_layer.py"
@@ -375,12 +383,12 @@ For a quick start, initialize `RedisCachePersistenceLayer` and pass your cluster
375383
For security, we enforce SSL connections by default; to disable it, set `ssl=False`.
376384

377385
=== "Redis quick start"
378-
```python hl_lines="7-9 12 26"
386+
```python title="getting_started_with_idempotency_redis_config.py" hl_lines="8-10 14 27"
379387
--8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_config.py"
380388
```
381389

382390
=== "Using an existing Redis client"
383-
```python hl_lines="4 9-11 14 22 36"
391+
```python title="getting_started_with_idempotency_redis_client.py" hl_lines="5 10-11 16 24 38"
384392
--8<-- "examples/idempotency/src/getting_started_with_idempotency_redis_client.py"
385393
```
386394

@@ -438,11 +446,9 @@ You can customize attribute names when instantiating `RedisCachePersistenceLayer
438446
| **data_attr** | | `data` | Stores results of successfully executed Lambda handlers |
439447
| **validation_key_attr** | | `validation` | Hashed representation of the parts of the event used for validation |
440448

441-
=== "Customizing RedisPersistenceLayer to suit your data structure"
442-
443-
```python hl_lines="9-16"
444-
--8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
445-
```
449+
```python title="customize_persistence_layer_redis.py" hl_lines="15-18"
450+
--8<-- "examples/idempotency/src/customize_persistence_layer_redis.py"
451+
```
446452

447453
### Common use cases
448454

@@ -777,7 +783,7 @@ You can change this window with the **`expires_after_seconds`** parameter:
777783

778784
=== "Adjusting idempotency record expiration"
779785

780-
```python hl_lines="11"
786+
```python hl_lines="14"
781787
--8<-- "examples/idempotency/src/working_with_record_expiration.py"
782788
```
783789

@@ -811,7 +817,7 @@ With **`payload_validation_jmespath`**, you can provide an additional JMESPath e
811817

812818
=== "Payload validation"
813819

814-
```python hl_lines="12 20 28"
820+
```python hl_lines="16 25 32"
815821
--8<-- "examples/idempotency/src/working_with_validation_payload.py"
816822
```
817823

@@ -847,7 +853,7 @@ This means that we will raise **`IdempotencyKeyError`** if the evaluation of **`
847853

848854
=== "Idempotency key required"
849855

850-
```python hl_lines="11"
856+
```python hl_lines="14"
851857
--8<-- "examples/idempotency/src/working_with_idempotency_key_required.py"
852858
```
853859

@@ -869,13 +875,13 @@ The **`boto_config`** and **`boto3_session`** parameters enable you to pass in a
869875

870876
=== "Custom session"
871877

872-
```python hl_lines="1 11 13"
878+
```python hl_lines="3 13 16"
873879
--8<-- "examples/idempotency/src/working_with_custom_session.py"
874880
```
875881

876882
=== "Custom config"
877883

878-
```python hl_lines="1 11 13"
884+
```python hl_lines="3 13 16"
879885
--8<-- "examples/idempotency/src/working_with_custom_config.py"
880886
```
881887

@@ -895,7 +901,7 @@ You can optionally set a static value for the partition key using the `static_pk
895901

896902
=== "Reusing a DynamoDB table that uses a composite primary key"
897903

898-
```python hl_lines="7"
904+
```python hl_lines="10"
899905
--8<-- "examples/idempotency/src/working_with_composite_key.py"
900906
```
901907

@@ -924,11 +930,9 @@ You can create your own persistent store from scratch by inheriting the `BasePer
924930
* **`_update_record()`** – Updates an item in the persistence store.
925931
* **`_delete_record()`** – Removes an item from the persistence store.
926932

927-
=== "Bring your own persistent store"
928-
929-
```python hl_lines="8 18 65 74 96 124"
930-
--8<-- "examples/idempotency/src/bring_your_own_persistent_store.py"
931-
```
933+
```python title="bring_your_own_persistent_store.py" hl_lines="8 18 65 74 96 124"
934+
--8<-- "examples/idempotency/src/bring_your_own_persistent_store.py"
935+
```
932936

933937
???+ danger
934938
Pay attention to the documentation for each - you may need to perform additional checks inside these methods to ensure the idempotency guarantees remain intact.
@@ -941,7 +945,7 @@ You can set up a `response_hook` in the `IdempotentConfig` class to manipulate t
941945

942946
=== "Using an Idempotent Response Hook"
943947

944-
```python hl_lines="19 21 27 34"
948+
```python hl_lines="20 22 28 36"
945949
--8<-- "examples/idempotency/src/working_with_response_hook.py"
946950
```
947951

@@ -971,7 +975,7 @@ When using response hooks to manipulate returned data from idempotent operations
971975

972976
See [Batch integration](#batch-integration) above.
973977

974-
### Validation utility
978+
### JSON Schema Validation
975979

976980
The idempotency utility can be used with the `validator` decorator. Ensure that idempotency is the innermost decorator.
977981

@@ -983,7 +987,7 @@ The idempotency utility can be used with the `validator` decorator. Ensure that
983987

984988
=== "Using Idempotency with JSONSchema Validation utility"
985989

986-
```python hl_lines="13"
990+
```python hl_lines="16"
987991
--8<-- "examples/idempotency/src/integrate_idempotency_with_validator.py"
988992
```
989993

examples/idempotency/src/customize_persistence_layer.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,14 @@
1+
import os
2+
13
from aws_lambda_powertools.utilities.idempotency import (
24
DynamoDBPersistenceLayer,
35
idempotent,
46
)
57
from aws_lambda_powertools.utilities.typing import LambdaContext
68

9+
table = os.getenv("IDEMPOTENCY_TABLE")
710
persistence_layer = DynamoDBPersistenceLayer(
8-
table_name="IdempotencyTable",
11+
table_name=table,
912
key_attr="idempotency_key",
1013
expiry_attr="expires_at",
1114
in_progress_expiry_attr="in_progress_expires_at",

examples/idempotency/src/customize_persistence_layer_redis.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import os
2+
13
from aws_lambda_powertools.utilities.idempotency import (
24
idempotent,
35
)
@@ -6,8 +8,9 @@
68
)
79
from aws_lambda_powertools.utilities.typing import LambdaContext
810

11+
redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
912
persistence_layer = RedisCachePersistenceLayer(
10-
host="localhost",
13+
host=redis_endpoint,
1114
port=6379,
1215
in_progress_expiry_attr="in_progress_expiration",
1316
status_attr="status",

examples/idempotency/src/getting_started_with_idempotency_redis_client.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import os
12
from dataclasses import dataclass, field
23
from uuid import uuid4
34

@@ -11,8 +12,9 @@
1112
)
1213
from aws_lambda_powertools.utilities.typing import LambdaContext
1314

15+
redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
1416
client = Redis(
15-
host="localhost",
17+
host=redis_endpoint,
1618
port=6379,
1719
socket_connect_timeout=5,
1820
socket_timeout=5,
@@ -29,8 +31,7 @@ class Payment:
2931
payment_id: str = field(default_factory=lambda: f"{uuid4()}")
3032

3133

32-
class PaymentError(Exception):
33-
...
34+
class PaymentError(Exception): ...
3435

3536

3637
@idempotent(persistence_store=persistence_layer)

examples/idempotency/src/getting_started_with_idempotency_redis_config.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import os
12
from dataclasses import dataclass, field
23
from uuid import uuid4
34

@@ -9,7 +10,8 @@
910
)
1011
from aws_lambda_powertools.utilities.typing import LambdaContext
1112

12-
persistence_layer = RedisCachePersistenceLayer(host="localhost", port=6379)
13+
redis_endpoint = os.getenv("REDIS_CLUSTER_ENDPOINT")
14+
persistence_layer = RedisCachePersistenceLayer(host=redis_endpoint, port=6379)
1315

1416

1517
@dataclass
@@ -19,8 +21,7 @@ class Payment:
1921
payment_id: str = field(default_factory=lambda: f"{uuid4()}")
2022

2123

22-
class PaymentError(Exception):
23-
...
24+
class PaymentError(Exception): ...
2425

2526

2627
@idempotent(persistence_store=persistence_layer)

examples/idempotency/src/integrate_idempotency_with_validator.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import os
2+
13
from aws_lambda_powertools.utilities.idempotency import (
24
DynamoDBPersistenceLayer,
35
IdempotencyConfig,
@@ -6,8 +8,9 @@
68
from aws_lambda_powertools.utilities.typing import LambdaContext
79
from aws_lambda_powertools.utilities.validation import envelopes, validator
810

11+
table = os.getenv("IDEMPOTENCY_TABLE")
912
config = IdempotencyConfig(event_key_jmespath='["message", "username"]')
10-
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
13+
persistence_layer = DynamoDBPersistenceLayer(table_name=table)
1114

1215

1316
@validator(envelope=envelopes.API_GATEWAY_HTTP)

examples/idempotency/src/working_with_composite_key.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
1+
import os
2+
13
from aws_lambda_powertools.utilities.idempotency import (
24
DynamoDBPersistenceLayer,
35
idempotent,
46
)
57
from aws_lambda_powertools.utilities.typing import LambdaContext
68

7-
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", sort_key_attr="sort_key")
9+
table = os.getenv("IDEMPOTENCY_TABLE")
10+
persistence_layer = DynamoDBPersistenceLayer(table_name=table, sort_key_attr="sort_key")
811

912

1013
@idempotent(persistence_store=persistence_layer)

examples/idempotency/src/working_with_custom_config.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import os
2+
13
from botocore.config import Config
24

35
from aws_lambda_powertools.utilities.idempotency import (
@@ -10,7 +12,8 @@
1012
# See: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore-config
1113
boto_config = Config()
1214

13-
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", boto_config=boto_config)
15+
table = os.getenv("IDEMPOTENCY_TABLE")
16+
persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto_config=boto_config)
1417

1518
config = IdempotencyConfig(event_key_jmespath="body")
1619

examples/idempotency/src/working_with_custom_session.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import os
2+
13
import boto3
24

35
from aws_lambda_powertools.utilities.idempotency import (
@@ -10,7 +12,8 @@
1012
# See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#module-boto3.session
1113
boto3_session = boto3.session.Session()
1214

13-
persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable", boto3_session=boto3_session)
15+
table = os.getenv("IDEMPOTENCY_TABLE")
16+
persistence_layer = DynamoDBPersistenceLayer(table_name=table, boto3_session=boto3_session)
1417

1518
config = IdempotencyConfig(event_key_jmespath="body")
1619

examples/idempotency/src/working_with_dataclass_deduced_output_serializer.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import os
12
from dataclasses import dataclass
23

34
from aws_lambda_powertools.utilities.idempotency import (
@@ -8,7 +9,8 @@
89
from aws_lambda_powertools.utilities.idempotency.serialization.dataclass import DataclassSerializer
910
from aws_lambda_powertools.utilities.typing import LambdaContext
1011

11-
dynamodb = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
12+
table = os.getenv("IDEMPOTENCY_TABLE")
13+
dynamodb = DynamoDBPersistenceLayer(table_name=table)
1214
config = IdempotencyConfig(event_key_jmespath="order_id") # see Choosing a payload subset section
1315

1416

0 commit comments

Comments
 (0)