Leveraging DynamoDB sort key for historic messages

[juliangrueber]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Have a single record for each interaction with the following key schema: Have the session_id as the primary key (hash key) and use a timestamp or another identifier as the sort key (range key).

In particular, for a single table design:

item = {
"PK" : "SESSIOND_ID#123456789",
"SK": "TIMESTAMP#202407011200000"
"message" : [...]
}

To get the entire history one would querying the table based on the given PK, for instance, with the condition:

from boto3.dynamodb.conditions import Key

response = table.query(
      KeyConditionExpression = Key('PK').eq("SESSIOND_ID#123456789") & Key('SK'). begins_with("TIMESTAMP") 
)

In the case of having a multi table design, the principle is the same. One would just use the SESSION_ID and TIMESTAMP instead of PK and SK, respectively.

Motivation

As of now, each session has exactly a single record in the DynamoDB table. That limits the size of the message history by 400 KB.
After each interaction the session is fully overwritten using the put_item operation which does not allow a full lineage.
If the history becomes large, retrieving might become inefficient

Proposal (If applicable)

No response