DynamoDB 400kB item limit causes session crashes

[rmhsilva]

Problem
Various parts of Teal session data can grow unbounded, but the item size in DynamoDB is limited to 400kB. e.g. Session events/logs eventually hit the limit, crashing the session.

This underlying infrastructure constraint should not be visible to programmer in any way.

Possible solutions

• use a different storage mechanism (e.g redis/elasticache)
• split the session data into smaller pieces that are very unlikely to be > 400kB (this is semantically nice, but could get expensive as the number of items grows huge)
• implement an optimising batch storage system (e.g. per-session events/logs, written as multiple binary blob items). Cheaper, but much more complex to implement.