Skip to content

fix(ai): truncate long message histories #4885

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 14 commits into
base: master
Choose a base branch
from
Draft

fix(ai): truncate long message histories #4885

wants to merge 14 commits into from

Conversation

shellmayr
Copy link
Member

@shellmayr shellmayr commented Oct 3, 2025 •
edited
Loading

  • Add truncation for long message histories to prevent spans from being rejected due to exceeding size limits
  • Messages are removed, oldest first, if the span attribute exceeds 20kB, and a _meta entry is added when removed
  • If there is a single message left exceeding the limit, the message string is truncated to stay under 16kB (80% of 20kB) to allow some space for other data, adding an ellipsis ... when truncated, in the style that relay currently does it.
  • Huggingface Hub currently doesn't seem to send spans that are OTEL compatible - the attribute is current just a string as evidenced by this test

Closes TET-1208

@linear Linear
Copy link

linear bot commented Oct 3, 2025

TET-1208 Python SDK: truncate gen_ai messages

@shellmayr shellmayr force-pushed the shellmayr/fix/truncate-long-gen_ai-messages branch from 92329fb to f66a772 Compare October 7, 2025 14:20
@shellmayr shellmayr force-pushed the shellmayr/fix/truncate-long-gen_ai-messages branch from c6716c6 to 294c66c Compare October 9, 2025 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@shellmayr