Truncation DevEx

[ivo-1]

Description

I think the experience when dealing with truncation could be improved.

• I would suggest the truncation replacement to be more explicit, e.g. [...truncated by logfire...] instead of just ... to prevent confusion. I've implemented that here: fix: be more explicit with truncation #1470
• The warning for truncation in the details tab is much appreciated, however it is missing from the Agent run tab. There should be a warning for truncated fields there as well.
• If it's being truncated the result seems to be much less than before the threshold. It would be better to truncate approximately up to the threshold. It seems there are competing max_length parameters at play. In my case I have 100k character long messages that are not being truncated but if they are being truncated they suddenly are only 10k characters long. This also led me down a wrong path before thinking about truncation.

Python, Logfire & OS Versions, related packages (not required)

logfire: 4.11.0

[alexmojaki]

Most truncation actually happens in the backend, not the SDK. In the SDK the truncation is usually about message formatting, where replacing ... with [...truncated by logfire...] is quite significant because the threshold is small and it's for human readability.

Yes, we should make that improvement in the frontend.

In the backend, I think what happens is that it only considers truncating a span if the whole span is above one limit. If it is then it starts truncating attributes, and the limit for attributes is significantly lower than the span limit in case many attributes are big. It's not the best algorithm and I do want to improve it, but more than that I want to store big attributes separately.