Add option to breakdown by metadata in widgets

[adumit]

Describe the feature or potential improvement

In the metadata widget:

I'd love the ability to specify a key and then the values of the metadata item are used to split the chart. This would allow for very interesting looks at outcomes of traces in addition to specific scores. E.g. what are the classes the model is most often predicting? Which is stored in output and metadata on the trace.

Additional information

No response

[marcklingen]

thanks for sharing this!

[flipace]

We also would have the use case for being able to trace basically different tenants via metadata.tenant breakdown. I don't think this is possible yet?

[LoreDema]

That would be relevant also for us!

[karlru]

This would be a nice addition. Having a similar case to @flipace, our current workaround is to use the trace's user_id attribute.

[flipace]

interesting - do you put the tenant info into as an additional suffix/prefix or so? i didn't think about doing it this way actually! nice workaround

[karlru]

We previously had no use for the user ID, so now its value is the tenant ID. If there's a user ID already set then adding tenant as suffix/prefix and using it in breakdown would essentially group by user + tenant, which might not be what you want. In that case, tags could be another possible workaround

[flipace]

yeah i was wondering whether it could be possible to filter maybe with a "ends width" (i didn't check now) or pattern or...

anways, now that you shared this, i think we could rethinkg whether user_id is something we want to keep on user level or wouldn't actually also be fine to just ditch and have as tenant field. thank you! 🙏

nonetheless, i think this feature would still be extremely valuable since it opens up very dynamic options to visualize data (making tags a lot more powerful)

[maxdeichmann]

Hi both of you. Thanks for sharing your thoughts here. We have this very high on our road map and want to make this work at scale. We will update this thread once there are news.