Skip to content

fix: render labeling queue payload with messages mode for LLM traces#1508

Open
IgnazioDS wants to merge 2 commits intolmnr-ai:mainfrom
IgnazioDS:fix/labeling-queue-message-rendering
Open

fix: render labeling queue payload with messages mode for LLM traces#1508
IgnazioDS wants to merge 2 commits intolmnr-ai:mainfrom
IgnazioDS:fix/labeling-queue-message-rendering

Conversation

@IgnazioDS
Copy link
Copy Markdown

@IgnazioDS IgnazioDS commented Mar 26, 2026

Summary

  • Auto-detects when a labeling queue item payload contains LLM message arrays and defaults to the messages rendering mode (same as the trace view)
  • Adds mode switcher (MESSAGES / JSON / YAML / TEXT) so users can toggle between formatted and raw views
  • No backend changes — this is a frontend-only display fix

Problem

When an LLM trace is added to the labeling queue, the output renders as raw JSON:

[{"role":"assistant","content":"{\"score\":2}"}]

But the same trace in the evaluations view renders cleanly via the Messages component, showing just the content under the role label.

Solution

Added a looksLikeMessageArray() detector that checks if the payload (or its data/target fields) contains arrays of objects with role + content fields. When detected, the ContentRenderer defaults to "messages" mode instead of "json".

Users can still switch to raw JSON/YAML/TEXT via the mode dropdown — this aligns with the "show raw data" toggle approach mentioned in #639.

Test Plan

  • Add an LLM trace to a labeling queue → should render with messages mode by default
  • Toggle to JSON mode → should show raw message array
  • Add a non-LLM trace → should default to JSON mode as before
  • Manually created queue items → should default to JSON mode

Fixes #639


Note

Low Risk
Low risk, frontend-only rendering changes limited to the labeling queue payload viewer; main risk is incorrect auto-detection causing an unexpected default view for some payload shapes.

Overview
Labeling queue payloads now auto-detect LLM chat message arrays (top-level or under data/target) and default ContentRenderer to messages mode instead of always json.

The payload viewer adds an explicit mode switcher (MESSAGES/JSON/YAML/TEXT), passes the extracted payload value to render, and forces a remount on item change via key={currentItem?.id} to avoid stale renderer state.

Written by Cursor Bugbot for commit 4cc0059. This will update automatically on new commits. Configure here.

When traces from LLM spans are added to the labeling queue, the output
shows raw message array format ([{"role":"assistant","content":"..."}])
instead of the clean formatted view used in the evaluations trace viewer.

This change:
- Auto-detects if a queue item payload contains LLM message arrays
  (objects with "role" and "content" fields)
- Defaults to "messages" rendering mode when detected, matching the
  trace view behavior
- Adds MESSAGES to available modes so users can toggle between
  formatted messages and raw JSON/YAML/TEXT views

Fixes lmnr-ai#639
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 26, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@IgnazioDS
Copy link
Copy Markdown
Author

Signed the CLA! Please recheck when ready.

…m change

Bug 1: getPayloadDefaultMode detected messages inside payload.data/target
but ContentRenderer received the entire payload object. Renamed to
getPayloadRendering and now returns both the mode and the extracted value
so Messages component gets the correct data shape.

Bug 2: ContentRenderer only reads defaultMode in its useState initializer,
so navigating between queue items kept the stale mode. Added key={currentItem?.id}
to force remounting when the item changes.

Also memoized the payload rendering computation to avoid redundant calls.

Addresses cursor review feedback on lmnr-ai#1508
@IgnazioDS
Copy link
Copy Markdown
Author

Addressed both issues from cursor review:

Bug 1 (High) — Messages mode received entire payload instead of message array:
Renamed getPayloadDefaultModegetPayloadRendering which now returns both { mode, value }. When messages are detected inside payload.data or payload.target, the extracted array is passed to ContentRenderer instead of the wrapper object.

Bug 2 (Medium) — Dynamic defaultMode ignored after initial render:
Added key={currentItem?.id} to ContentRenderer so it remounts when navigating between queue items, allowing the auto-detected mode to take effect for each item.

Also memoized the rendering computation via useMemo to avoid redundant calls.

@IgnazioDS
Copy link
Copy Markdown
Author

@Rainhunter13 Hi! CLA is signed, and I've addressed both cursor review findings:

  1. Fixed message array extraction (was passing full payload wrapper instead of the message array)
  2. Added key prop to reset mode when navigating between queue items

Ready for review when you have a moment!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Traces appear in evaluations differently from labeling queue

2 participants