Skip to content

Conversation

@shawn-mcdonald-dev
Copy link
Contributor

@shawn-mcdonald-dev shawn-mcdonald-dev commented Nov 3, 2025

Issue Reference

This PR fixes #2451

Summary

Updated LLM call selection logic to correctly pick the earliest successful call by lexicographic HttpRequestId ordering. This ensures deterministic selection when multiple successful calls exist.

Changes

This PR addresses an issue where the code prioritized completion time instead of lexicographically earliest ulid uuid.

  1. Filter to keep only candidates where is_success = true
  2. Sort these successful candidates by http request id, ulid uuid, lexicographically)
  3. Choose the first candidate from this sorted list

Details

  • Filters candidates to only successful calls
  • Sorts by request_id.to_string() lexicographically (since HttpRequestId lacks Ord)
  • Picks the first entry in the sorted list
  • Verified via cargo test -p baml-runtime -q

Note

Refactors LLM call selection to pick the earliest successful call by lexicographic HttpRequestId and marks only that call as selected, with new tests covering success-vs-failure and ordering.

  • Tracing/FunctionLog (storage.rs)
    • Refactor LLM call assembly: introduce CallCandidate to gather data for each request_id.
    • Selection logic: filter successful candidates, sort by request_id.to_string() lexicographically, select the first; mark only that call as selected across Basic and Stream kinds.
  • Tests
    • Add test_selected_call_prefers_success_over_failure and test_selected_call_chooses_earlier_success_if_last_failed to verify selection behavior.

Written by Cursor Bugbot for commit 9a3143e. This will update automatically on new commits. Configure here.

@vercel
Copy link

vercel bot commented Nov 3, 2025

@shawn-mcdonald-dev is attempting to deploy a commit to the Boundary Team on Vercel.

A member of the Team first needs to authorize it.

@hellovai
Copy link
Contributor

hellovai commented Nov 4, 2025

@aaronvg can you take a look at this one?

@aaronvg aaronvg had a problem deploying to boundary-tools-dev November 5, 2025 07:25 — with GitHub Actions Failure
@aaronvg aaronvg had a problem deploying to boundary-tools-dev November 5, 2025 07:25 — with GitHub Actions Failure
@aaronvg aaronvg had a problem deploying to boundary-tools-dev November 5, 2025 07:25 — with GitHub Actions Failure
@aaronvg aaronvg added this pull request to the merge queue Nov 5, 2025
Merged via the queue into BoundaryML:canary with commit 516ef6f Nov 5, 2025
13 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants