Fix: populate token/model fields in native-path routing log entries#466
Merged
Conversation
… log entries The native LLM path (NativeToolLoopAsync) was writing TierRoutingEntry records with ModelId set but InputTokens/OutputTokens left null. TierRoutingAnalyzer.BuildProjectedCost skips any entry with null tokens, so all 15 entries were unpriced -> \.00 total cost reported. Fix: - Add InputTokens, OutputTokens, ModelId to LoopDiagnostics so the loop runner can surface token data back to callers without changing RunAsync's Task<string> return type. - RunNativeLoopAsync now populates those fields from the FICC response. - RunTextBasedLoopAsync tracks lastModelId across iterations and populates the fields in its finally block alongside the existing OTel recording. - NativeToolLoopAsync creates a LoopDiagnostics, passes it to RunAsync, and uses diag.InputTokens / OutputTokens / ToolCalls / ModelId when writing the TierRoutingEntry (falls back to registry.GetModelId when the response carries no model ID). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Patch release for routing log token attribution fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
0.12.28 was already built/deployed without the routing-log token attribution fix; bump to 0.12.29 so the fixed agent image has a unique, traceable tag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The agent's daily cost guard reported .00 cost — nothing was being priced, despite a populated pricing table.
Root cause: The native LLM path (
NativeToolLoopAsyncinUserMessageHandler) wroteTierRoutingEntryrecords withModelIdset butInputTokens/OutputTokensleft null.TierRoutingAnalyzer.BuildProjectedCostskips any entry with null tokens, so every entry was unpriced → total cost collapsed to .00.Fix
InputTokens,OutputTokens,ModelIdtoLoopDiagnosticsso the loop runner can surface token data to callers without changingRunAsync'sTask<string>return type.RunNativeLoopAsyncpopulates those fields from the FICC response.RunTextBasedLoopAsynctrackslastModelIdacross iterations and populates the fields in itsfinallyblock alongside existing OTel recording.NativeToolLoopAsyncpasses aLoopDiagnosticstoRunAsyncand usesdiag.InputTokens/OutputTokens/ToolCalls/ModelIdwhen writing the routing entry (falls back toregistry.GetModelIdwhen the response carries no model ID).Validation
RockBot.Host.Testspass.rockbot-agent:0.12.29to the cluster; new routing-log entries now carry token + model fields and cost attribution is non-zero.Version bumped 0.12.27 → 0.12.29.