chore(ai): Add metric for tracking ai cost calculation#5560
chore(ai): Add metric for tracking ai cost calculation#5560
Conversation
bb51b0b to
807f933
Compare
Dav1dde
left a comment
There was a problem hiding this comment.
Nice, just a tiny nit and what we discussed that it'll only cover a single pipeline not transaction and span streaming.
| let integration = map_origin_to_integration(origin); | ||
| let platform = platform_tag(platform); | ||
|
|
||
| let Some(costs) = ai::calculate_costs(model_cost, tokens) else { |
There was a problem hiding this comment.
As discussed in Slack, the change here would be only for the span streaming pipeline, not the transaction one (the only one in active use atm)
Maybe you can move the metric emission into ai::calculate_costs, then it covers both pipelines at once.
| } | ||
| } | ||
|
|
||
| fn platform_tag(platform: Option<&str>) -> &'static str { |
There was a problem hiding this comment.
This is a bit awkward because we already have such a function in relay-server, but no good way of sharing it :(
Fine to keep here, would consider just starting utils.rs module in the crate to hide it away, but also not a big deal, just would move it at the end of the file, generally we like to keep public interfaces at the top.
There was a problem hiding this comment.
I would move this to statsd.rs module because it is a concern of metrics reporting (same for map_origin_to_integration).
| let metric_label = if (costs.input > 0.0 && costs.output > 0.0) | ||
| || ((costs.input > 0.0 && costs.output == 0.0) | ||
| || (costs.input == 0.0 && costs.output > 0.0)) | ||
| { | ||
| "calculation_positive" | ||
| } else if costs.input < 0.0 || costs.output < 0.0 { | ||
| "calculation_negative" | ||
| } else { | ||
| "calculation_zero" | ||
| }; |
There was a problem hiding this comment.
Something like this might be more concise (untested):
| let metric_label = if (costs.input > 0.0 && costs.output > 0.0) | |
| || ((costs.input > 0.0 && costs.output == 0.0) | |
| || (costs.input == 0.0 && costs.output > 0.0)) | |
| { | |
| "calculation_positive" | |
| } else if costs.input < 0.0 || costs.output < 0.0 { | |
| "calculation_negative" | |
| } else { | |
| "calculation_zero" | |
| }; | |
| let metric_label = match (costs.input.signum(), costs.output.signum()) { | |
| (-1, _) | (_, -1) => "calculation_negative", | |
| (0, 0) => "calculation_zero", | |
| _ => "calculation_positive", | |
| }; |
| } | ||
| } | ||
|
|
||
| fn platform_tag(platform: Option<&str>) -> &'static str { |
There was a problem hiding this comment.
I would move this to statsd.rs module because it is a concern of metrics reporting (same for map_origin_to_integration).
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| return; | ||
| }; | ||
|
|
||
| // Overwrite all values, the attributes should reflect the values we used to calculate the total. |
There was a problem hiding this comment.
I assume removing that comment was a mistake?
genai.cost_calculation.resultto track positive, negative and zero costs using a metricnormalize_ai_costs, emit that metric depending on the outcomeCloses TET-1675