Skip to content

Commit 3ed80f5

Browse files
authored
telemetry(amazonq): adding a new metric amazonq_invokeLLM for agentic chat (#1027)
## Problem - IDE has `amazonq_toolUseSuggested` metric emitted per tool execution but this count will be high in the future. - No latency metrics for tool execution. ## Solution - `amazonq_invokeLLM` is emitted per LLM call in the agentic loop, this emits the list of toolName's, list of toolId's and list of tool execution latencies. ``` { "name": "amazonq_invokeLLM", "description": "This metric is emitted per LLM call in the agentic loop with or without tool execution.", "metadata": [ { "type": "credentialStartUrl", "required": false }, { "type": "cwsprChatConversationId" }, { "type": "cwsprChatConversationType" }, { "type": "cwsprToolName" }, { "type": "cwsprToolUseId" }, { "type": "enabled", "required": false }, { "type": "languageServerVersion", "required": false }, { "type": "latency", "required": false }, ] }, ``` <!--- REMINDER: - Read CONTRIBUTING.md first. - Add test coverage for your changes. - Link to related issues/commits. - Testing: how did you test your changes? - Screenshots if applicable --> ## License By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
1 parent af7a228 commit 3ed80f5

File tree

1 file changed

+41
-2
lines changed

1 file changed

+41
-2
lines changed

telemetry/definitions/commonDefinitions.json

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1307,12 +1307,12 @@
13071307
{
13081308
"name": "cwsprToolName",
13091309
"type": "string",
1310-
"description": "Client side tool name (ex: fsWrite, executeBash)"
1310+
"description": "Client side tool/s name (ex: fsWrite, executeBash)"
13111311
},
13121312
{
13131313
"name": "cwsprToolUseId",
13141314
"type": "string",
1315-
"description": "The id for when a client side tool is used."
1315+
"description": "The id/s when a client side tool/s is used."
13161316
},
13171317
{
13181318
"name": "databaseCredentials",
@@ -1807,6 +1807,11 @@
18071807
"type": "string",
18081808
"description": "The version of the language server"
18091809
},
1810+
{
1811+
"name": "latency",
1812+
"type": "string",
1813+
"description": "latency/s from any operation or an execution like tool execution."
1814+
},
18101815
{
18111816
"name": "loadFileTime",
18121817
"type": "int",
@@ -2806,6 +2811,40 @@
28062811
}
28072812
]
28082813
},
2814+
{
2815+
"name": "amazonq_invokeLLM",
2816+
"description": "This metric is emitted per LLM call in the agentic loop with or without tool execution.",
2817+
"metadata": [
2818+
{
2819+
"type": "credentialStartUrl",
2820+
"required": false
2821+
},
2822+
{
2823+
"type": "cwsprChatConversationId"
2824+
},
2825+
{
2826+
"type": "cwsprChatConversationType"
2827+
},
2828+
{
2829+
"type": "cwsprToolName"
2830+
},
2831+
{
2832+
"type": "cwsprToolUseId"
2833+
},
2834+
{
2835+
"type": "enabled",
2836+
"required": false
2837+
},
2838+
{
2839+
"type": "languageServerVersion",
2840+
"required": false
2841+
},
2842+
{
2843+
"type": "latency",
2844+
"required": false
2845+
}
2846+
]
2847+
},
28092848
{
28102849
"name": "amazonq_isAcceptedCodeChanges",
28112850
"description": "User applied code changes generated for the task.",

0 commit comments

Comments
 (0)