diff --git a/.chloggen/schema_of_gen_ai_tool_definitions.yaml b/.chloggen/schema_of_gen_ai_tool_definitions.yaml new file mode 100644 index 0000000000..86a27424df --- /dev/null +++ b/.chloggen/schema_of_gen_ai_tool_definitions.yaml @@ -0,0 +1,26 @@ +# Use this changelog template to create an entry for release notes. +# +# If your change doesn't affect end users you should instead start +# your pull request title with [chore] or use the "Skip Changelog" label. + +# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' +change_type: enhancement + +# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db) +component: gen-ai + +# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). +note: Enhance the definition of `gen_ai.tool.definitions` attribute. + +# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. +# The values here must be integers. +issues: [2721, 1835] + +# (Optional) One or more lines of additional information to render under the primary note. +# These lines will be padded with 2 spaces and then inserted directly into the document. +# Use pipe (|) for multiline entries. +subtext: | + The schema of `gen_ai.tool.definitions` attribute is now enhanced to: + - Add JSON schema of `gen_ai.tool.definitions` attribute. + - Document the behavior of capturing tool definitions. + - Capture the tool definitions in a simplified format if the content capturing is disabled. diff --git a/docs/gen-ai/aws-bedrock.md b/docs/gen-ai/aws-bedrock.md index 6bbe413fa0..601f1d060e 100644 --- a/docs/gen-ai/aws-bedrock.md +++ b/docs/gen-ai/aws-bedrock.md @@ -194,13 +194,19 @@ section for more details. **[14] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. + +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/gen-ai/azure-ai-inference.md b/docs/gen-ai/azure-ai-inference.md index ea3d446b62..e2b5ab1047 100644 --- a/docs/gen-ai/azure-ai-inference.md +++ b/docs/gen-ai/azure-ai-inference.md @@ -195,13 +195,19 @@ section for more details. **[14] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. + +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/gen-ai/gen-ai-agent-spans.md b/docs/gen-ai/gen-ai-agent-spans.md index 2eaeaa51a3..ed0a8e1e7e 100644 --- a/docs/gen-ai/gen-ai-agent-spans.md +++ b/docs/gen-ai/gen-ai-agent-spans.md @@ -334,13 +334,19 @@ section for more details. **[15] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. + +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/gen-ai/gen-ai-events.md b/docs/gen-ai/gen-ai-events.md index 0ccae8ab5b..b51cf0ce00 100644 --- a/docs/gen-ai/gen-ai-events.md +++ b/docs/gen-ai/gen-ai-events.md @@ -175,13 +175,19 @@ section for more details. **[13] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. + +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/gen-ai/gen-ai-spans.md b/docs/gen-ai/gen-ai-spans.md index 2c33f05809..02c7eb00cd 100644 --- a/docs/gen-ai/gen-ai-spans.md +++ b/docs/gen-ai/gen-ai-spans.md @@ -207,13 +207,19 @@ section for more details. **[14] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. + +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/gen-ai/gen-ai-tool-definitions.json b/docs/gen-ai/gen-ai-tool-definitions.json new file mode 100644 index 0000000000..d5c449228d --- /dev/null +++ b/docs/gen-ai/gen-ai-tool-definitions.json @@ -0,0 +1,84 @@ +{ + "$defs": { + "FunctionToolDefinition": { + "additionalProperties": true, + "description": "Represents a tool definition in the form of a function.", + "properties": { + "type": { + "const": "function", + "description": "The type of the tool.", + "title": "Type", + "type": "string" + }, + "name": { + "description": "The name of the tool.", + "title": "Name", + "type": "string" + }, + "description": { + "default": null, + "description": "The description of the tool.", + "title": "Description", + "type": "string" + }, + "parameters": { + "default": null, + "description": "Schema that defines the parameters accepted by the tool. The RECOMMENDED format is JSON Schema.", + "title": "Parameters" + }, + "response": { + "default": null, + "description": "Schema that defines the response returned by the tool. The RECOMMENDED format is JSON Schema.", + "title": "Response" + } + }, + "required": [ + "type", + "name" + ], + "title": "FunctionToolDefinition", + "type": "object" + }, + "GenericToolDefinition": { + "additionalProperties": true, + "description": "Represents a tool definition in any forms.", + "properties": { + "type": { + "description": "The type of the tool.", + "title": "Type", + "type": "string" + }, + "name": { + "description": "The name of the tool.", + "title": "Name", + "type": "string" + }, + "description": { + "default": null, + "description": "The description of the tool.", + "title": "Description", + "type": "string" + } + }, + "required": [ + "type", + "name" + ], + "title": "GenericToolDefinition", + "type": "object" + } + }, + "description": "Represents the list of tool definitions available to the GenAI agent or model.", + "items": { + "anyOf": [ + { + "$ref": "#/$defs/FunctionToolDefinition" + }, + { + "$ref": "#/$defs/GenericToolDefinition" + } + ] + }, + "title": "ToolDefinitions", + "type": "array" +} \ No newline at end of file diff --git a/docs/gen-ai/non-normative/examples-llm-calls.md b/docs/gen-ai/non-normative/examples-llm-calls.md index 9965ffb6f2..c927b35afd 100644 --- a/docs/gen-ai/non-normative/examples-llm-calls.md +++ b/docs/gen-ai/non-normative/examples-llm-calls.md @@ -248,6 +248,18 @@ They are likely to be siblings if there is an encompassing span. | `gen_ai.usage.output_tokens` | `17` | | `gen_ai.usage.input_tokens` | `47` | | `gen_ai.response.finish_reasons`| `["tool_calls"]` | +| `gen_ai.tool.definitions` | [`gen_ai.tool.definitions`](#gen-ai-tool-definitions-tool-call-span-0) | + +`gen_ai.tool.definitions` value + +```json +[ + { + "type": "function", + "name": "get_weather" + } +] +``` **Tool call:** @@ -298,6 +310,7 @@ They are likely to be siblings if there is an encompassing span. | `gen_ai.response.finish_reasons`| `["tool_calls"]` | | `gen_ai.input.messages` | [`gen_ai.input.messages`](#gen-ai-input-messages-tool-call-span-1) | | `gen_ai.output.messages` | [`gen_ai.output.messages`](#gen-ai-output-messages-tool-call-span-1) | +| `gen_ai.tool.definitions` | [`gen_ai.tool.definitions`](#gen-ai-tool-definitions-tool-call-span-1) | `gen_ai.input.messages` value @@ -336,6 +349,30 @@ They are likely to be siblings if there is an encompassing span. ] ``` +`gen_ai.tool.definitions` value + +```json +[ + { + "type": "function", + "name": "get_weather", + "description": "Get the weather in a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The city and state, e.g. San Francisco, CA" + }, + "required": [ + "location" + ] + } + } + } +] +``` + **Tool call:** If tool call is [instrumented according to execute-tool span definition](../gen-ai-spans.md#execute-tool-span), it may look like this: diff --git a/docs/gen-ai/non-normative/models.ipynb b/docs/gen-ai/non-normative/models.ipynb index 891bed9423..6f003094ce 100644 --- a/docs/gen-ai/non-normative/models.ipynb +++ b/docs/gen-ai/non-normative/models.ipynb @@ -41,7 +41,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "id": "5124fe15", "metadata": {}, "outputs": [], @@ -68,7 +68,7 @@ " type: Literal[\"tool_call\"] = Field(description=\"The type of the content captured in this part.\")\n", " id: Optional[str] = Field(default=None, description=\"Unique identifier for the tool call.\")\n", " name: str = Field(description=\"Name of the tool.\")\n", - " arguments: Any = Field(None, description=\"Arguments for the tool call.\")\n", + " arguments: Any = Field(default=None, description=\"Arguments for the tool call.\")\n", "\n", " class Config:\n", " extra = \"allow\"\n", @@ -222,6 +222,77 @@ "# Print the JSON schema for the SystemInstructions model\n", "print(json.dumps(SystemInstructions.model_json_schema(), indent=4))" ] + }, + { + "cell_type": "markdown", + "id": "f019c33a", + "metadata": {}, + "source": [ + "## `gen_ai.tool.definitions` model\n", + "\n", + "Corresponding attribute: [`gen_ai.tool.definitions`](/docs/registry/attributes/gen-ai.md#gen-ai-tool-definitions).\n", + "JSON schema: [`gen_ai-tool-definitions.json`](../gen-ai-tool-definitions.json)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9e84726", + "metadata": {}, + "outputs": [], + "source": [ + "class FunctionToolDefinition(BaseModel):\n", + " \"\"\"\n", + " Represents a tool definition in the form of a function.\n", + " \"\"\"\n", + " type: Literal[\"function\"] = Field(description=\"The type of the tool.\")\n", + " name: str = Field(description=\"The name of the tool.\")\n", + " description: str = Field(None, description=\"The description of the tool.\")\n", + " parameters: Any = Field(\n", + " default=None,\n", + " description=(\n", + " \"Schema that defines the parameters accepted by the tool. \"\n", + " \"The RECOMMENDED format is JSON Schema.\"\n", + " )\n", + " )\n", + " response: Any = Field(\n", + " default=None,\n", + " description=(\n", + " \"Schema that defines the response returned by the tool. \"\n", + " \"The RECOMMENDED format is JSON Schema.\"\n", + " )\n", + " )\n", + "\n", + " class Config:\n", + " extra = \"allow\"\n", + "\n", + "class GenericToolDefinition(BaseModel):\n", + " \"\"\"\n", + " Represents a tool definition in any forms.\n", + " \"\"\"\n", + " type: str = Field(description=\"The type of the tool.\")\n", + " name: str = Field(description=\"The name of the tool.\")\n", + " description: str = Field(None, description=\"The description of the tool.\")\n", + "\n", + " class Config:\n", + " extra = \"allow\"\n", + "\n", + "ToolDefinition = Union[\n", + " FunctionToolDefinition,\n", + " GenericToolDefinition, # Catch-all for any other type\n", + " # Add other tool definition types here as needed,\n", + " # e.g. file search, code interpreter, etc\n", + "]\n", + "\n", + "class ToolDefinitions(RootModel[List[ToolDefinition]]):\n", + " \"\"\"\n", + " Represents the list of tool definitions available to the GenAI agent or model.\n", + " \"\"\"\n", + " pass\n", + "\n", + "# Print the JSON schema for the ToolDefinitions model\n", + "print(json.dumps(ToolDefinitions.model_json_schema(), indent=4))" + ] } ], "metadata": { @@ -240,7 +311,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.13.5" + "version": "3.13.7" } }, "nbformat": 4, diff --git a/docs/gen-ai/openai.md b/docs/gen-ai/openai.md index 4af34bd316..79d3f2fd6d 100644 --- a/docs/gen-ai/openai.md +++ b/docs/gen-ai/openai.md @@ -200,13 +200,19 @@ section for more details. **[15] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. + +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. --- diff --git a/docs/registry/attributes/gen-ai.md b/docs/registry/attributes/gen-ai.md index abc61bcf9b..800b6b8344 100644 --- a/docs/registry/attributes/gen-ai.md +++ b/docs/registry/attributes/gen-ai.md @@ -157,13 +157,19 @@ deserialize it to an object. When recorded on spans, it MAY be recorded as a JSO **[12] `gen_ai.tool.definitions`:** The value of this attribute matches source system tool definition format. -It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available -to the instrumentation, the instrumentation SHOULD do the best effort to -deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. +It's expected to be an array of objects, each representing a tool definition, +and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). +In case a serialized string is available to the instrumentation, the instrumentation +SHOULD do the best effort to deserialize it to an array. + +When the attribute is recorded on events, it MUST be recorded in structured +form. When recorded on spans, it MAY be recorded as a JSON string if structured +format is not supported and SHOULD be recorded in structured form otherwise. -Since this attribute could be large, it's NOT RECOMMENDED to populate -it by default. Instrumentations MAY provide a way to enable -populating this attribute. +If instrumentations can reliably deserialize and extract the tool definitions, +it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition +objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. +Instrumentations MAY provide a way to enable populating this attribute. **[13] `gen_ai.tool.type`:** Extension: A tool executed on the agent-side to directly call external APIs, bridging the gap between the agent and real-world systems. Agent-side operations involve actions that are performed by the agent on the server or within the agent's controlled environment. diff --git a/model/gen-ai/registry.yaml b/model/gen-ai/registry.yaml index 8b3fa51dd4..1d03880d3e 100644 --- a/model/gen-ai/registry.yaml +++ b/model/gen-ai/registry.yaml @@ -303,13 +303,19 @@ groups: note: | The value of this attribute matches source system tool definition format. - It's expected to be an array of objects where each object represents a tool definition. In case a serialized string is available - to the instrumentation, the instrumentation SHOULD do the best effort to - deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise. + It's expected to be an array of objects, each representing a tool definition, + and the structure of the array is expected to match the [Tool Definitions JSON Schema](/docs/gen-ai/gen-ai-tool-definitions.json). + In case a serialized string is available to the instrumentation, the instrumentation + SHOULD do the best effort to deserialize it to an array. + + When the attribute is recorded on events, it MUST be recorded in structured + form. When recorded on spans, it MAY be recorded as a JSON string if structured + format is not supported and SHOULD be recorded in structured form otherwise. - Since this attribute could be large, it's NOT RECOMMENDED to populate - it by default. Instrumentations MAY provide a way to enable - populating this attribute. + If instrumentations can reliably deserialize and extract the tool definitions, + it's RECOMMENDED to only populate required fields (`name`, `type`) of the definition + objects by default. Otherwise, it's NOT RECOMMENDED to populate it by default. + Instrumentations MAY provide a way to enable populating this attribute. examples: - | [