Skip to content

Commit 110c9ad

Browse files
authored
Tools extension guide (#7792)
* First partial draft * Add tool calling with prompt-tsx * Add tool implementation * Fix typo * Update related content * Update after review * Update ToC
1 parent 5d8840d commit 110c9ad

File tree

2 files changed

+254
-1
lines changed

2 files changed

+254
-1
lines changed

api/extension-guides/tools.md

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
---
2+
# DO NOT TOUCH — Managed by doc writer
3+
ContentId: aa6d312f-cbac-4633-8579-64d3cb4d17be
4+
DateApproved: 10/29/2024
5+
6+
# Summarize the whole topic in less than 300 characters for SEO purpose
7+
MetaDescription: A guide to creating a language model tool and how to implement tool calling in a chat extension
8+
---
9+
10+
# LanguageModelTool API
11+
12+
In this extension guide, you'll learn how to create a language model tool and how to implement tool calling in a chat extension.
13+
14+
## What is tool calling in an LLM?
15+
16+
Tool calling enables you to extend the functionality of a large language model (LLM) by connecting it to external tools and systems to perform tasks that go beyond text processing.
17+
18+
A language model tool is a function that can be invoked as part of language model request. For example, you might have a function that retrieves information from a database, finds files, or performs some calculation. You can implement a language model tool in your extension, or use publicly available tools from other extensions.
19+
20+
The LLM never actually executes the tool itself, instead the LLM generates the parameters that can be used to call your tool, which your code can then choose how to handle by calling the indicated function. Your extension is always in full control of the tool calling process.
21+
22+
Read more about [function calling](https://platform.openai.com/docs/guides/function-calling) in the OpenAI documentation.
23+
24+
## Why use tool calling?
25+
26+
There are multiple scenarios where you might want to use tool calling in a chat extension. Some examples include:
27+
28+
- **Let the LLM dynamically ask for more context**. For example, you can use a tool to retrieve information from a database, or find relevant files.
29+
- **Let the LLM take some action dynamically**. The LLM itself can't perform calculations or make calls to other systems. For example, use a tool to run a terminal command and return the output to the LLM.
30+
- **Hook up some context/behavior that is contributed by another VS Code extension**. For example, you might have a tool that uses the Git extension to retrieve information about the current repository.
31+
32+
## Tool-calling flow
33+
34+
The tool-calling flow in a chat extension is as follows:
35+
36+
1. Retrieve the list of relevant tools
37+
1. Send the request to the LLM, providing the list of tool definitions to consider
38+
1. The LLM generates a response, which may include one or more requests to invoke a tool
39+
1. Invoke the tool by using the parameter values provided in the LLM response
40+
1. Send another request to the LLM, including the tool results
41+
1. The LLM generates the final user response, which may incorporate tool responses
42+
43+
If the LLM response includes more requests for tool invocations, repeat steps 4-6 until there are no more tool requests.
44+
45+
### Implement tool calling with the chat extension library
46+
47+
You can use the [`@vscode/chat-extension-utils` library](https://www.npmjs.com/package/@vscode/chat-extension-utils) to simplify the process of calling tools in a chat extension.
48+
49+
Implement tool calling in the `vscode.ChatRequestHandler` function of your [chat participant](/api/extension-guides/chat).
50+
51+
1. Determine the relevant tools for the current chat context. You can access all available tools by using `vscode.lm.tools`.
52+
53+
The following code snippet shows how to filter the tools to only those that have a specific tag.
54+
55+
```ts
56+
const tools = request.command === 'all' ?
57+
vscode.lm.tools :
58+
vscode.lm.tools.filter(tool => tool.tags.includes('chat-tools-sample'));
59+
```
60+
61+
1. Send the request and tool definitions to the LLM by using `sendChatParticipantRequest`.
62+
63+
```ts
64+
const libResult = chatUtils.sendChatParticipantRequest(
65+
request,
66+
chatContext,
67+
{
68+
prompt: 'You are a cat! Answer as a cat.',
69+
responseStreamOptions: {
70+
stream,
71+
references: true,
72+
responseText: true
73+
},
74+
tools
75+
},
76+
token);
77+
```
78+
79+
The `ChatHandlerOptions` object has the following properties:
80+
81+
- `prompt`: (optional) Instructions for the chat participant prompt.
82+
- `model`: (optional) The model to use for the request. If not specified, the model from the chat context is used.
83+
- `tools`: (optional) The list of tools to consider for the request.
84+
- `requestJustification`: (optional) A string that describes why the request is being made.
85+
- `responseStreamOptions`: (optional) Enable `sendChatParticipantRequest` to stream the response back to VS Code. Optionally, you can also enable references and/or response text.
86+
87+
1. Return the result from the LLM. This might contain error details or tool-calling metadata.
88+
89+
```ts
90+
return await libResult.result;
91+
```
92+
93+
The full source code of this [tool-calling sample](https://github.com/microsoft/vscode-extension-samples/blob/main/chat-sample/src/chatUtilsSample.ts) is available in the VS Code Extension Samples repository.
94+
95+
### Implement tool calling yourself
96+
97+
For more advanced scenarios, you can also implement tool calling yourself. Optionally, you can use the `@vscode/prompt-tsx` library for crafting the LLM prompts. By implementing tool calling yourself, you have more control over the tool-calling process. For example, to perform additional validation or to handle tool responses in a specific way before sending them to the LLM.
98+
99+
View the full source code for implementing [tool calling by using prompt-tsx](https://github.com/microsoft/vscode-extension-samples/blob/main/chat-sample/src/toolParticipant.ts) in the VS Code Extension Samples repository.
100+
101+
## Create a language model tool
102+
103+
When calling tools, you can call publicly available language model tools contributed by other extensions, or you can create your own tools. When you create a tool, you can choose whether to register it with the VS Code API, or just use it within your extension as a *private* tool.
104+
105+
When you publish a tool with the VS Code API, that tool is available to all extensions.
106+
107+
### Deciding between registering a tool and using it as a private tool
108+
109+
Register a tool with the VS Code API if:
110+
111+
- The tool makes sense to other extensions, and could be used without special handling for the particular tool
112+
- The extension needs to provide a progress message and confirmation
113+
114+
Use a private tool if:
115+
116+
- The tool can't be made public, for example because it's specific to your company or retrieves non-public data
117+
- The tool requires some special handling and is specific to your extension
118+
119+
### Implement a language model tool
120+
121+
To implement a language model tool:
122+
123+
1. Define the tool in the `contributes` property in the `package.json`
124+
125+
The following example shows how to define a tool that counts the number of active tabs in a tab group.
126+
127+
```json
128+
"contributes": {
129+
"languageModelTools": [
130+
{
131+
"name": "chat-tools-sample_tabCount",
132+
"tags": [
133+
"editors",
134+
"chat-tools-sample"
135+
],
136+
"toolReferenceName": "tabCount",
137+
"displayName": "Tab Count",
138+
"modelDescription": "The number of active tabs in a tab group",
139+
"icon": "$(files)",
140+
"inputSchema": {
141+
"type": "object",
142+
"properties": {
143+
"tabGroup": {
144+
"type": "number",
145+
"description": "The index of the tab group to check. This is optional- if not specified, the active tab group will be checked.",
146+
"default": 0
147+
}
148+
}
149+
}
150+
}
151+
]
152+
}
153+
```
154+
155+
A language model tool has the following properties:
156+
157+
- `name`: The unique name of the tool. This is used to reference the tool in the extension implementation code.
158+
- `tags`: An array of tags that describe the tool. This is used to filter the list of tools that are relevant for a specific request.
159+
- `toolReferenceName`: If enabled, the name for users to reference the tool in a chat prompt via `#`.
160+
- `displayName`: The user-friendly name of the tool, used for displaying in the UI.
161+
- `modelDescription`: Description of the tool, which can be used by the language model to select it.
162+
- `icon`: The icon to display for the tool in the UI.
163+
- `inputSchema`: The JSON schema that describes the input parameters for the tool. This is used by the language model to provide parameter values for the tool invocation.
164+
165+
1. (optional) Register tool with `vscode.lm.registerTool`
166+
167+
If you want to publish the tool for use by other extensions, you must register the tool with the `vscode.lm.registerTool` API. Provide the name of the tool as you specified it in the `package.json` file.
168+
169+
```ts
170+
export function registerChatTools(context: vscode.ExtensionContext) {
171+
context.subscriptions.push(vscode.lm.registerTool('chat-tools-sample_tabCount', new TabCountTool()));
172+
}
173+
```
174+
175+
1. Implement the language model tool by implementing the `vscode.LanguageModelTool<>` interface.
176+
177+
- Implement `prepareInvocation` to provide a confirmation message for the tool invocation.
178+
179+
The following example shows how to provide a confirmation message for the tab count tool.
180+
181+
```ts
182+
async prepareInvocation(
183+
options: vscode.LanguageModelToolInvocationPrepareOptions<ITabCountParameters>,
184+
_token: vscode.CancellationToken
185+
) {
186+
const confirmationMessages = {
187+
title: 'Count the number of open tabs',
188+
message: new vscode.MarkdownString(
189+
`Count the number of open tabs?` +
190+
(options.input.tabGroup !== undefined
191+
? ` in tab group ${options.input.tabGroup}`
192+
: '')
193+
),
194+
};
195+
196+
return {
197+
invocationMessage: 'Counting the number of tabs',
198+
confirmationMessages,
199+
};
200+
}
201+
```
202+
203+
- Define an interface that describes the tool input parameters. This interface is used in the `invoke` method.
204+
205+
The following example shows the interface for the tab count tool.
206+
207+
```ts
208+
export interface ITabCountParameters {
209+
tabGroup?: number;
210+
}
211+
```
212+
213+
- Implement `invoke`, which is called when the tool is invoked. It receives the tool input parameters in the `options` parameter.
214+
215+
The following example shows the implementation of the tab count tool. The result of the tool is an instance of type `vscode.LanguageModelToolResult`.
216+
217+
```ts
218+
async invoke(
219+
options: vscode.LanguageModelToolInvocationOptions<ITabCountParameters>,
220+
_token: vscode.CancellationToken
221+
) {
222+
const params = options.input;
223+
if (typeof params.tabGroup === 'number') {
224+
const group = vscode.window.tabGroups.all[Math.max(params.tabGroup - 1, 0)];
225+
const nth =
226+
params.tabGroup === 1
227+
? '1st'
228+
: params.tabGroup === 2
229+
? '2nd'
230+
: params.tabGroup === 3
231+
? '3rd'
232+
: `${params.tabGroup}th`;
233+
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open in the ${nth} tab group.`)]);
234+
} else {
235+
const group = vscode.window.tabGroups.activeTabGroup;
236+
return new vscode.LanguageModelToolResult([new vscode.LanguageModelTextPart(`There are ${group.tabs.length} tabs open.`)]);
237+
}
238+
}
239+
```
240+
241+
View the full source code for implementing a [language model tool](https://github.com/microsoft/vscode-extension-samples/blob/main/chat-sample/src/tools.ts) in the VS Code Extension Samples repository.
242+
243+
## Getting started
244+
245+
- [Chat extension sample](https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample)
246+
247+
## Related content
248+
249+
- [Get started with the Language Model API](/api/extension-guides/language-model)
250+
- [Build a chat extension](/api/extension-guides/chat)
251+
- [Use Prompt-tsx](/api/extension-guides/prompt-tsx)
252+
- [@vscode/vscode-chat-extension-utils library](https://github.com/microsoft/vscode-chat-extension-utils)

api/toc.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@
3131
["Chat Tutorial", "/api/extension-guides/chat-tutorial"],
3232
["Language Model", "/api/extension-guides/language-model"],
3333
["Language Model Tutorial", "/api/extension-guides/language-model-tutorial"],
34-
["Language Model Prompts", "/api/extension-guides/prompt-tsx"],
34+
["Language Model Tools", "/api/extension-guides/tools"],
35+
["Prompt TSX", "/api/extension-guides/prompt-tsx"],
3536
["Tree View", "/api/extension-guides/tree-view"],
3637
["Webview", "/api/extension-guides/webview"],
3738
["Notebook", "/api/extension-guides/notebook"],

0 commit comments

Comments
 (0)