Skip to content

Commit 62f930f

Browse files
sicoylemsfussellmarcduiker
authored
feat: conversation api toolcalling support documentation (#4769)
* docs: add tool call support in conversation api docs Signed-off-by: Samantha Coyle <[email protected]> * fix: address feedback Signed-off-by: Samantha Coyle <[email protected]> --------- Signed-off-by: Samantha Coyle <[email protected]> Signed-off-by: Marc Duiker <[email protected]> Co-authored-by: Mark Fussell <[email protected]> Co-authored-by: Marc Duiker <[email protected]>
1 parent ef0e7d5 commit 62f930f

File tree

3 files changed

+174
-33
lines changed

3 files changed

+174
-33
lines changed

‎daprdocs/content/en/developing-applications/building-blocks/conversation/conversation-overview.md‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,12 @@ Dapr's conversation API reduces the complexity of securely and reliably interact
1414

1515
<img src="/images/conversation-overview.png" width=800 alt="Diagram showing the flow of a user's app communicating with Dapr's LLM components.">
1616

17-
In additon to enabling critical performance and security functionality (like [prompt caching]({{% ref "#prompt-caching" %}}) and [PII scrubbing]({{% ref "#personally-identifiable-information-pii-obfuscation" %}})), you can also pair the conversation API with Dapr functionalities, like:
17+
In addition to enabling critical performance and security functionality (like [prompt caching]({{% ref "#prompt-caching" %}}) and [PII scrubbing]({{% ref "#personally-identifiable-information-pii-obfuscation" %}})), the conversation API also provides:
18+
19+
- **Tool calling capabilities** that allow LLMs to interact with external functions and APIs, enabling more sophisticated AI applications
20+
- **OpenAI-compatible interface** for seamless integration with existing AI workflows and tools
21+
22+
You can also pair the conversation API with Dapr functionalities, like:
1823
- Resiliency circuit breakers and retries to circumvent limit and token errors, or
1924
- Middleware to authenticate requests coming to and from the LLM
2025

@@ -45,6 +50,17 @@ The PII scrubber obfuscates the following user information:
4550
- SHA-256 hex
4651
- MD5 hex
4752

53+
### Tool calling support
54+
55+
The conversation API supports advanced tool calling capabilities that allow LLMs to interact with external functions and APIs. This enables you to build sophisticated AI applications that can:
56+
57+
- Execute custom functions based on user requests
58+
- Integrate with external services and databases
59+
- Provide dynamic, context-aware responses
60+
- Create multi-step workflows and automation
61+
62+
Tool calling follows [OpenAI's function calling format](https://platform.openai.com/docs/guides/function-calling), making it easy to integrate with existing AI development workflows and tools.
63+
4864
## Demo
4965

5066
Watch the demo presented during [Diagrid's Dapr v1.15 celebration](https://www.diagrid.io/videos/dapr-1-15-deep-dive) to see how the conversation API works using the .NET SDK.

‎daprdocs/content/en/operations/support/alpha-beta-apis.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,7 @@ description: "List of current alpha and beta APIs"
1717
| Cryptography | [Crypto proto](https://github.com/dapr/dapr/blob/5aba3c9aa4ea9b3f388df125f9c66495b43c5c9e/dapr/proto/runtime/v1/dapr.proto#L118) | `v1.0-alpha1/crypto` | The cryptography API enables you to perform **high level** cryptography operations for encrypting and decrypting messages. | [Cryptography API]({{% ref "cryptography-overview.md" %}}) | v1.11 |
1818
| Jobs | [Jobs proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L212-219) | `v1.0-alpha1/jobs` | The jobs API enables you to schedule and orchestrate jobs. | [Jobs API]({{% ref "jobs-overview.md" %}}) | v1.14 |
1919
| Streaming Subscription | [Streaming Subscription proto](https://github.com/dapr/dapr/blob/310c83140b2f0c3cb7d2bef19624df88af3e8e0a/dapr/proto/runtime/v1/dapr.proto#L454) | N/A | Subscription is defined in the application code. Streaming subscriptions are dynamic, meaning they allow for adding or removing subscriptions at runtime. | [Streaming Subscription API]({{% ref "subscription-methods/#streaming-subscriptions" %}}) | v1.14 |
20-
| Conversation | [Conversation proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L221-222) | `v1.0-alpha1/conversation` | Converse between different large language models using the conversation API. | [Conversation API]({{% ref "conversation-overview.md" %}}) | v1.15 |
21-
20+
| Conversation | [Conversation proto](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto#L226) | `v1.0-alpha2/conversation` | Converse between different large language models using the conversation API. | [Conversation API]({{% ref "conversation-overview.md" %}}) | v1.15 |
2221

2322
## Beta APIs
2423

‎daprdocs/content/en/reference/api/conversation_api.md‎

Lines changed: 156 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,16 @@ weight: 1400
1010
The conversation API is currently in [alpha]({{% ref "certification-lifecycle.md#certification-levels" %}}).
1111
{{% /alert %}}
1212

13-
Dapr provides an API to interact with Large Language Models (LLMs) and enables critical performance and security functionality with features like prompt caching and PII data obfuscation.
13+
Dapr provides an API to interact with Large Language Models (LLMs) and enables critical performance and security functionality with features like prompt caching, PII data obfuscation, and tool calling capabilities.
14+
15+
Tool calling follow's OpenAI's function calling format, making it easy to integrate with existing AI development workflows and tools.
1416

1517
## Converse
1618

17-
This endpoint lets you converse with LLMs.
19+
This endpoint lets you converse with LLMs using the Alpha2 version of the API, which provides enhanced tool calling support and alignment with OpenAI's interface.
1820

1921
```
20-
POST http://localhost:<daprPort>/v1.0-alpha1/conversation/<llm-name>/converse
22+
POST http://localhost:<daprPort>/v1.0-alpha2/conversation/<llm-name>/converse
2123
```
2224

2325
### URL parameters
@@ -30,34 +32,117 @@ POST http://localhost:<daprPort>/v1.0-alpha1/conversation/<llm-name>/converse
3032

3133
| Field | Description |
3234
| --------- | ----------- |
35+
| `name` | The name of the conversation component. Required |
36+
| `contextId` | The ID of an existing chat (like in ChatGPT). Optional |
3337
| `inputs` | Inputs for the conversation. Multiple inputs at one time are supported. Required |
34-
| `cacheTTL` | A time-to-live value for a prompt cache to expire. Uses Golang duration format. Optional |
35-
| `scrubPII` | A boolean value to enable obfuscation of sensitive information returning from the LLM. Set this value if all PII (across contents) in the request needs to be scrubbed. Optional |
36-
| `temperature` | A float value to control the temperature of the model. Used to optimize for consistency and creativity. Optional |
38+
| `parameters` | Parameters for all custom fields. Optional |
3739
| `metadata` | [Metadata](#metadata) passed to conversation components. Optional |
40+
| `scrubPii` | A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional |
41+
| `temperature` | A float value to control the temperature of the model. Used to optimize for consistency (0) or creativity (1). Optional |
42+
| `tools` | Tools register the tools available to be used by the LLM during the conversation. Optional |
43+
| `toolChoice` | Controls which (if any) tool is called by the model. Values: `auto`, `required`, or specific tool name. Defaults to `auto` if tools are present. Optional |
3844

3945
#### Input body
4046

4147
| Field | Description |
4248
| --------- | ----------- |
43-
| `content` | The message content to send to the LLM. Required |
44-
| `role` | The role for the LLM to assume. Possible values: 'user', 'tool', 'assistant' |
45-
| `scrubPII` | A boolean value to enable obfuscation of sensitive information present in the content field. Set this value if PII for this specific content needs to be scrubbed exclusively. Optional |
49+
| `messages` | Array of conversation messages. Required |
50+
| `scrubPii` | A boolean value to enable obfuscation of sensitive information present in the content field. Optional |
51+
52+
#### Message types
53+
54+
The API supports different message types:
55+
56+
- **`ofDeveloper`**: Developer role messages with optional name and content
57+
- **`ofSystem`**: System role messages with optional name and content
58+
- **`ofUser`**: User role messages with optional name and content
59+
- **`ofAssistant`**: Assistant role messages with optional name, content, and tool calls
60+
- **`ofTool`**: Tool role messages with tool ID, name, and content
61+
62+
#### Tool calling
63+
64+
Tools can be defined using the `tools` field with function definitions:
65+
66+
| Field | Description |
67+
| --------- | ----------- |
68+
| `function.name` | The name of the function to be called. Required |
69+
| `function.description` | A description of what the function does. Optional |
70+
| `function.parameters` | JSON Schema object describing the function parameters. Optional |
4671

47-
### Request content example
72+
### Request content examples
73+
74+
#### Basic conversation
4875

4976
```json
5077
REQUEST = {
51-
"inputs": [
52-
{
53-
"content": "What is Dapr?",
54-
"role": "user", // Optional
55-
"scrubPII": "true", // Optional. Will obfuscate any sensitive information found in the content field
78+
"name": "openai",
79+
"inputs": [{
80+
"messages": [{
81+
"of_user": {
82+
"content": [{
83+
"text": "What is Dapr?"
84+
}]
85+
}
86+
}]
87+
}],
88+
"parameters": {},
89+
"metadata": {}
90+
}
91+
```
92+
93+
#### Conversation with tool calling
94+
95+
```json
96+
{
97+
"name": "openai",
98+
"inputs": [{
99+
"messages": [{
100+
"of_user": {
101+
"content": [{
102+
"text": "What is the weather like in San Francisco in celsius?"
103+
}]
104+
}
105+
}],
106+
"scrub_pii": false
107+
}],
108+
"parameters": {
109+
"max_tokens": {
110+
"@type": "type.googleapis.com/google.protobuf.Int64Value",
111+
"value": "100"
56112
},
57-
],
58-
"cacheTTL": "10m", // Optional
59-
"scrubPII": "true", // Optional. Will obfuscate any sensitive information returning from the LLM
60-
"temperature": 0.5 // Optional. Optimizes for consistency (0) or creativity (1)
113+
"model": {
114+
"@type": "type.googleapis.com/google.protobuf.StringValue",
115+
"value": "claude-3-5-sonnet-20240620"
116+
}
117+
},
118+
"metadata": {
119+
"api_key": "test-key",
120+
"version": "1.0"
121+
},
122+
"scrub_pii": false,
123+
"temperature": 0.7,
124+
"tools": [{
125+
"function": {
126+
"name": "get_weather",
127+
"description": "Get the current weather for a location",
128+
"parameters": {
129+
"type": "object",
130+
"properties": {
131+
"location": {
132+
"type": "string",
133+
"description": "The city and state, e.g. San Francisco, CA"
134+
},
135+
"unit": {
136+
"type": "string",
137+
"enum": ["celsius", "fahrenheit"],
138+
"description": "The temperature unit to use"
139+
}
140+
},
141+
"required": ["location"]
142+
}
143+
}
144+
}],
145+
"tool_choice": "auto"
61146
}
62147
```
63148

@@ -71,21 +156,62 @@ Code | Description
71156

72157
### Response content
73158

159+
#### Basic conversation response
160+
74161
```json
75-
RESPONSE = {
76-
"outputs": {
77-
{
78-
"result": "Dapr is distribution application runtime ...",
79-
"parameters": {},
80-
},
81-
{
82-
"result": "Dapr can help developers ...",
83-
"parameters": {},
84-
}
85-
},
162+
RESPONSE = {
163+
"outputs": [{
164+
"choices": [{
165+
"finish_reason": "stop",
166+
"index": 0,
167+
"message": {
168+
"content": "Dapr is a distributed application runtime that makes it easy for developers to build resilient, stateless and stateful applications that run on the cloud and edge.",
169+
"tool_calls": []
170+
}
171+
}]
172+
}]
173+
}
174+
```
175+
176+
#### Tool calling response
177+
178+
```json
179+
{
180+
"outputs": [{
181+
"choices": [{
182+
"finish_reason": "tool_calls",
183+
"index": 0,
184+
"message": {
185+
"content": null,
186+
"tool_calls": [{
187+
"id": "call_123",
188+
"function": {
189+
"name": "get_weather",
190+
"arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"celsius\"}"
191+
}
192+
}]
193+
}
194+
}]
195+
}]
86196
}
87197
```
88198

199+
### Tool choice options
200+
201+
The `tool_choice` is an optional parameter that controls how the model can use available tools:
202+
203+
- **`auto`**: The model can pick between generating a message or calling one or more tools (default when tools are present)
204+
- **`required`**: Requires one or more functions to be called
205+
- **`{tool_name}`**: Forces the model to call a specific tool by name
206+
207+
## Legacy Alpha1 API
208+
209+
The previous Alpha1 version of the API is still supported for backward compatibility but is deprecated. For new implementations, use the Alpha2 version described above.
210+
211+
```
212+
POST http://localhost:<daprPort>/v1.0-alpha1/conversation/<llm-name>/converse
213+
```
214+
89215
## Next steps
90216

91217
- [Conversation API overview]({{% ref conversation-overview.md %}})

0 commit comments

Comments
 (0)