You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: daprdocs/content/en/developing-applications/building-blocks/conversation/conversation-overview.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,10 +20,10 @@ In addition to enabling critical performance and security functionality (like [p
20
20
-**OpenAI-compatible interface** for seamless integration with existing AI workflows and tools
21
21
22
22
You can also pair the conversation API with Dapr functionalities, like:
23
-
- Resiliency circuit breakers and retries to circumvent limit and token errors, or
24
-
- Middleware to authenticate requests coming to and from the LLM
25
23
26
-
Dapr provides observability by issuing metrics for your LLM interactions.
24
+
- Resiliency policies including circuit breakers to handle repeated errors, timeouts to safeguards from slow responses, and retries for temporary network failures
25
+
- Observability with metrics and distributed tracing using OpenTelemetry and Zipkin
26
+
- Middleware to authenticate requests to and from the LLM
27
27
28
28
## Features
29
29
@@ -67,7 +67,7 @@ Watch the demo presented during [Diagrid's Dapr v1.15 celebration](https://www.d
@@ -12,7 +12,7 @@ The conversation API is currently in [alpha]({{% ref "certification-lifecycle.md
12
12
13
13
Dapr provides an API to interact with Large Language Models (LLMs) and enables critical performance and security functionality with features like prompt caching, PII data obfuscation, and tool calling capabilities.
14
14
15
-
Tool calling follow's OpenAI's function calling format, making it easy to integrate with existing AI development workflows and tools.
15
+
Tool calling follows OpenAI's function calling format, making it easy to integrate with existing AI development workflows and tools.
16
16
17
17
## Converse
18
18
@@ -32,32 +32,34 @@ POST http://localhost:<daprPort>/v1.0-alpha2/conversation/<llm-name>/converse
32
32
33
33
| Field | Description |
34
34
| --------- | ----------- |
35
-
|`name`| The name of the conversation component. Required |
36
-
|`contextId`| The ID of an existing chat (like in ChatGPT). Optional |
35
+
|`context_id`| The ID of an existing chat (like in ChatGPT). Optional |
37
36
|`inputs`| Inputs for the conversation. Multiple inputs at one time are supported. Required |
38
37
|`parameters`| Parameters for all custom fields. Optional |
39
-
|`metadata`|[Metadata](#metadata) passed to conversation components. Optional |
40
-
|`scrubPii`| A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional |
38
+
|`metadata`| Metadata passed to conversation components. Optional |
39
+
|`scrub_pii`| A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional |
41
40
|`temperature`| A float value to control the temperature of the model. Used to optimize for consistency (0) or creativity (1). Optional |
42
41
|`tools`| Tools register the tools available to be used by the LLM during the conversation. Optional |
43
-
|`toolChoice`| Controls which (if any) tool is called by the model. Values: `auto`, `required`, or specific tool name. Defaults to `auto` if tools are present. Optional |
42
+
|`tool_choice`| Controls which (if any) tool is called by the model. Values: `auto`, `required`, or specific tool name. Defaults to `auto` if tools are present. Optional |
44
43
45
44
#### Input body
46
45
47
46
| Field | Description |
48
47
| --------- | ----------- |
49
48
|`messages`| Array of conversation messages. Required |
50
-
|`scrubPii`| A boolean value to enable obfuscation of sensitive information present in the content field. Optional |
49
+
|`scrub_pii`| A boolean value to enable obfuscation of sensitive information present in the content field. Optional |
51
50
52
51
#### Message types
53
52
54
53
The API supports different message types:
55
54
56
-
-**`ofDeveloper`**: Developer role messages with optional name and content
57
-
-**`ofSystem`**: System role messages with optional name and content
58
-
-**`ofUser`**: User role messages with optional name and content
59
-
-**`ofAssistant`**: Assistant role messages with optional name, content, and tool calls
60
-
-**`ofTool`**: Tool role messages with tool ID, name, and content
55
+
| Type | Description |
56
+
| ---- | ----------- |
57
+
|`of_developer`| Developer role messages with optional name and content |
58
+
|`of_system`| System role messages with optional name and content |
59
+
|`of_user`| User role messages with optional name and content |
60
+
|`of_assistant`| Assistant role messages with optional name, content, and tool calls |
61
+
|`of_tool`| Tool role messages with tool ID, name, and content |
62
+
61
63
62
64
#### Tool calling
63
65
@@ -69,81 +71,136 @@ Tools can be defined using the `tools` field with function definitions:
69
71
|`function.description`| A description of what the function does. Optional |
70
72
|`function.parameters`| JSON Schema object describing the function parameters. Optional |
71
73
74
+
75
+
#### Tool choice options
76
+
77
+
The `tool_choice` is an optional parameter that controls how the model can use available tools:
78
+
79
+
-**`auto`**: The model can pick between generating a message or calling one or more tools (default when tools are present)
80
+
-**`required`**: Requires one or more functions to be called
81
+
-**`{tool_name}`**: Forces the model to call a specific tool by name
82
+
83
+
84
+
#### Metadata
85
+
The `metadata` field serves as a dynamic configuration mechanism that allows you to pass additional configuration and authentication information to conversation components on a per-request basis. This metadata overrides any corresponding fields configured in the component's YAML configuration file, enabling dynamic configuration without modifying static component definitions.
86
+
87
+
**Common metadata fields:**
88
+
89
+
| Field | Description | Example |
90
+
| ----- | ----------- | ------- |
91
+
|`api_key`| API key for authenticating with the LLM service |`"sk-1234567890abcdef"`|
92
+
|`model`| Specific model identifier |`"gpt-4-turbo"`, `"claude-3-sonnet"`|
93
+
|`version`| API version or service version |`"1.0"`, `"2023-12-01"`|
94
+
|`endpoint`| Custom endpoint URL for the service |`"https://api.custom-llm.com/v1"`|
95
+
96
+
{{% alert title="Note" color="primary" %}}
97
+
The exact metadata fields supported depend on the specific conversation component implementation. Refer to the component's documentation for the complete list of supported metadata fields.
98
+
{{% /alert %}}
99
+
100
+
In addition to passing metadata in the request body, you can also pass metadata as URL query parameters without modifying the request payload. Here is the format:
101
+
102
+
-**Prefix**: All metadata parameters must be prefixed with `metadata.`
103
+
-**Format**: `?metadata.<field_name>=<value>`
104
+
-**Multiple parameters**: Separate with `&` (e.g., `?metadata.api_key=sk-123&metadata.model=gpt-4`)
105
+
106
+
Example of model override:
107
+
```bash
108
+
POST http://localhost:3500/v1.0-alpha2/conversation/openai/converse?metadata.model=sk-gpt-4-turbo
109
+
```
110
+
111
+
URL metadata parameters are merged with request body metadata, URL parameters take precedence if conflicts exist, and both override component configuration in the YAML file.
112
+
72
113
### Request content examples
73
114
74
115
#### Basic conversation
75
116
76
117
```json
77
-
REQUEST = {
78
-
"name": "openai",
79
-
"inputs": [{
80
-
"messages": [{
81
-
"of_user": {
82
-
"content": [{
83
-
"text": "What is Dapr?"
84
-
}]
85
-
}
86
-
}]
87
-
}],
88
-
"parameters": {},
89
-
"metadata": {}
90
-
}
118
+
curl -X POST http://localhost:3500/v1.0-alpha2/conversation/openai/converse \
119
+
-H "Content-Type: application/json" \
120
+
-d '{
121
+
"inputs": [
122
+
{
123
+
"messages": [
124
+
{
125
+
"of_user": {
126
+
"content": [
127
+
{
128
+
"text": "What is Dapr?"
129
+
}
130
+
]
131
+
}
132
+
}
133
+
]
134
+
}
135
+
],
136
+
"parameters": {},
137
+
"metadata": {}
138
+
}'
91
139
```
92
140
93
141
#### Conversation with tool calling
94
142
95
143
```json
96
-
{
97
-
"name": "openai",
98
-
"inputs": [{
99
-
"messages": [{
100
-
"of_user": {
101
-
"content": [{
102
-
"text": "What is the weather like in San Francisco in celsius?"
"description": "Get the current weather for a location",
184
+
"parameters": {
185
+
"type": "object",
186
+
"properties": {
187
+
"location": {
188
+
"type": "string",
189
+
"description": "The city and state, e.g. San Francisco, CA"
190
+
},
191
+
"unit": {
192
+
"type": "string",
193
+
"enum": ["celsius", "fahrenheit"],
194
+
"description": "The temperature unit to use"
195
+
}
196
+
},
197
+
"required": ["location"]
198
+
}
199
+
}
200
+
}
201
+
],
202
+
"tool_choice": "auto"
203
+
}'
147
204
```
148
205
149
206
### HTTP response codes
@@ -159,50 +216,49 @@ Code | Description
159
216
#### Basic conversation response
160
217
161
218
```json
162
-
RESPONSE = {
163
-
"outputs": [{
164
-
"choices": [{
165
-
"finish_reason": "stop",
166
-
"index": 0,
167
-
"message": {
168
-
"content": "Dapr is a distributed application runtime that makes it easy for developers to build resilient, stateless and stateful applications that run on the cloud and edge.",
0 commit comments