File tree Expand file tree Collapse file tree 4 files changed +35
-96
lines changed
Expand file tree Collapse file tree 4 files changed +35
-96
lines changed Original file line number Diff line number Diff line change @@ -33,14 +33,22 @@ Sampling requests use a standardized message format:
3333 {
3434 role: " user" | " assistant" ,
3535 content: {
36- type: " text" | " image" ,
36+ type: " text" | " image" | " audio " | " resource " ,
3737
3838 // For text:
3939 text?: string ,
4040
41- // For images:
41+ // For images/audio :
4242 data?: string , // base64 encoded
43- mimeType?: string
43+ mimeType?: string ,
44+
45+ // For resources:
46+ resource?: {
47+ uri: string ,
48+ mimeType: string ,
49+ text?: string , // for text resources
50+ blob?: string // for blob resources, base64 encoded
51+ }
4452 }
4553 }
4654 ],
@@ -70,7 +78,8 @@ The `messages` array contains the conversation history to send to the LLM. Each
7078- ` role ` : Either "user" or "assistant"
7179- ` content ` : The message content, which can be:
7280 - Text content with a ` text ` field
73- - Image content with ` data ` (base64) and ` mimeType ` fields
81+ - Image/audio content with ` data ` (base64) and ` mimeType ` fields
82+ - An embedded [ resource] ( /docs/concepts/resources )
7483
7584### Model preferences
7685
You can’t perform that action at this time.
0 commit comments