Skip to content

Commit 9d77690

Browse files
committed
Merge remote-tracking branch 'origin/main' into feat/server-multimodal
2 parents f99a484 + d9bb489 commit 9d77690

File tree

191 files changed

+6989
-2548
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

191 files changed

+6989
-2548
lines changed

assets/prism-logo.webp

-94 Bytes
Loading

composer.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
],
2020
"require": {
2121
"php": "^8.2",
22+
"ext-fileinfo": "*",
2223
"laravel/framework": "^11.0|^12.0"
2324
},
2425
"config": {

docs/.vitepress/config.mts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,10 @@ export default defineConfig({
114114
text: "Embeddings",
115115
link: "/core-concepts/embeddings",
116116
},
117+
{
118+
text: "Image Generation",
119+
link: "/core-concepts/image-generation",
120+
},
117121
{
118122
text: "Schemas",
119123
link: "/core-concepts/schemas",

docs/advanced/custom-providers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,5 +70,5 @@ use Prism\Prism\Facades\Prism;
7070
$response = Prism::text()
7171
->using('my-custom-provider', 'model-name')
7272
->withPrompt('Hello, custom AI!')
73-
->generate();
73+
->asText();
7474
```

docs/advanced/rate-limits.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ try {
3838
Prism::text()
3939
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
4040
->withPrompt('Hello world!')
41-
->generate();
41+
->asText();
4242
}
4343
catch (PrismRateLimitedException $e) {
4444
/** @var ProviderRateLimit $rate_limit */
@@ -110,7 +110,7 @@ use Prism\Prism\ValueObjects\ProviderRateLimit;
110110
$response = Prism::text()
111111
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
112112
->withPrompt('Hello world!')
113-
->generate();
113+
->asText();
114114

115115
/** @var ProviderRateLimit $rate_limit */
116116
foreach ($response->meta->rateLimits as $rate_limit) {

docs/components/ProviderSupport.vue

Lines changed: 33 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -169,20 +169,10 @@ export default {
169169
"Documents",
170170
],
171171
providers: [
172-
{
173-
name: "Amazon Bedrock",
174-
text: Planned,
175-
streaming: Unsupported,
176-
structured: Unsupported,
177-
embeddings: Planned,
178-
image: Planned,
179-
tools: Planned,
180-
documents: Unsupported,
181-
},
182172
{
183173
name: "Anthropic",
184174
text: Supported,
185-
streaming: Planned,
175+
streaming: Supported,
186176
structured: Adapted,
187177
embeddings: Unsupported,
188178
image: Supported,
@@ -200,12 +190,42 @@ export default {
200190
documents: Unsupported,
201191
},
202192
{
203-
name: "DeepSeek",
193+
name: "Bedrock - Anthropic",
204194
text: Supported,
205195
streaming: Unsupported,
206196
structured: Supported,
207197
embeddings: Unsupported,
198+
image: Supported,
199+
tools: Supported,
200+
documents: Unsupported,
201+
},
202+
{
203+
name: "Bedrock - Cohere",
204+
text: Unsupported,
205+
streaming: Unsupported,
206+
structured: Unsupported,
207+
embeddings: Supported,
208208
image: Unsupported,
209+
tools: Unsupported,
210+
documents: Unsupported,
211+
},
212+
{
213+
name: "Bedrock - Converse",
214+
text: Supported,
215+
streaming: Unsupported,
216+
structured: Supported,
217+
embeddings: Unsupported,
218+
image: Supported,
219+
tools: Supported,
220+
documents: Supported,
221+
},
222+
{
223+
name: "DeepSeek",
224+
text: Supported,
225+
streaming: Unsupported,
226+
structured: Supported,
227+
embeddings: Unsupported,
228+
image: Supported,
209229
tools: Supported,
210230
documents: Unsupported,
211231
},
@@ -257,7 +277,7 @@ export default {
257277
embeddings: Supported,
258278
image: Supported,
259279
tools: Supported,
260-
documents: Unsupported,
280+
documents: Supported,
261281
},
262282
{
263283
name: "VoyageAI",
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# Image Generation
2+
3+
Generate stunning images from text prompts using AI-powered models. Prism provides a clean, consistent API for image generation across different providers, starting with comprehensive OpenAI support.
4+
5+
## Getting Started
6+
7+
Creating images with Prism is as simple as describing what you want:
8+
9+
```php
10+
use Prism\Prism\Prism;
11+
12+
$response = Prism::image()
13+
->using('openai', 'dall-e-3')
14+
->withPrompt('A cute baby sea otter floating on its back in calm blue water')
15+
->generate();
16+
17+
$image = $response->firstImage();
18+
echo $image->url; // https://oaidalleapiprodscus.blob.core.windows.net/...
19+
```
20+
21+
## Provider Support
22+
23+
Currently, Prism supports image generation through:
24+
25+
- **OpenAI**: DALL-E 2, DALL-E 3, and GPT-Image-1 models
26+
27+
Additional providers will be added in future releases as the ecosystem evolves.
28+
29+
## Basic Usage
30+
31+
### Simple Generation
32+
33+
The most straightforward way to generate an image:
34+
35+
```php
36+
$response = Prism::image()
37+
->using('openai', 'dall-e-3')
38+
->withPrompt('A serene mountain landscape at sunset')
39+
->generate();
40+
41+
// Access the generated image
42+
$image = $response->firstImage();
43+
if ($image->hasUrl()) {
44+
echo "Image URL: " . $image->url;
45+
}
46+
if ($image->hasBase64()) {
47+
echo "Base64 Image Data: " . $image->base64;
48+
}
49+
```
50+
51+
### Working with Responses
52+
53+
The response object provides helpful methods for accessing generated content:
54+
55+
```php
56+
$response = Prism::image()
57+
->using('openai', 'dall-e-2')
58+
->withPrompt('Abstract geometric patterns in vibrant colors')
59+
->generate();
60+
61+
// Check if images were generated
62+
if ($response->hasImages()) {
63+
echo "Generated {$response->imageCount()} image(s)";
64+
65+
// Access all images
66+
foreach ($response->images as $image) {
67+
if ($image->hasUrl()) {
68+
echo "Image: {$image->url}\n";
69+
}
70+
71+
if ($image->hasBase64()) {
72+
echo "Base64 Image: " . substr($image->base64, 0, 50) . "...\n";
73+
}
74+
75+
if ($image->hasRevisedPrompt()) {
76+
echo "Revised prompt: {$image->revisedPrompt}\n";
77+
}
78+
}
79+
80+
// Or just get the first one
81+
$firstImage = $response->firstImage();
82+
}
83+
84+
// Check usage information
85+
echo "Prompt tokens: {$response->usage->promptTokens}";
86+
echo "Model used: {$response->meta->model}";
87+
```
88+
89+
## Provider-Specific Options
90+
91+
While Prism provides a consistent API, you can access provider-specific features using the `withProviderOptions()` method.
92+
93+
### OpenAI Options
94+
95+
OpenAI offers various customization options depending on the model:
96+
97+
#### DALL-E 3 Options
98+
99+
```php
100+
$response = Prism::image()
101+
->using('openai', 'dall-e-3')
102+
->withPrompt('A beautiful sunset over mountains')
103+
->withProviderOptions([
104+
'size' => '1792x1024', // 1024x1024, 1024x1792, 1792x1024
105+
'quality' => 'hd', // standard, hd
106+
'style' => 'vivid', // vivid, natural
107+
'response_format' => 'url', // url, b64_json
108+
])
109+
->generate();
110+
```
111+
112+
#### GPT-Image-1 (Base64 Only)
113+
114+
The GPT-Image-1 model always returns base64-encoded images, regardless of the `response_format` setting:
115+
116+
```php
117+
$response = Prism::image()
118+
->using('openai', 'gpt-image-1')
119+
->withPrompt('A cute baby sea otter floating on its back')
120+
->withProviderOptions([
121+
'size' => '1024x1024', // 1024x1024, 1536x1024, 1024x1536, auto
122+
'quality' => 'high', // auto, high, medium, low
123+
'background' => 'transparent', // transparent, opaque, auto
124+
'output_format' => 'png', // png, jpeg, webp
125+
'output_compression' => 90, // 0-100 (for jpeg/webp)
126+
])
127+
->generate();
128+
129+
$image = $response->firstImage();
130+
if ($image->hasBase64()) {
131+
// Save the base64 image to a file
132+
file_put_contents('generated-image.png', base64_decode($image->base64));
133+
echo "Base64 image saved to generated-image.png";
134+
}
135+
```
136+
137+
#### Base64 vs URL Responses
138+
139+
Different models return images in different formats:
140+
141+
- **GPT-Image-1**: Always returns base64-encoded images in the `base64` property
142+
- **DALL-E 2 & 3**: Return URLs by default, but can return base64 when `response_format` is set to `'b64_json'`
143+
144+
```php
145+
// Request base64 format from DALL-E 3
146+
$response = Prism::image()
147+
->using('openai', 'dall-e-3')
148+
->withPrompt('Abstract art')
149+
->withProviderOptions([
150+
'response_format' => 'b64_json',
151+
])
152+
->generate();
153+
154+
$image = $response->firstImage();
155+
if ($image->hasBase64()) {
156+
echo "Received base64 image data";
157+
}
158+
```
159+
160+
## Testing
161+
162+
Prism provides convenient fakes for testing image generation:
163+
164+
```php
165+
use Prism\Prism\Prism;
166+
use Prism\Prism\Testing\PrismFake;
167+
168+
test('can generate images', function () {
169+
$fake = PrismFake::create()->image();
170+
Prism::fake($fake);
171+
172+
$response = Prism::image()
173+
->using('openai', 'dall-e-3')
174+
->withPrompt('Test image')
175+
->generate();
176+
177+
expect($response->hasImages())->toBeTrue();
178+
expect($response->firstImage()->url)->toContain('fake-image-url');
179+
});
180+
```
181+
182+
Need help with a specific provider or use case? Check the [provider documentation](/providers/openai) for detailed configuration options and examples.

docs/core-concepts/streaming-output.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,13 @@ foreach ($response as $chunk) {
3232
// The text fragment in this chunk
3333
echo $chunk->text;
3434

35+
if ($chunk->usage) {
36+
echo "Prompt tokens: " . $chunk->usage->promptTokens;
37+
echo "Completion tokens: " . $chunk->usage->completionTokens;
38+
}
39+
3540
// Check if this is the final chunk
36-
if ($chunk->finishReason) {
41+
if ($chunk->finishReason === FinishReason::Stop) {
3742
echo "Generation complete: " . $chunk->finishReason->name;
3843
}
3944
}

docs/core-concepts/structured-output.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,10 @@ Different AI providers handle structured output in two main ways:
5151
5252
## Provider-Specific Options
5353

54-
Providers may offer additional options for structured output. For example, OpenAI supports a "strict mode" for even tighter schema validation:
54+
Providers may offer additional options for structured output:
55+
56+
### OpenAI: Strict Mode
57+
OpenAI supports a "strict mode" for even tighter schema validation:
5558

5659
```php
5760
use Prism\Prism\Prism;
@@ -66,6 +69,29 @@ $response = Prism::structured()
6669
// ... rest of your configuration
6770
```
6871

72+
### Anthropic: Tool Calling Mode
73+
Anthropic doesn't have native structured output, but Prism provides two approaches. For more reliable JSON parsing, especially with complex content or non-English text, use tool calling mode:
74+
75+
```php
76+
use Prism\Prism\Prism;
77+
use Prism\Prism\Enums\Provider;
78+
79+
$response = Prism::structured()
80+
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
81+
->withSchema($schema)
82+
->withPrompt('天氣怎麼樣?應該穿什麼?') // Chinese text with potential quotes
83+
->withProviderOptions(['use_tool_calling' => true])
84+
->asStructured();
85+
```
86+
87+
**When to use tool calling mode with Anthropic:**
88+
- Working with non-English content that may contain quotes
89+
- Complex JSON structures that might confuse prompt-based parsing
90+
- When you need the most reliable structured output possible
91+
92+
> [!NOTE]
93+
> Tool calling mode cannot be used with Anthropic's citations feature.
94+
6995
> [!TIP]
7096
> Check the provider-specific documentation pages for additional options and features that might be available for structured output.
7197

0 commit comments

Comments
 (0)