Skip to content

Commit 4130efb

Browse files
committed
feature #1759 [Platform] Replace untyped stream chunks with semantic DeltaInterface deltas (fabpot, chr-hertel)
This PR was squashed before being merged into the main branch. Discussion ---------- [Platform] Replace untyped stream chunks with semantic DeltaInterface deltas | Q | A | ------------- | --- | Bug fix? | yes | New feature? | yes <!-- please update src/**/CHANGELOG.md files --> | Docs? | yes/no <!-- required for new features --> | Issues | n/a | License | MIT This PR introduces semantic streaming event objects so that one can react to specific LLM events without parsing the raw JSON. Think of this PR as being a first step towards full streaming support ;) Commits ------- 232275d [Platform] Replace untyped stream chunks with semantic DeltaInterface deltas
2 parents 6239534 + 232275d commit 4130efb

111 files changed

Lines changed: 1868 additions & 686 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

UPGRADE.md

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,12 @@ Agent
2727
```
2828

2929
* The `SimilaritySearch` tool now accepts an optional `$promptTemplate` parameter to customize the result header (default: `'Found documents with the following information:'`)
30+
* `Toolbox\StreamListener::onChunk()` has been renamed to `onDelta()`.
31+
The listener now checks `$delta instanceof TextDelta` instead of
32+
`is_string($chunk)` to accumulate the assistant text buffer.
33+
* The `StreamListener` now reacts to `ToolCallComplete` instead of
34+
`ToolCallResult`. If you have a custom `StreamListener` or check for
35+
`ToolCallResult` in stream consumers, update to `ToolCallComplete`.
3036

3137
Mate
3238
----
@@ -42,13 +48,76 @@ AI Bundle
4248
* The `endpoint` option for `Ollama` is now `null` by default to allow the usage of a `ScopingHttpClient`
4349

4450
Platform
45-
-----
51+
--------
4652

4753
* `ModelCatalog` in `Ollama` has been replaced by `OllamaApiCatalog`
4854
* `OllamaApiCatalog` in `Ollama` has been replaced to `ModelCatalog`
4955
* `Ollama` model is now `final`
5056
* `ModelCatalog` in `ElevenLabs` has been replaced by `ElevenLabsApiCatalog`
5157
* `ElevenLabsApiCatalog` in `ElevenLabs` has been replaced to `ModelCatalog`
58+
* `ChunkEvent` has been replaced by `DeltaEvent`. The `onChunk(ChunkEvent)` method
59+
on `ListenerInterface` is now `onDelta(DeltaEvent)`. Update all implementations:
60+
- `onChunk(ChunkEvent $event)``onDelta(DeltaEvent $event)`
61+
- `$event->getChunk()``$event->getDelta()`
62+
- `$event->setChunk(...)``$event->setDelta(...)`
63+
- `$event->skipChunk()``$event->skipDelta()`
64+
- `$event->isChunkSkipped()``$event->isDeltaSkipped()`
65+
* Stream generators in bridge `ResultConverter` classes now yield typed
66+
`DeltaInterface` objects instead of raw strings. Consumers iterating
67+
`StreamResult::getContent()` that expect `string` chunks must check for
68+
`TextDelta` instead:
69+
- `is_string($chunk)``$chunk instanceof TextDelta`, then use `$chunk->getText()`
70+
* New streaming delta types in `Symfony\AI\Platform\Result\Stream\Delta\`:
71+
`TextDelta`, `ThinkingDelta`, `ThinkingSignature`, `ToolCallStart`,
72+
`ToolInputDelta`, `BinaryDelta`, `ChoiceDelta`, `ToolCallComplete`,
73+
`ThinkingComplete`. These are yielded by bridge converters during streaming.
74+
* `Symfony\AI\Platform\Bridge\Ollama\OllamaMessageChunk` has been removed.
75+
Ollama streams now yield semantic deltas like the other bridges:
76+
- text content → `TextDelta`
77+
- thinking content → `ThinkingDelta`
78+
- completed tool calls → `ToolCallComplete`
79+
- final usage → `TokenUsage`
80+
81+
```diff
82+
-if ($chunk instanceof OllamaMessageChunk) {
83+
- echo $chunk->getContent();
84+
-}
85+
+if ($chunk instanceof TextDelta) {
86+
+ echo $chunk->getText();
87+
+}
88+
```
89+
* `Symfony\AI\Platform\Bridge\Perplexity\PerplexitySearchResults`,
90+
`Symfony\AI\Platform\Bridge\Perplexity\PerplexityCitations`, and
91+
`Symfony\AI\Platform\Bridge\Perplexity\StreamListener` have been
92+
removed. Perplexity now emits generic `MetadataDelta` instances internally,
93+
which are promoted to result metadata during stream consumption.
94+
95+
If you relied on those deltas directly, read metadata instead:
96+
97+
```diff
98+
-if ($delta instanceof PerplexitySearchResults) {
99+
- $results = $delta->getSearchResults();
100+
-}
101+
-if ($delta instanceof PerplexityCitations) {
102+
- $citations = $delta->getCitations();
103+
-}
104+
+$results = $result->getMetadata()->get('search_results');
105+
+$citations = $result->getMetadata()->get('citations');
106+
```
107+
* `BinaryResult`, `ChoiceResult`, `ToolCallResult`, and `ThinkingContent` no
108+
longer implement `DeltaInterface`. Stream generators now yield proper delta
109+
types instead of result objects:
110+
- `ToolCallResult``ToolCallComplete`: use `$delta instanceof ToolCallComplete`
111+
and `$delta->getToolCalls()`.
112+
- `ThinkingContent``ThinkingComplete`: use `$delta instanceof ThinkingComplete`
113+
and `$delta->getThinking()` / `$delta->getSignature()`.
114+
- `BinaryResult``BinaryDelta`: use `$delta instanceof BinaryDelta`
115+
and `$delta->getData()` / `$delta->getMimeType()`.
116+
- `ChoiceResult``ChoiceDelta`: use `$delta instanceof ChoiceDelta`
117+
and `$delta->getDeltas()`.
118+
* `ThinkingContent` has been removed in favor of `ThinkingComplete`.
119+
* The `Usage` delta class has been removed. Bridges now yield `TokenUsage`
120+
objects directly in streams instead.
52121

53122
Store
54123
-----

demo/src/Blog/Command/StreamCommand.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
use Symfony\AI\Agent\AgentInterface;
1515
use Symfony\AI\Platform\Message\Message;
1616
use Symfony\AI\Platform\Message\MessageBag;
17+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1718
use Symfony\AI\Platform\Result\StreamResult;
1819
use Symfony\Component\Console\Attribute\AsCommand;
1920
use Symfony\Component\Console\Style\SymfonyStyle;
@@ -45,8 +46,10 @@ public function __invoke(SymfonyStyle $io): int
4546

4647
\assert($result instanceof StreamResult);
4748

48-
foreach ($result->getContent() as $word) {
49-
$io->write($word);
49+
foreach ($result->getContent() as $delta) {
50+
if ($delta instanceof TextDelta) {
51+
$io->write((string) $delta);
52+
}
5053
}
5154

5255
$io->newLine(2);

demo/src/Stream/Chat.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
use Symfony\AI\Platform\Message\Message;
1717
use Symfony\AI\Platform\Message\MessageBag;
1818
use Symfony\AI\Platform\Message\UserMessage;
19+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1920
use Symfony\Component\DependencyInjection\Attribute\Autowire;
2021
use Symfony\Component\HttpFoundation\RequestStack;
2122

@@ -56,8 +57,10 @@ public function getAssistantResponse(MessageBag $messages): \Generator
5657
\assert(is_iterable($stream));
5758

5859
$response = '';
59-
foreach ($stream as $chunk) {
60-
yield $response .= $chunk;
60+
foreach ($stream as $delta) {
61+
if ($delta instanceof TextDelta) {
62+
yield $response .= (string) $delta;
63+
}
6164
}
6265

6366
$assistantMessage = Message::ofAssistant($response);

demo/tests/Blog/Command/StreamCommandTest.php

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
use Symfony\AI\Platform\Metadata\Metadata;
1818
use Symfony\AI\Platform\Result\RawResultInterface;
1919
use Symfony\AI\Platform\Result\ResultInterface;
20+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
2021
use Symfony\Component\Console\Input\ArrayInput;
2122
use Symfony\Component\Console\Output\BufferedOutput;
2223
use Symfony\Component\Console\Style\SymfonyStyle;
@@ -31,10 +32,10 @@ public function testStreamCommandOutputsStreamedContentAndSuccess()
3132
->willReturn(new class implements ResultInterface {
3233
public function getContent(): iterable
3334
{
34-
yield 'Hello';
35-
yield ' ';
36-
yield 'world';
37-
yield '!';
35+
yield new TextDelta('Hello');
36+
yield new TextDelta(' ');
37+
yield new TextDelta('world');
38+
yield new TextDelta('!');
3839
}
3940

4041
public function getMetadata(): Metadata

examples/amazeeai/stream.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
use Symfony\AI\Platform\Bridge\AmazeeAi\PlatformFactory;
1313
use Symfony\AI\Platform\Message\Message;
1414
use Symfony\AI\Platform\Message\MessageBag;
15+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1516

1617
require_once dirname(__DIR__).'/bootstrap.php';
1718

@@ -22,7 +23,9 @@
2223
'stream' => true,
2324
]);
2425

25-
foreach ($result->asStream() as $word) {
26-
echo $word;
26+
foreach ($result->asStream() as $delta) {
27+
if ($delta instanceof TextDelta) {
28+
echo $delta;
29+
}
2730
}
2831
echo \PHP_EOL;

examples/anthropic/stream.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
use Symfony\AI\Platform\Bridge\Anthropic\PlatformFactory;
1313
use Symfony\AI\Platform\Message\Message;
1414
use Symfony\AI\Platform\Message\MessageBag;
15+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1516

1617
require_once dirname(__DIR__).'/bootstrap.php';
1718

@@ -23,7 +24,9 @@
2324
);
2425
$result = $platform->invoke('claude-sonnet-4-5-20250929', $messages, ['stream' => true]);
2526

26-
foreach ($result->asStream() as $word) {
27-
echo $word;
27+
foreach ($result->asStream() as $delta) {
28+
if ($delta instanceof TextDelta) {
29+
echo $delta;
30+
}
2831
}
2932
echo \PHP_EOL;

examples/anthropic/toolcall-stream.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
use Symfony\AI\Platform\Bridge\Anthropic\PlatformFactory;
1717
use Symfony\AI\Platform\Message\Message;
1818
use Symfony\AI\Platform\Message\MessageBag;
19+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1920

2021
require_once dirname(__DIR__).'/bootstrap.php';
2122

@@ -34,8 +35,10 @@
3435
'stream' => true, // enable streaming of response text
3536
]);
3637

37-
foreach ($result->getContent() as $word) {
38-
echo $word;
38+
foreach ($result->getContent() as $delta) {
39+
if ($delta instanceof TextDelta) {
40+
echo $delta;
41+
}
3942
}
4043

4144
echo \PHP_EOL;

examples/cerebras/stream.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
use Symfony\AI\Platform\Bridge\Cerebras\PlatformFactory;
1313
use Symfony\AI\Platform\Message\Message;
1414
use Symfony\AI\Platform\Message\MessageBag;
15+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1516

1617
require_once dirname(__DIR__).'/bootstrap.php';
1718

@@ -26,7 +27,9 @@
2627
'stream' => true,
2728
]);
2829

29-
foreach ($result->asStream() as $word) {
30-
echo $word;
30+
foreach ($result->asStream() as $delta) {
31+
if ($delta instanceof TextDelta) {
32+
echo $delta;
33+
}
3134
}
3235
echo \PHP_EOL;

examples/chat/stream-chat.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
use Symfony\AI\Platform\Bridge\OpenAi\PlatformFactory;
1616
use Symfony\AI\Platform\Message\Message;
1717
use Symfony\AI\Platform\Message\MessageBag;
18+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1819

1920
require_once dirname(__DIR__).'/bootstrap.php';
2021

@@ -30,7 +31,9 @@
3031
$chat->initiate($messages);
3132
$chat->submit(Message::ofUser('My name is Christopher.'));
3233

33-
foreach ($chat->stream(Message::ofUser('Tell me a story about the sun')) as $chunk) {
34-
echo $chunk;
34+
foreach ($chat->stream(Message::ofUser('Tell me a story about the sun')) as $delta) {
35+
if ($delta instanceof TextDelta) {
36+
echo $delta;
37+
}
3538
}
3639
echo \PHP_EOL;

examples/claude-code/stream.php

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
use Symfony\AI\Platform\Bridge\ClaudeCode\PlatformFactory;
1313
use Symfony\AI\Platform\Message\Message;
1414
use Symfony\AI\Platform\Message\MessageBag;
15+
use Symfony\AI\Platform\Result\Stream\Delta\TextDelta;
1516

1617
require_once dirname(__DIR__).'/bootstrap.php';
1718

@@ -30,7 +31,9 @@
3031
'max_turns' => 1,
3132
]);
3233

33-
foreach ($result->asStream() as $word) {
34-
echo $word;
34+
foreach ($result->asStream() as $delta) {
35+
if ($delta instanceof TextDelta) {
36+
echo $delta;
37+
}
3538
}
3639
echo \PHP_EOL;

0 commit comments

Comments
 (0)