Skip to content

Commit 02dc9da

Browse files
committed
[Agent] Add Platform tool wrapper for using platforms as tools
Introduces a new Platform tool that wraps any PlatformInterface instance, allowing agents to use specialized platforms as tools. This enables scenarios like an OpenAI agent using ElevenLabs for speech-to-text. - Add Platform tool class with support for Text, Audio, and Image results - Add OpenAI example demonstrating ElevenLabs platform as tool - Add documentation explaining the concept and usage - Update demo AI Bundle configuration with platform tool examples
1 parent 0b2a28c commit 02dc9da

File tree

4 files changed

+212
-1
lines changed

4 files changed

+212
-1
lines changed

demo/config/packages/ai.yaml

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ ai:
22
platform:
33
openai:
44
api_key: '%env(OPENAI_API_KEY)%'
5+
elevenlabs:
6+
api_key: '%env(ELEVEN_LABS_API_KEY)%'
57
agent:
68
blog:
79
model: 'gpt-4o-mini'
@@ -49,6 +51,15 @@ ai:
4951
model: 'gpt-4o-mini'
5052
prompt: 'You are a helpful general assistant. Assist users with any questions or tasks they may have.'
5153
tools: false
54+
elevenlabs_audio:
55+
model: 'gpt-4o-mini'
56+
prompt: 'You are a helpful assistant with access to speech-to-text capabilities via ElevenLabs.'
57+
tools:
58+
# Platform as tool 🚀
59+
- platform: 'elevenlabs'
60+
model: 'scribe_v1'
61+
name: 'transcribe_audio'
62+
description: 'Transcribes audio files to text using ElevenLabs.'
5263
multi_agent:
5364
support:
5465
orchestrator: 'orchestrator'
@@ -89,7 +100,16 @@ services:
89100

90101
Symfony\AI\Store\Document\Loader\RssFeedLoader: ~
91102
Symfony\AI\Store\Document\Transformer\TextTrimTransformer: ~
92-
103+
104+
# Example: Using a Platform as a Tool
105+
# This demonstrates how to wrap any platform as a tool that can be used by agents
106+
app.tool.elevenlabs_transcription:
107+
class: 'Symfony\AI\Agent\Toolbox\Tool\Platform'
108+
arguments:
109+
$platform: '@ai.platform.elevenlabs'
110+
$model: 'scribe_v1'
111+
$options: []
112+
93113
app.filter.week_of_symfony:
94114
class: 'Symfony\AI\Store\Document\Filter\TextContainsFilter'
95115
arguments:
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Platform as Tool
2+
3+
This example demonstrates how to use one AI platform as a tool within an agent powered by another platform.
4+
5+
## Concept
6+
7+
Just like agents can use other agents as tools, platforms can also be wrapped as tools. This enables powerful compositions where:
8+
9+
- An OpenAI-based agent can use ElevenLabs for speech-to-text
10+
- An Anthropic-based agent can use OpenAI's DALL-E for image generation
11+
- Any platform's specialized capabilities can be leveraged by agents using different platforms
12+
13+
## Example: Speech-to-Text with ElevenLabs
14+
15+
The `platform-as-tool.php` example shows an OpenAI agent that uses ElevenLabs' speech-to-text capabilities:
16+
17+
```php
18+
use Symfony\AI\Agent\Toolbox\Tool\Platform;
19+
20+
// Create ElevenLabs platform
21+
$elevenLabsPlatform = ElevenLabsPlatformFactory::create(
22+
apiKey: env('ELEVEN_LABS_API_KEY'),
23+
httpClient: http_client()
24+
);
25+
26+
// Wrap it as a tool
27+
$speechToText = new Platform($elevenLabsPlatform, 'scribe_v1');
28+
29+
// Add to agent's toolbox
30+
$toolbox = new Toolbox([$speechToText]);
31+
```
32+
33+
## AI Bundle Configuration
34+
35+
In a Symfony application using the AI Bundle, you can configure platforms as tools:
36+
37+
```yaml
38+
ai:
39+
platform:
40+
openai:
41+
api_key: '%env(OPENAI_API_KEY)%'
42+
elevenlabs:
43+
api_key: '%env(ELEVEN_LABS_API_KEY)%'
44+
45+
agent:
46+
my_agent:
47+
model: 'gpt-4o-mini'
48+
tools:
49+
# Use platform as tool via configuration
50+
- platform: 'elevenlabs'
51+
model: 'scribe_v1'
52+
name: 'transcribe_audio'
53+
description: 'Transcribes audio files to text'
54+
```
55+
56+
Or define it as a service:
57+
58+
```yaml
59+
services:
60+
app.tool.elevenlabs_transcription:
61+
class: 'Symfony\AI\Agent\Toolbox\Tool\Platform'
62+
arguments:
63+
$platform: '@ai.platform.elevenlabs'
64+
$model: 'scribe_v1'
65+
$options: []
66+
```
67+
68+
## Use Cases
69+
70+
1. **Speech-to-Text**: Use ElevenLabs or Whisper for transcription while using another platform for reasoning
71+
2. **Image Generation**: Use DALL-E or Stable Diffusion for images while using another platform for chat
72+
3. **Specialized Models**: Leverage platform-specific models (e.g., code generation, embeddings) from any agent
73+
4. **Multi-Modal Workflows**: Combine different platforms' strengths in a single agent workflow
74+
75+
## Benefits
76+
77+
- **Best Tool for the Job**: Choose the best platform for each specific task
78+
- **Flexibility**: Mix and match platforms based on cost, performance, or features
79+
- **Composability**: Build complex AI systems by combining multiple platforms
80+
- **Simplicity**: Use the same tool interface whether wrapping agents or platforms
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
use Symfony\AI\Agent\Agent;
13+
use Symfony\AI\Agent\Toolbox\AgentProcessor;
14+
use Symfony\AI\Agent\Toolbox\Tool\Platform;
15+
use Symfony\AI\Agent\Toolbox\Toolbox;
16+
use Symfony\AI\Platform\Bridge\ElevenLabs\PlatformFactory as ElevenLabsPlatformFactory;
17+
use Symfony\AI\Platform\Bridge\OpenAi\PlatformFactory;
18+
use Symfony\AI\Platform\Message\Content\Audio;
19+
use Symfony\AI\Platform\Message\Message;
20+
use Symfony\AI\Platform\Message\MessageBag;
21+
22+
require_once dirname(__DIR__).'/bootstrap.php';
23+
24+
// Create the main OpenAI platform
25+
$openAiPlatform = PlatformFactory::create(env('OPENAI_API_KEY'), http_client());
26+
27+
// Create ElevenLabs platform as a tool for speech-to-text
28+
$elevenLabsPlatform = ElevenLabsPlatformFactory::create(
29+
apiKey: env('ELEVEN_LABS_API_KEY'),
30+
httpClient: http_client()
31+
);
32+
33+
// Wrap ElevenLabs platform as a tool
34+
$speechToText = new Platform($elevenLabsPlatform, 'scribe_v1');
35+
36+
// Create toolbox with the platform tool
37+
$toolbox = new Toolbox([$speechToText], logger: logger());
38+
$processor = new AgentProcessor($toolbox);
39+
40+
// Create agent with OpenAI platform but with ElevenLabs tool available
41+
$agent = new Agent($openAiPlatform, 'gpt-4o-mini', [$processor], [$processor], logger: logger());
42+
43+
// The agent can now use ElevenLabs for speech-to-text while using OpenAI for reasoning
44+
$audioPath = dirname(__DIR__, 2).'/fixtures/audio.mp3';
45+
$messages = new MessageBag(
46+
Message::ofUser(sprintf(
47+
'I have an audio file. Please transcribe it and tell me what it says. Audio: %s',
48+
Audio::fromFile($audioPath)
49+
))
50+
);
51+
52+
$result = $agent->call($messages);
53+
54+
echo $result->getContent().\PHP_EOL;
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\AI\Agent\Toolbox\Tool;
13+
14+
use Symfony\AI\Platform\PlatformInterface;
15+
use Symfony\AI\Platform\Result\AudioResult;
16+
use Symfony\AI\Platform\Result\ImageResult;
17+
use Symfony\AI\Platform\Result\TextResult;
18+
19+
/**
20+
* Wraps a Platform instance as a tool, allowing agents to use specialized platforms for specific tasks.
21+
*
22+
* This enables scenarios where an agent using one platform (e.g., OpenAI) can leverage
23+
* another platform (e.g., ElevenLabs for speech-to-text) as a tool.
24+
*
25+
* @author Oskar Stark <[email protected]>
26+
*/
27+
final readonly class Platform
28+
{
29+
/**
30+
* @param array<string, mixed> $options
31+
*/
32+
public function __construct(
33+
private PlatformInterface $platform,
34+
private string $model,
35+
private array $options = [],
36+
) {
37+
}
38+
39+
/**
40+
* @param array<mixed>|string|object $input the input to pass to the platform
41+
*/
42+
public function __invoke(array|string|object $input): string
43+
{
44+
$result = $this->platform->invoke(
45+
$this->model,
46+
$input,
47+
$this->options,
48+
)->await();
49+
50+
return match (true) {
51+
$result instanceof TextResult => $result->getContent(),
52+
$result instanceof AudioResult => $result->getText() ?? base64_encode($result->getAudio()),
53+
$result instanceof ImageResult => $result->getText() ?? base64_encode($result->getImage()),
54+
default => throw new \LogicException(\sprintf('Unsupported result type "%s".', $result::class)),
55+
};
56+
}
57+
}

0 commit comments

Comments
 (0)