Skip to content

Add support for ElevenLabs Conversational AI Agents #387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: next
Choose a base branch
from
Open

Add support for ElevenLabs Conversational AI Agents #387

wants to merge 3 commits into from

Conversation

@jvaldesgonzalez
Copy link

@jvaldesgonzalez jvaldesgonzalez commented May 11, 2025

ElevenLabs Conversational AI is a product from ElevenLabs for deploying customized, conversational and multimodal voice agents.

This PR adds:

  • A WebSocket-based plugin(similar to OpenAI realtime plugin) for real-time, bi-directional audio/text streaming
  • Connection lifecycle and event handling (connect, message, error, close)
  • Example script on how to use it (examples/src/elevenlabs_agent.ts)

@jvaldesgonzalez
feat(elevenlabs): add support for ElevenLabs Conversational AI (Realt…
61d97a1 
…ime)

Add plugin for ElevenLabs Conversational AI using the WebSockets API.

- A WebSocket connection was used(similar to OpenAi Realtime)
- Handled bi-directional streaming of audio and text messages
- Managed WebSocket events. (WIP)
- Add example on how to use the plugin

Refs:
https://elevenlabs.io/docs/conversational-ai/overview
https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket
@jvaldesgonzalez
feat(elevenlabs): add support for using private agents
3f3280e 
You can use your ElevenLabs agents with an API key from ElevenLabs.
If no API key is provided, then you can only access public agents.

TODO:
- support client tools
Maybe:
- support vad score
@jvaldesgonzalez
feat(elevenlabs): support client-side tools
4972ec1
@changeset-bot
Copy link

changeset-bot bot commented May 11, 2025

⚠️ No Changeset found

Latest commit: 4972ec1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@CLAassistant
Copy link

CLAassistant commented May 11, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jvaldesgonzalez @CLAassistant