Agent Disconnects with CLIENT_MESSAGE_TIMEOUT Before Audio Starts Streaming

[Afzal-Hameeth]

I’m building a real-time voice agent using Deepgram’s Python SDK (AsyncDeepgramClient) with microphone input via sounddevice. The agent connects successfully, applies settings, and opens the WebSocket, but within 2–3 seconds I receive CLIENT_MESSAGE_TIMEOUT and the socket closes before the first audio chunk is sent. I’ve tried sending silence chunks every 0.5s, adding KeepAlive messages, and verified that my mic captures audio (chunks ~832 bytes, saved to WAV), but the agent still disconnects too quickly. Is there a recommended way to keep the connection alive until the user speaks? Can the timeout be configured or extended? Are there best practices for streaming silence vs. real audio during idle periods? Any guidance or examples would be greatly appreciated

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The deepgram product you are using (e.g Speech to Text, Agent API)
• A request ID that triggered your error or issue.

[Afzal-Hameeth]

I’m using Deepgram Agent API for real-time voice interaction.
The request ID that triggered the error is: d5433058-3e98-4319-b31c-a23dc5fac8e

[deepgram-community[bot]]

this can happen if you are trying to send data to the Agent API before the Welcome Message is received and the Settings Message is sent.

Voice Agent Message Flow

1. Connection → Welcome (receive)

Client connects to WebSocket
↓
Server sends: { "type": "Welcome", "request_id": "uuid" }

2. Settings → SettingsApplied (send/receive)

Client sends: { "type": "Settings", audio: {...}, agent: {...} }
↓
Server sends: { "type": "SettingsApplied" }

3. Audio Streaming + Text Injection (send)
After SettingsApplied, you can:

• Send binary audio data (PCM) continuously
• Send InjectUserMessage for text input:

  { "type": "InjectUserMessage", "content": "Hello" }

4. Receive Events (ongoing)
Server sends these as the conversation happens:

• UserStartedSpeaking User begins talking
• ConversationText (role: user) User's speech transcribed
• AgentThinking Agent is processing
• ConversationText (role: assistant) Agent's response text
• [binary audio] Agent's audio response
• AgentAudioDone Agent finished speaking
• InjectionRefused If InjectUserMessage sent at wrong time
• Error / Warning

This message was sent by John Vajda from Deepgram, via our community automation.