AI Gateway: Use Google AI SDK for using Vertex AI's Live API for real-time bidirectional audio streaming

[tomek-labuk]

How-to for using Vertex AI's Live API for real-time bidirectional audio streaming through Kong AI Gateway. This requires async support with the aio.live.connect method.

Python script:

import asyncio
from google import genai

async def live_audio_session():
    client = genai.Client(
        vertexai=True,
        project="your-project-id",
        location="us-central1",
        http_options={
            "api_version": "v1alpha",
            "base_url": "http://localhost:8000/gemini"
        }
    )
    
    config = {
        "model": "gemini-2.0-flash-exp",
        "generation_config": {"response_modalities": ["AUDIO"]}
    }
    
    async with client.aio.live.connect(config=config) as session:
        with open("audio_input.wav", "rb") as f:
            await session.send({
                "data": f.read(),
                "mime_type": "audio/wav"
            })
        
        async for response in session.receive():
            if response.data and response.mime_type == "audio/pcm":
                with open("output_audio.wav", "ab") as f:
                    f.write(response.data)

asyncio.run(live_audio_session())