FastAPI Chat Application with AI SDK 5 Data Stream Protocol

This FastAPI application integrates with OpenAI's GPT-4 and implements the AI SDK 5 Data Stream Protocol for streaming responses.

Reference: AI SDK Stream Protocols Documentation

Features

• ✅ Full AI SDK 5 Data Stream Protocol implementation
• ✅ Server-Sent Events (SSE) streaming
• ✅ Tool/function calling support
• ✅ Multi-part message support (text, images, files)
• ✅ Error handling with proper error events
• ✅ Compatible with useChat from @ai-sdk/react

Setup

1. Create and activate a virtual environment:

python3 -m venv venv
source venv/bin/activate  # On macOS/Linux

2. Install dependencies:

pip install -r requirements.txt

3. Set up your OpenAI API key:

echo "OPENAI_API_KEY=your_actual_openai_api_key_here" > .env.local

Running the Application

uvicorn main:app --reload

The server will start on http://localhost:8000

API Endpoint

POST /api/chat

Streaming chat endpoint compatible with AI SDK 5.

Required Header:

x-vercel-ai-ui-message-stream: v1

Request Body:

{
  "id": "conversation-id",
  "messages": [
    {
      "id": "message-id",
      "role": "user",
      "parts": [
        { "type": "text", "text": "Hello!" }
      ]
    }
  ],
  "trigger": "submit-message"
}

AI SDK 5 Data Stream Protocol

This implementation follows the exact protocol from AI SDK documentation.

Supported Event Types

Message Start Part

Indicates the beginning of a new message.

data: {"type":"start","messageId":"msg_abc123"}

Step Parts

data: {"type":"start-step"}
data: {"type":"finish-step"}

Text Parts

Text content is streamed using start/delta/end pattern.

data: {"type":"text-start","id":"msg_abc123"}
data: {"type":"text-delta","id":"msg_abc123","delta":"Hello"}
data: {"type":"text-delta","id":"msg_abc123","delta":" World"}
data: {"type":"text-end","id":"msg_abc123"}

Reasoning Parts

For models that support reasoning (like o1).

data: {"type":"reasoning-start","id":"reasoning_123"}
data: {"type":"reasoning-delta","id":"reasoning_123","delta":"Let me think..."}
data: {"type":"reasoning-end","id":"reasoning_123"}

Tool Parts

For function/tool calling.

data: {"type":"tool-input-start","toolCallId":"call_xxx","toolName":"get_current_weather"}
data: {"type":"tool-input-delta","toolCallId":"call_xxx","inputTextDelta":"{\"location\":\"SF\"}"}
data: {"type":"tool-input-available","toolCallId":"call_xxx","toolName":"get_current_weather","input":{"location":"SF"}}
data: {"type":"tool-output-available","toolCallId":"call_xxx","output":{"temperature":72,"unit":"fahrenheit"}}

Source Parts

For referencing external content.

data: {"type":"source-url","sourceId":"src_1","url":"https://example.com"}
data: {"type":"source-document","sourceId":"src_2","mediaType":"pdf","title":"Document"}

File Part

For file references.

data: {"type":"file","url":"https://example.com/image.png","mediaType":"image/png"}

Data Parts

Custom data with type-specific handling.

data: {"type":"data-weather","data":{"location":"SF","temperature":72}}

Error Part

For error messages.

data: {"type":"error","errorText":"Something went wrong"}

Finish Message Part

Indicates completion.

data: {"type":"finish"}

Stream Termination

data: [DONE]

Complete Response Example

Simple Text Response:

data: {"type":"start","messageId":"msg_abc123"}
data: {"type":"start-step"}
data: {"type":"text-start","id":"msg_def456"}
data: {"type":"text-delta","id":"msg_def456","delta":"Hello"}
data: {"type":"text-delta","id":"msg_def456","delta":"!"}
data: {"type":"text-delta","id":"msg_def456","delta":" How"}
data: {"type":"text-delta","id":"msg_def456","delta":" can"}
data: {"type":"text-delta","id":"msg_def456","delta":" I"}
data: {"type":"text-delta","id":"msg_def456","delta":" help"}
data: {"type":"text-delta","id":"msg_def456","delta":"?"}
data: {"type":"text-end","id":"msg_def456"}
data: {"type":"finish-step"}
data: {"type":"finish"}
data: [DONE]

Tool Call Response (e.g., "get weather of Dhaka"):

data: {"type":"start","messageId":"msg_abc123"}
data: {"type":"start-step"}
data: {"type":"tool-input-start","toolCallId":"call_xyz","toolName":"get_current_weather"}
data: {"type":"tool-input-delta","toolCallId":"call_xyz","inputTextDelta":"{\"location\":"}
data: {"type":"tool-input-delta","toolCallId":"call_xyz","inputTextDelta":"\"Dhaka\""}
data: {"type":"tool-input-delta","toolCallId":"call_xyz","inputTextDelta":",\"unit\":\"celsius\"}"}
data: {"type":"tool-input-available","toolCallId":"call_xyz","toolName":"get_current_weather","input":{"location":"Dhaka","unit":"celsius"}}
data: {"type":"tool-output-available","toolCallId":"call_xyz","output":{"temperature":28,"unit":"celsius","location":"Dhaka"}}
data: {"type":"finish-step"}
data: {"type":"finish"}
data: [DONE]

Frontend Integration

Use with AI SDK's useChat hook:

import { useChat } from '@ai-sdk/react';

export default function Chat() {
  const { messages, sendMessage } = useChat({
    api: 'http://localhost:8000/api/chat',
  });

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          {m.role}: {m.parts.map(p => p.type === 'text' ? p.text : null)}
        </div>
      ))}
    </div>
  );
}

Available Tools

• get_current_weather: Returns weather data for a given location

Project Structure

fastapp/
├── main.py              # FastAPI app with AI SDK 5 protocol
├── utils/
│   ├── __init__.py
│   ├── types.py         # Pydantic models for message parts
│   ├── prompt.py        # Message conversion utilities
│   └── tools.py         # Tool implementations
├── requirements.txt
└── README.md

CORS Configuration

By default, CORS allows all origins. For production:

app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://your-frontend.com"],
    ...
)