-
Notifications
You must be signed in to change notification settings - Fork 2k
Feat/fastapi modular server contribution #2967
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
FrigaZzz
wants to merge
24
commits into
google:main
Choose a base branch
from
FrigaZzz:feat/fastapi-modular-server-contribution
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,066
β0
Open
Changes from 9 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
1593ca0
feat: add FastAPI modular server example with SSE streaming support
FrigaZzz 67e6bf5
Merge branch 'google:main' into feat/fastapi-modular-server-contribution
FrigaZzz 7beab8c
refactor: replace __import__("time") with time module import
FrigaZzz 0a3a0a4
refactor: improve code readability and maintainability
FrigaZzz 235bca7
refactor: improve type hints and code organization
FrigaZzz c393957
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz d4cc10c
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz a7ea7f3
style: reorganize imports for better readability and consistency (./aβ¦
FrigaZzz c5b3545
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz ff07aae
docs: Update README to reflect changes in custom server file structurβ¦
FrigaZzz 3f88d35
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 9d1f87b
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz a3532e9
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 41284fe
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz ab74d1b
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 4a90e70
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 29e7a19
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz b3e1e78
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 2a0b124
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 5cba2aa
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz f187108
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 5abac2a
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz 76f71d6
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz cf0fa1a
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# Application Configuration | ||
DEBUG=true | ||
PORT=8881 | ||
HOST=localhost | ||
LOG_LEVEL=INFO | ||
LOGGING=true | ||
|
||
# ADK Configuration | ||
SERVE_WEB_INTERFACE=true | ||
RELOAD_AGENTS=true | ||
|
||
# Google Gemini Configuration | ||
GOOGLE_API_KEY=YOUR_GOOGLE_API_KEY_HERE | ||
|
||
# Model Configuration | ||
MODEL_PROVIDER=google |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Logs | ||
*.log | ||
|
||
# Environment variables | ||
.env | ||
|
||
# Python bytecode | ||
*.pyc | ||
__pycache__/ | ||
|
||
# Test artifacts | ||
.pytest_cache/ | ||
|
||
# Virtual environments | ||
.venv/ | ||
venv/ | ||
|
||
# IDE configuration | ||
.vscode/ | ||
.idea/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,314 @@ | ||
# π Google ADK FastAPI Modular Server | ||
|
||
A **production-ready template** for extending Google's Agent Development Kit (ADK) with custom FastAPI endpoints, optimized SSE streaming, and modular architecture patterns. | ||
|
||
## π― Purpose & Value | ||
|
||
The **FastAPI Modular Server** serves as a **template and reference implementation** for teams who want to: | ||
|
||
- **Extend ADK's built-in server** without modifying core behavior | ||
- **Accelerate production deployment** with battle-tested patterns | ||
- **Add custom business logic** through modular router systems | ||
- **Enable hot-reload capabilities** for faster development cycles | ||
|
||
## β¨ Key Features | ||
|
||
### π§ **Modular Router Architecture** | ||
- Clean separation of concerns with dedicated router classes | ||
- Easy to add new endpoints without touching core server code | ||
|
||
### β‘ **Optimized SSE Streaming** | ||
- **3 optimization levels** for different use cases: | ||
- `MINIMAL`: Essential content only (author + text) | ||
- `BALANCED`: Core data with invocation tracking | ||
- `FULL_COMPAT`: Complete ADK event compatibility | ||
- Reduced payload sizes for improved performance | ||
- Custom event filtering and mapping | ||
|
||
### π **Hot-Reload Development** | ||
- Automatic agent reloading on file changes | ||
- File system monitoring with `watchdog` | ||
- Development-friendly with production stability | ||
|
||
|
||
## π Project Structure | ||
|
||
``` | ||
fastapi_modular_server/ | ||
βββ .env.example # Environment variables template | ||
βββ README.md # Project documentation | ||
βββ __init__.py # Package initialization | ||
βββ app/ # Main application directory | ||
β βββ __init__.py # App package initialization | ||
β βββ agents/ # Agent definitions | ||
β β βββ greetings_agent/ # Greetings agent module | ||
β β βββ __init__.py # Agent package init | ||
β β βββ greetings_agent.py # Greetings agent implementation | ||
β βββ api/ # API layer | ||
β β βββ __init__.py # API package init | ||
β β βββ routers/ # API route definitions | ||
β β β βββ __init__.py # Routers package init | ||
β β β βββ agent_router.py # Agent-related API routes | ||
β β βββ custom_adk_server.py # FastAPI server configuration | ||
β βββ config/ # Configuration management | ||
β β βββ settings.py # Application settings | ||
β βββ core/ # Core application components | ||
β β βββ __init__.py # Core package init | ||
β β βββ dependencies.py # Dependency injection | ||
β β βββ logging.py # Logging configuration | ||
β β βββ mapping/ # Data mapping utilities | ||
β β βββ __init__.py # Mapping package init | ||
β β βββ sse_event_mapper.py # Server-Sent Events mapper | ||
β βββ models/ # Data models | ||
β βββ __init__.py # Models package init | ||
β βββ streaming_request.py # Streaming data models | ||
βββ main.py # Application entry point | ||
``` | ||
|
||
## π Quick Start | ||
|
||
### 1. **Configuration** | ||
```bash | ||
# Copy environment template | ||
cp .env.example .env | ||
|
||
# Edit .env with your settings | ||
vim .env | ||
|
||
# Set the API KEY | ||
|
||
``` | ||
|
||
### 2. **Run the Server** | ||
```bash | ||
# Development mode with hot-reload | ||
python main.py | ||
|
||
# Production mode | ||
uvicorn main:app --host 0.0.0.0 --port 8881 | ||
``` | ||
|
||
|
||
## π§ Customization Guide | ||
|
||
### **Adding New Routers** | ||
|
||
Create a new router following the established pattern: | ||
|
||
```python | ||
# app/api/routers/my_custom_router.py | ||
from fastapi import APIRouter, Depends | ||
from app.core.dependencies import ADKServices, get_adk_services | ||
|
||
class MyCustomRouter: | ||
def __init__(self, web_server_instance): | ||
self.web_server = web_server_instance | ||
self.router = APIRouter(prefix="/custom", tags=["Custom"]) | ||
self._setup_routes() | ||
|
||
def _setup_routes(self): | ||
@self.router.get("/endpoint") | ||
async def my_endpoint( | ||
): | ||
# Access any ADK service | ||
sessions = await self.web_server.session_service.list_sessions() | ||
return {"data": "custom response", "session_count": len(sessions)} | ||
|
||
def get_router(self) -> APIRouter: | ||
return self.router | ||
``` | ||
|
||
Register it in the custom server: | ||
|
||
```python | ||
# In app/api/server.py - CustomAdkWebServer class | ||
def _initialize_routers(self): | ||
try: | ||
self.agent_router = AgentRouter(self) | ||
self.my_custom_router = MyCustomRouter(self) # Add this | ||
logger.info("All routers initialized successfully.") | ||
except Exception as e: | ||
logger.error(f"Failed to initialize routers: {e}", exc_info=True) | ||
|
||
def _register_modular_routers(self, app: FastAPI): | ||
# ... existing code ... | ||
|
||
if self.my_custom_router: | ||
app.include_router(self.my_custom_router.get_router()) | ||
logger.info("Registered MyCustomRouter.") | ||
``` | ||
|
||
### **Overriding ADK Endpoints** | ||
|
||
#### **Method 1: Route Removal (Current Approach)** | ||
|
||
```python | ||
def _register_modular_routers(self, app: FastAPI): | ||
# Remove specific ADK routes | ||
routes_to_remove = [] | ||
for route in app.routes: | ||
if route.path in [ | ||
"/run_sse", | ||
"/apps/{app_name}/users/{user_id}/sessions" | ||
FrigaZzz marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
] and hasattr(route, 'methods') and 'POST' in route.methods: | ||
FrigaZzz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
routes_to_remove.append(route) | ||
|
||
# Remove the routes | ||
for route in routes_to_remove: | ||
app.routes.remove(route) | ||
``` | ||
|
||
#### **Method 2: Middleware Interception** | ||
|
||
For more complex overrides, use middleware: | ||
|
||
```python | ||
from fastapi import Request, Response | ||
from starlette.middleware.base import BaseHTTPMiddleware | ||
|
||
class RouteOverrideMiddleware(BaseHTTPMiddleware): | ||
async def dispatch(self, request: Request, call_next): | ||
# Intercept specific routes | ||
if request.url.path == "/run_sse" and request.method == "POST": | ||
# Handle with custom logic | ||
return await self.handle_custom_sse(request) | ||
|
||
return await call_next(request) | ||
``` | ||
|
||
### **Accessing ADK Services and Runners** | ||
|
||
#### **From Router Classes** | ||
```python | ||
class AgentRouter: | ||
def __init__(self, web_server_instance): | ||
self.web_server = web_server_instance | ||
|
||
async def my_endpoint(self, adk_services: ADKServices = Depends(get_adk_services)): | ||
# Access services | ||
agents = self.web_server.agent_loader.list_agents() | ||
session = await self.web_server.session_service.list_sessions() | ||
|
||
# Access runners through web server | ||
runner = await self.web_server.get_runner_async("your_app_name") | ||
|
||
# Access other web server properties | ||
runners_cache = self.web_server.runners_to_clean | ||
``` | ||
|
||
|
||
|
||
### **Optimizing SSE Streaming** | ||
|
||
#### **Custom Event Filtering** | ||
|
||
Extend the SSE mapper for more sophisticated filtering: | ||
|
||
```python | ||
# app/models/streaming_request.py | ||
class OptimizationLevel(str, Enum): | ||
"""Enumeration for the available SSE optimization levels.""" | ||
|
||
MINIMAL = "minimal" | ||
BALANCED = "balanced" | ||
FULL_COMPAT = "full_compat" | ||
ULTRA_MINIMAL = "ultra_minimal" | ||
|
||
# app/core/mapping/sse_mapper.py | ||
class AdvancedSSEEventMapper(SSEEventMapper): | ||
def map_event_to_sse_message(self, event: Event, optimization_level: OptimizationLevel) -> Optional[str]: | ||
# Custom filtering logic | ||
if self._should_skip_event(event): | ||
return None | ||
|
||
# Custom payload creation | ||
payload = self._create_custom_payload(event, optimization_level) | ||
|
||
# Custom serialization | ||
return self._serialize_payload(payload) | ||
|
||
def _should_skip_event(self, event: Event) -> bool: | ||
# Skip system events, debug events, empty events, etc. | ||
if event.author in ["system", "debug"]: | ||
return True | ||
if not event.content or not event.content.parts: | ||
return True | ||
return False | ||
|
||
def _create_custom_payload(self, event: Event, level: OptimizationLevel) -> Dict: | ||
if level == OptimizationLevel.ULTRA_MINIMAL: | ||
# Even more minimal than minimal | ||
return {"t": self._extract_text_only(event)} | ||
|
||
return super()._create_minimal_payload(event) | ||
``` | ||
|
||
#### **Streaming Performance Optimizations** | ||
|
||
1. **Batch Events**: Combine multiple streaming events (single chunk) into a single SSE message to reduce overhead. | ||
```python | ||
async def _generate_events_batched(self, req, sse_mapper, adk_services): | ||
batch = [] | ||
batch_size = 5 | ||
|
||
async for event in self._get_events(): | ||
batch.append(event) | ||
|
||
if len(batch) >= batch_size: | ||
# Process batch | ||
combined_payload = self._combine_events(batch) | ||
yield f"data: {json.dumps(combined_payload)}\n\n" | ||
batch.clear() | ||
``` | ||
|
||
2. **Compression**: | ||
```python | ||
import gzip | ||
import json | ||
|
||
def _create_compressed_sse(self, payload): | ||
json_str = json.dumps(payload, separators=(',', ':')) | ||
compressed = gzip.compress(json_str.encode()) | ||
# Use binary SSE or base64 encoding | ||
return f"data: {base64.b64encode(compressed).decode()}\n\n" | ||
FrigaZzz marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
``` | ||
FrigaZzz marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
||
3. **Event Deduplication**: | ||
```python | ||
class DedupSSEMapper(SSEEventMapper): | ||
def __init__(self): | ||
self._last_payloads = {} | ||
|
||
def map_event_to_sse_message(self, event, level): | ||
payload = super().map_event_to_sse_message(event, level) | ||
|
||
# Skip if identical to last payload for this session | ||
session_key = f"{event.session_id}_{event.author}" | ||
if self._last_payloads.get(session_key) == payload: | ||
return None | ||
|
||
self._last_payloads[session_key] = payload | ||
return payload | ||
``` | ||
|
||
|
||
## π€ Contributing | ||
|
||
This template is designed to be extended and customized for your specific needs. Key extension points: | ||
|
||
1. **Router Classes**: Add domain-specific endpoints | ||
2. **SSE Mappers**: Custom event processing and optimization | ||
3. **Middleware**: Cross-cutting concerns | ||
4. **Services**: Additional business logic services | ||
5. **Configuration**: Environment-specific settings | ||
|
||
## π Further Resources | ||
|
||
- **Google ADK Documentation**: https://google.github.io/adk-docs/ | ||
- **FastAPI Documentation**: https://fastapi.tiangolo.com/ | ||
- **Pydantic Settings**: https://docs.pydantic.dev/latest/concepts/pydantic_settings/ | ||
- **Server-Sent Events**: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events | ||
|
||
--- | ||
|
||
**Happy coding!** π This template provides a solid foundation for building production-ready ADK extensions with modern Python patterns and performance optimizations. |
Empty file.
1 change: 1 addition & 0 deletions
1
contributing/samples/fastapi_modular_server/app/agents/greetings_agent/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
from app.agents.greetings_agent.greetings_agent import root_agent |
9 changes: 9 additions & 0 deletions
9
contributing/samples/fastapi_modular_server/app/agents/greetings_agent/greetings_agent.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
from google.adk.agents import LlmAgent | ||
|
||
root_agent = LlmAgent( | ||
model="gemini-2.5-flash", | ||
name="greetings_agent", | ||
description="A friendly Google Gemini-powered agent", | ||
instruction="You are a helpful AI assistant powered by Google Gemini.", | ||
tools=[], | ||
) |
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.