Skip to content

Commit 88a70d7

Browse files
authored
ADK Live API - Agent Engine Bidi in Agent Engine (#371)
* **Refactors Real-Time Agent:** Renamed the `live_api` agent to `adk_live` and migrated its core logic to use the Agent Development Kit. * **Enables Bidirectional Streaming:** Integrates with Agent Engine and the Gemini Live API for real-time communication. * **Updates Frontend & CI/CD:** Aligned the React frontend, build configurations, and CI/CD pipelines to support the new ADK-based agent in Agent Engine, adding new tests and deployment targets.
1 parent 00f9719 commit 88a70d7

File tree

116 files changed

+23432
-6811
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

116 files changed

+23432
-6811
lines changed

.cloudbuild/ci/test_templated_agents.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,4 @@ logsBucket: gs://${PROJECT_ID}-logs-data/build-logs
3535
options:
3636
defaultLogsBucketBehavior: REGIONAL_USER_OWNED_BUCKET
3737
env:
38-
- "_TEST_AGENT_COMBINATION=live_api,cloud_run"
38+
- "_TEST_AGENT_COMBINATION=${_TEST_AGENT_COMBINATION}"

.cloudbuild/terraform/backend.tf

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
terraform {
2+
backend "gcs" {
3+
bucket = "agent-starter-pack-cicd-terraform-state"
4+
prefix = "cloudbuild"
5+
}
6+
}

.cloudbuild/terraform/build_triggers.tf

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,12 @@ locals {
9191
value = "agentic_rag,cloud_run,--include-data-ingestion,--datastore,vertex_ai_vector_search"
9292
},
9393
{
94-
name = "live_api-cloud_run"
95-
value = "live_api,cloud_run"
94+
name = "adk_live-agent_engine"
95+
value = "adk_live,agent_engine"
96+
},
97+
{
98+
name = "adk_live-cloud_run"
99+
value = "adk_live,cloud_run"
96100
},
97101
{
98102
name = "adk_base-cloud_run-alloydb"
@@ -132,6 +136,10 @@ agent_testing_included_files = { for combo in local.agent_testing_combinations :
132136
name = "agentic_rag-agent_engine-vertex_ai_search-github"
133137
value = "agentic_rag,agent_engine,--include-data-ingestion,--datastore,vertex_ai_search,--cicd-runner,github_actions"
134138
},
139+
{
140+
name = "adk_live-agent_engine-github"
141+
value = "adk_live,agent_engine,--cicd-runner,github_actions"
142+
},
135143
{
136144
name = "adk_base-agent_engine"
137145
value = "adk_base,agent_engine,-dir,tag"
@@ -153,8 +161,12 @@ agent_testing_included_files = { for combo in local.agent_testing_combinations :
153161
value = "agentic_rag,cloud_run,--include-data-ingestion,--datastore,vertex_ai_vector_search"
154162
},
155163
{
156-
name = "live_api-cloud_run"
157-
value = "live_api,cloud_run"
164+
name = "adk_live-agent_engine"
165+
value = "adk_live,agent_engine"
166+
},
167+
{
168+
name = "adk_live-cloud_run"
169+
value = "adk_live,cloud_run"
158170
},
159171
{
160172
name = "adk_base-cloud_run-alloydb"
@@ -180,8 +192,8 @@ agent_testing_included_files = { for combo in local.agent_testing_combinations :
180192
"agents/agentic_rag/**",
181193
"src/data_ingestion/**",
182194
"pyproject.toml",
183-
] : substr(combo.name, 0, 8) == "live_api" ? [
184-
"agents/live_api/**",
195+
] : substr(combo.name, 0, 8) == "adk_live" ? [
196+
"agents/adk_live/**",
185197
"pyproject.toml",
186198
] : [
187199
# Only include files for the specific agent being tested
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
cicd_runner_project_id = "agent-starter-pack-cicd"
2+
3+
region = "europe-west4"
4+
5+
host_connection_name = "git-connection"
6+
7+
repository_name = "GoogleCloudPlatform-agent-starter-pack"
8+
9+
e2e_test_project_mapping = {
10+
dev = "agent-starter-pack-e2e-dev"
11+
staging = "agent-starter-pack-e2e-st"
12+
prod = "agent-starter-pack-e2e-pr"
13+
}

.gitignore

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -195,8 +195,6 @@ target
195195
set_projects.sh
196196
delete_genai_repos.sh
197197
cleanup_e2e_projects.sh
198-
.cloudbuild/terraform/backend.tf
199-
.cloudbuild/terraform/vars/env.tfvars
200198
docs/.vitepress/cache
201199
tests/cicd/delete_gcp_resources.sh
202200
tests/cicd/delete_agent_engines.py

GEMINI.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ Changes often require coordinated updates:
139139

140140
### Conditional Logic Patterns
141141
```jinja
142-
{%- if cookiecutter.agent_name == "live_api" %}
142+
{%- if cookiecutter.agent_name == "adk_live" %}
143143
# Agent-specific logic
144144
{%- elif cookiecutter.deployment_target == "cloud_run" %}
145145
# Deployment-specific logic
@@ -148,15 +148,15 @@ Changes often require coordinated updates:
148148

149149
### Testing Strategy
150150
Test changes across multiple dimensions:
151-
- Agent types (live_api, adk_base, etc.)
151+
- Agent types (adk_live, adk_base, etc.)
152152
- Deployment targets (cloud_run, agent_engine)
153153
- Feature combinations (data_ingestion, frontend_type)
154154
- Example command for testing the starter pack creation - from the root of the repo run: `uv run agent-starter-pack create myagent-$(date +%s) --output-dir target`
155155

156156
### Common Pitfalls
157157
- **Hardcoded URLs**: Use relative paths for frontend connections
158158
- **Missing Conditionals**: Wrap agent-specific code in proper `{% if %}` blocks
159-
- **Dependency Conflicts**: Some agents lack certain extras (e.g., live_api + lint)
159+
- **Dependency Conflicts**: Some agents lack certain extras (e.g., adk_live + lint)
160160

161161
## File Modification Checklist
162162

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ See [Installation Guide](https://googlecloudplatform.github.io/agent-starter-pac
7272
| `agentic_rag` | A RAG agent for document retrieval and Q&A. Supporting [Vertex AI Search](https://cloud.google.com/generative-ai-app-builder/docs/enterprise-search-introduction) and [Vector Search](https://cloud.google.com/vertex-ai/docs/vector-search/overview). |
7373
| `langgraph_base_react` | An agent implementing a base ReAct agent using LangGraph |
7474
| `crewai_coding_crew` | A multi-agent system implemented with CrewAI created to support coding activities |
75-
| `live_api` | A real-time multimodal RAG agent powered by Gemini, supporting audio/video/text chat with vector DB-backed responses |
75+
| `adk_live` | A real-time multimodal RAG agent powered by Gemini, supporting audio/video/text chat with vector DB-backed responses |
7676

7777
**More agents are on the way!** We are continuously expanding our [agent library](https://googlecloudplatform.github.io/agent-starter-pack/agents/overview). Have a specific agent type in mind? [Raise an issue as a feature request!](https://github.com/GoogleCloudPlatform/agent-starter-pack/issues/new?labels=enhancement)
7878

agents/adk_base/.template/templateconfig.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ settings:
1818
requires_data_ingestion: false
1919
requires_session: true
2020
deployment_targets: ["agent_engine", "cloud_run"]
21-
extra_dependencies: ["google-adk~=1.14.0"]
21+
extra_dependencies: ["google-adk>=1.15.0,<2.0.0"]
2222
tags: ["adk"]
2323
frontend_type: "None"
2424

agents/live_api/.template/templateconfig.yaml renamed to agents/adk_live/.template/templateconfig.yaml

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,11 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15-
description: "A real-time multimodal RAG agent powered by Gemini Live API, supporting audio/video/text chat with vector DB-backed responses"
15+
description: "Real-time multimodal agent with ADK and Gemini Live API for low-latency voice and video interaction."
1616
settings:
1717
requires_data_ingestion: false
18-
frontend_type: "live_api_react"
19-
deployment_targets: ["cloud_run"]
20-
extra_dependencies: [
21-
"backoff~=2.2.1",
22-
"google-genai>=1.16.0",
23-
]
18+
frontend_type: "adk_live_react"
19+
deployment_targets: ["agent_engine", "cloud_run"]
20+
extra_dependencies: ["google-adk>=1.15.0,<2.0.0", "click>=8.0.0,<9.0.0", "uvicorn>=0.18.0,<1.0.0", "fastapi>=0.75.0,<1.0.0", "backoff>=2.0.0,<3.0.0"]
21+
tags: ["adk", "adk_live"]
2422
example_question: "What's the weather in San Francisco?"

agents/adk_live/README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# ADK Live Agent
2+
3+
Real-time conversational agent built with Google ADK and Gemini's live audio model. Supports audio, video, and text interactions with native tool calling.
4+
5+
![adk_live_diagram](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/adk_live_diagram.png)
6+
7+
**Key components:**
8+
9+
- **Python Backend** (in `app/` folder): ADK-powered agent using Gemini's live audio model with native tool calling and deployment support for Cloud Run and Agent Engine
10+
11+
- **React Frontend** (in `frontend/` folder): Web console for interacting with the live agent via audio, video, and text
12+
13+
![live api demo](https://storage.googleapis.com/github-repo/generative-ai/sample-apps/e2e-gen-ai-app-starter-pack/adk_live_pattern_demo.gif)
14+
15+
Once running, click the play button to connect and interact with the agent. Try asking "What's the weather like in San Francisco?" to see tool calling in action.
16+
17+
## Additional Resources for Multimodal Live API
18+
19+
Explore these resources to learn more about the Multimodal Live API and see examples of its usage:
20+
21+
- [Project Pastra](https://github.com/heiko-hotz/gemini-multimodal-live-dev-guide/tree/main): a comprehensive developer guide for the Gemini Multimodal Live API.
22+
- [Google Cloud Multimodal Live API demos and samples](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini/multimodal-live-api): Collection of code samples and demo applications leveraging multimodal live API in Vertex AI
23+
- [Gemini 2 Cookbook](https://github.com/google-gemini/cookbook/tree/main/gemini-2): Practical examples and tutorials for working with Gemini 2
24+
- [Multimodal Live API Web Console](https://github.com/google-gemini/multimodal-live-api-web-console): Interactive React-based web interface for testing and experimenting with Gemini Multimodal Live API.
25+
26+
## Current Status & Future Work
27+
28+
This pattern is under active development. Key areas planned for future enhancement include:
29+
30+
* **Observability:** Implementing comprehensive monitoring and tracing features.
31+
* **Load Testing:** Integrating load testing capabilities.

0 commit comments

Comments
 (0)