Skip to content

Commit 219e909

Browse files
Adk app (#214)
* Initiate ADK app project * Support all flows with default behavior * Add support for multimodal input * Add support for grounding in ADK app * Fix intermittent engine call failures * Add alternative implementation in single card * Add core logic of event handler * Complete first runnable version of sample in Cloud functions * Add support for context in common agent * Fix minor issues * Changed to disabled context by default, fixed a few minor issues * Refactor project for better readibility and modularity * Fixed context and readibility improvements * Improve text formatting * Fix line breaks in addon text result * Minor fixes * Merge google workspace utils in a single file * Improve UI rendering * Handle image URL issues * Improve code quality and readibility * Add error handling and stabilize impl * Fix management of missing event data * Remove unnecessary scopes * Relocate outside of chat samples and initiate README file
1 parent ef08a87 commit 219e909

File tree

15 files changed

+1092
-0
lines changed

15 files changed

+1092
-0
lines changed

python/chat/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules/
2+
**/credentials*.json
3+
**/client_secrets*.json
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# This file specifies files that are *not* uploaded to Google Cloud
2+
# using gcloud. It follows the same syntax as .gitignore, with the addition of
3+
# "#!include" directives (which insert the entries of the given .gitignore-style
4+
# file at that point).
5+
#
6+
# For more information, run:
7+
# $ gcloud topic gcloudignore
8+
#
9+
.gcloudignore
10+
README.md
11+
# If you would like to upload your .git directory, .gitignore file or files
12+
# from your .gitignore file, remove the corresponding line
13+
# below:
14+
.git
15+
.gitignore
16+
.venv
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
credentials*.json
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Travel Concierge ADK AI multi-agent published as Google Workspace add-on
2+
3+
This project integrates the Travel Concierge—an advanced, multi-agent AI from the Agent Development Kit (ADK) samples—into Google Workspace.
4+
5+
## Showcase
6+
7+
### Chat
8+
9+
[![Chat Showcase](./img/chat-preview.png)](https://drive.google.com/file/d/1KjmINbApQhn5mFNQjs1A5d-gVtvijSb7/view?usp=sharing)
10+
11+
### Gmail, Calendar, Drive
12+
13+
[![Gmail Showcase](./img/gmail-preview.png)](https://drive.google.com/file/d/1K4ycmpgz5fVF4U9_1N1bcZqe5UMvYnIF/view?usp=sharing)
14+
15+
## Overview
16+
17+
While the ADK provides default interfaces for developers (chat and text), this project focuses on creating a Graphical User Interface (GUI) dedicated to end-users. It demonstrates how to publish the Travel Concierge as a single Google Workspace add-on, making it accessible directly within applications like Google Chat, Gmail, Calendar, and Drive.
18+
19+
This solution serves as a practical guide and a reusable framework for developers who want to connect powerful, custom AI agents to Google Workspace, allowing users to leverage advanced AI capabilities seamlessly within their existing workflows.
20+
21+
## Agent
22+
23+
The [Travel Concierge ADK AI multi-agent being used](https://github.com/google/adk-samples/tree/main/python/agents/travel-concierge) is a conversational agent sample developed with the ADK. It's an advanced example, really close to a real-life AI agent. It uses tools like the Google Places API, Google Search Grounding, and a Model Context Protocol (MCP) server.
24+
25+
To expose this agent to end-users, a GUI with specific features is necessary as discussed [here](https://github.com/google/adk-samples/blob/main/python/agents/travel-concierge/README.md#gui). This sample shows how to build such a GUI by extending Google Workspace.
26+
27+
<img src="travel-concierge-arch.png" alt="Travel Concierge's Multi-Agent Architecture" width="800"/>
28+
29+
## Architecture
30+
31+
This solution relies on the following technologies: Google Chat API, People API, Vertex AI Agent Engine API, Python, and Google Cloud Functions.
32+
33+
## Features
34+
35+
* **User Sessions:** Sessions are managed in Vertex AI. Each user shares one session across all Workspace apps. Users can manually reset their session to start a new conversation.
36+
* **Rich Messaging:** Users send text messages and receive responses with rich text and graphical card elements (carousels, images, buttons, links).
37+
* **Error Handling:** Features configurable retries and informs users of unexpected failures with graceful interactions.
38+
* **Gmail Context:** Users can include the current email's subject and body in their message to the agent.
39+
* **Chat Streaming:** Agent interactions are displayed in real-time as they are processed.
40+
* **Chat Attachments:** Users can send messages with recorded audio or file attachments for extra context.
41+
* **User Context:** The agent can optionally include user profile data, such as birthdays (available in Gmail, Calendar, Drive).
42+
* **Switch to Chat:** Access the Chat app DM in a single click from other apps (Gmail, Calendar, Drive) to access Chat-only features.
43+
44+
## Prerequisites
45+
46+
* Google Cloud Project with billing enabled.
47+
48+
## Set up
49+
50+
1. Configure the Google Cloud project
51+
1. Enable the Google Workspace Marketplace SDK then the Vertex AI, Cloud Functions, Chat, Places, and People APIs
52+
1. Create a Service Account and grant the role `Vertex AI User`
53+
1. Create a private key with type JSON. This will download the JSON file.
54+
1. Setup, install, and deploy the LLM Auditor ADK AI Agent sample
55+
1. Use Vertex AI
56+
1. Use the same Google Cloud project
57+
1. Use the location `us-central1`
58+
1. Use the Vertex AI Agent Engine
59+
1. Publish HTTP endpoint to Google Cloud Functions
60+
1. Add Chat app credentials file `credentials-chat.json` to the project root directory
61+
1. Create and deploy a new Cloud Function with the project source code
62+
1. Configure the environment variables and redeploy
63+
1. Publish Google Workspace add-on
64+
1. Configure Google Chat API
65+
1. Complete manifest and configure a new HTTP deployment with it
66+
67+
## Customization
68+
69+
The core logic supports any ADK AI agent hosted in Vertex AI Agent Engine. Key customization points are:
70+
71+
* `main.py`: Defines the main UI layouts and user interaction logic (Google Workspace event handlers). Example: Add support for Calendar event or Drive document context.
72+
73+
* `vertex_ai.py`: Manages the streamed agent events, sessions, and error handling. Example: Enable multi-sessions for separate user conversations.
74+
75+
* `agent_handler.py`: Implements abstracted functions for orchestrating operations. Example: Synchronize message history across all host applications.
76+
77+
* `google_workspace.py`: Handles API interactions with other systems to gather context or take actions. Example: Add functions to retrieve details of a Calendar event.
78+
79+
* `travel_agent_ui_render.py`: Controls how agent responses are displayed to end-users. Example: Design a new card to show a person's profile and avatar.
Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# Copyright 2025 Google LLC. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the 'License');
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an 'AS IS' BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import markdown
16+
import re
17+
from google_workspace import create_message, update_message, download_chat_attachment
18+
from vertex_ai import IAiAgentHandler, IAiAgentUiRender
19+
from typing import Any
20+
21+
# Error message to display when something goes wrong
22+
ERROR_MESSAGE = "❌ Something went wrong"
23+
24+
def snake_to_user_readable(snake_case_string="") -> str:
25+
"""Converts a snake_case_string to a user-readable Title Case string."""
26+
return snake_case_string.replace('_', ' ').title()
27+
28+
class AgentCommon(IAiAgentHandler):
29+
"""AI Agent handler implementation for non-Chat host apps."""
30+
31+
# ----- IAiAgentHandler interface implementation
32+
33+
def __init__(self, ui_render: IAiAgentUiRender):
34+
super().__init__(ui_render)
35+
self.turn_card_sections = []
36+
37+
def extract_content_from_input(self, input) -> dict:
38+
# For non-Chat host apps, the input is a simple text string
39+
return { "role": "user", "parts": [{ "text": input }] }
40+
41+
def final_answer(self, author: str, text: str, success: bool, failure: bool):
42+
"""Adds the final answer section to the turn card sections."""
43+
self.add_section(section=self.build_section(author=author, text=text, widgets=[], success=success, failure=failure))
44+
45+
def function_calling_initiation(self, author: str, name: str) -> Any:
46+
"""Adds a function calling initiation section to the turn card sections."""
47+
return self.add_section(section=self.build_section(
48+
author=name,
49+
text=f"Working on **{snake_to_user_readable(author)}**'s request...",
50+
widgets=[],
51+
success=False,
52+
failure=False
53+
))
54+
55+
def function_calling_completion(self, author: str, name: str, response, output_id):
56+
"""Updates the function calling section with the completion response."""
57+
self.update_section(
58+
index=output_id,
59+
section=self.build_section(
60+
author=name,
61+
text="",
62+
widgets=self.ui_render.get_agent_response_widgets(name=name, response=response),
63+
success=True,
64+
failure=False
65+
)
66+
)
67+
68+
def function_calling_failure(self, name: str, output_id: str):
69+
"""Updates the function calling section with a failure status."""
70+
self.update_section(
71+
index=output_id,
72+
section=self.build_section(
73+
author=name,
74+
text=ERROR_MESSAGE,
75+
widgets=[],
76+
success=False,
77+
failure=True
78+
)
79+
)
80+
81+
# ------ Utility functions
82+
83+
def add_section(self, section) -> int:
84+
"""Adds a new section to the turn card sections and returns its index."""
85+
print(f"Adding section in stack...")
86+
self.turn_card_sections.append(section)
87+
return len(self.turn_card_sections) - 1
88+
89+
def update_section(self, index: int, section):
90+
"""Updates an existing section in the turn card sections."""
91+
print(f"Updating section in stack...")
92+
self.turn_card_sections[index] = section
93+
94+
def get_answer_sections(self) -> list:
95+
"""Returns the turn card sections in reverse order for display."""
96+
return self.turn_card_sections[::-1]
97+
98+
def build_section(self, author, text, widgets, success: bool, failure: bool) -> dict:
99+
"""Builds a card section for the given author, text, and widgets."""
100+
displayedText = f"{self.ui_render.get_author_emoji(author)} **{snake_to_user_readable(author)}**{f' ✅' if success else ''}{f'\n\n{text}' if text else ''}"
101+
textWidgets = [{ "text_paragraph": { "text": markdown.markdown(self.substitute_listings_from_markdown(displayedText)).replace('\n', '\n\n') }}]
102+
return { "widgets": textWidgets + widgets + ([] if success or failure else self.ui_render.create_status_accessory_widgets(text="In progress...", material_icon_name="progress_activity")) }
103+
104+
def substitute_listings_from_markdown(self, markdown: str) -> str:
105+
"""Removes markdown listings (bulleted and numbered) from the given markdown text."""
106+
pattern = re.compile(r'^\s*([*-+]|\d+\.)\s+', re.MULTILINE)
107+
return pattern.sub('-> ', markdown)
108+
109+
class AgentChat(IAiAgentHandler):
110+
"""AI Agent handler implementation for Chat apps."""
111+
112+
# ----- IAiAgentHandler interface implementation
113+
114+
def extract_content_from_input(self, input) -> dict:
115+
# For Chat host apps, the input can contain text and attachments
116+
parts = [{ "text": input.get("text") }]
117+
if "attachment" in input:
118+
for attachment in input.get("attachment"):
119+
attachmentBase64Data = download_chat_attachment(attachment.get("attachmentDataRef").get("resourceName"))
120+
inline_data_part = { "inline_data": {
121+
"mime_type": attachment.get("contentType"),
122+
"data": attachmentBase64Data
123+
}}
124+
parts.append(inline_data_part)
125+
return { "role": "user", "parts": parts }
126+
127+
def final_answer(self, author: str, text: str, success: bool, failure: bool):
128+
"""Sends the final answer as a Chat message."""
129+
create_message(message=self.build_message(author=author, text=text, cards_v2=[], success=success, failure=failure))
130+
131+
def function_calling_initiation(self, author: str, name: str) -> Any:
132+
"""Sends a function calling initiation message in Chat and returns the message name as output ID."""
133+
return create_message(message=self.build_message(
134+
author=name,
135+
text=f"Working on **{snake_to_user_readable(author)}**'s request...",
136+
cards_v2=[],
137+
success=False,
138+
failure=False
139+
))
140+
141+
def function_calling_completion(self, author: str, name: str, response, output_id):
142+
"""Updates the function calling message in Chat with the completion response."""
143+
widgets = self.ui_render.get_agent_response_widgets(name=name, response=response)
144+
update_message(
145+
name=output_id,
146+
message=self.build_message(
147+
author=name,
148+
text="",
149+
cards_v2=self.wrap_widgets_in_cards_v2(widgets) if len(widgets) > 0 else [],
150+
success=True,
151+
failure=False
152+
)
153+
)
154+
155+
def function_calling_failure(self, name: str, output_id: str):
156+
"""Updates the function calling section with a failure status."""
157+
update_message(
158+
name=output_id,
159+
message=self.build_message(
160+
author=name,
161+
text=ERROR_MESSAGE,
162+
cards_v2=[],
163+
success=False,
164+
failure=True
165+
)
166+
)
167+
168+
# ------ Utility functions
169+
170+
def build_message(self, author, text, cards_v2, success: bool, failure: bool) -> dict:
171+
"""Builds a Chat message for the given author, text, and cards_v2."""
172+
if text:
173+
cards_v2.insert(0, { "card": { "sections": [{ "widgets": [{ "text_paragraph": { "text": text.replace('\n', '\n\n'), "text_syntax": "MARKDOWN" }}]}]}})
174+
return {
175+
"text": f"{self.ui_render.get_author_emoji(author)} *{snake_to_user_readable(author)}*{f' ✅' if success else ''}",
176+
"cards_v2": cards_v2,
177+
"accessory_widgets": [] if success or failure else self.ui_render.create_status_accessory_widgets(text="In progress...", material_icon_name="progress_activity")
178+
}
179+
180+
def wrap_widgets_in_cards_v2(self, widgets=[]) -> list:
181+
"""Wraps the given widgets in Chat cards_v2 structure."""
182+
return [{ "card": { "sections": [{ "widgets": widgets }]}}]
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
{
2+
"oauthScopes": [
3+
"https://www.googleapis.com/auth/userinfo.email",
4+
"https://www.googleapis.com/auth/user.birthday.read",
5+
6+
"https://www.googleapis.com/auth/gmail.addons.execute",
7+
"https://www.googleapis.com/auth/gmail.addons.current.message.readonly",
8+
9+
"https://www.googleapis.com/auth/calendar.addons.execute",
10+
11+
"https://www.googleapis.com/auth/drive.addons.metadata.readonly"
12+
],
13+
"addOns": {
14+
"common": {
15+
"name": "Travel ADK AI Agent",
16+
"logoUrl": "https://goo.gle/3SfMkjb",
17+
"homepageTrigger": {
18+
"runFunction": $BASE_URL
19+
}
20+
},
21+
"gmail": { "contextualTriggers": [{
22+
"unconditional": {},
23+
"onTriggerFunction": $BASE_URL
24+
}]},
25+
"calendar": { "eventOpenTrigger": {
26+
"runFunction": $BASE_URL
27+
}},
28+
"drive": { "onItemsSelectedTrigger": {
29+
"runFunction": $BASE_URL
30+
}},
31+
"httpOptions": {
32+
"granularOauthPermissionSupport": "OPT_IN"
33+
}
34+
}
35+
}

python/travel-adk-ai-agent/env.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Copyright 2025 Google LLC. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the 'License');
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an 'AS IS' BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Service that handles runtime environment."""
16+
17+
import os
18+
19+
# Environment variables
20+
PROJECT_NUMBER = os.environ.get('PROJECT_NUMBER', 'your-google-cloud-project-number')
21+
LOCATION = os.environ.get('LOCATION', 'your-location')
22+
ENGINE_ID = os.environ.get('ENGINE_ID', 'your-engine-id')
23+
MAX_AI_AGENT_RETRIES = os.environ.get('MAX_AI_AGENT_RETRIES', 10)
24+
25+
BASE_URL = os.environ.get('BASE_URL', 'your-google-cloud-function-url')
26+
27+
RESET_SESSION_COMMAND_ID = os.environ.get('RESET_SESSION_COMMAND_ID', 1)
28+
29+
NA_IMAGE_URL = os.environ.get('NA_IMAGE_URL', 'https://upload.wikimedia.org/wikipedia/commons/d/d1/Image_not_available.png?20210219185637')
30+
31+
DEBUG = os.environ.get('DEBUG', 0)
32+
33+
def is_in_debug_mode() -> bool:
34+
"""Returns whether the application is running in debug mode."""
35+
return DEBUG == 1

0 commit comments

Comments
 (0)