-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Expand file tree
/
Copy pathllm.txt
More file actions
321 lines (242 loc) · 11.7 KB
/
llm.txt
File metadata and controls
321 lines (242 loc) · 11.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
---
# Starter Pack Guidance
Guide for AI agents working with **generated projects** (projects created by Agent Starter Pack).
> **Scope**: This document is for developing, testing, and deploying **generated agent projects**. For contributing to the **Agent Starter Pack repository itself** (the template generator), see [GEMINI.md](./GEMINI.md).
---
### Section 1: Project Overview
* **Project Name:** Agent Starter Pack
* **Purpose:** Accelerate development of production-ready GenAI Agents on Google Cloud.
* **Tagline:** Production-Ready Agents on Google Cloud, faster.
**The "Production Gap":**
While prototyping GenAI agents is quick, production deployment often takes 3-9 months.
**Key Challenges Addressed:**
* **Customization:** Business logic, data grounding, security/compliance.
* **Evaluation:** Metrics, quality assessment, test datasets.
* **Deployment:** Cloud infrastructure, CI/CD, UI integration.
* **Observability:** Performance tracking, user feedback.
**Solution: Agent Starter Pack**
Provides MLOps and infrastructure templates so developers focus on agent logic.
* **You Build:** Prompts, LLM interactions, business logic, agent orchestration.
* **We Provide:**
* Deployment infrastructure, CI/CD, testing
* Logging, monitoring
* Evaluation tools
* Data connections, UI playground
* Security best practices
Establishes production patterns from day one, saving setup time.
---
### Section 2: Creating & Enhancing Agent Projects
Start by creating a new agent project from a predefined template, or enhance an existing project with agent capabilities. Both processes support interactive and fully automated setup.
**Prerequisites:**
Before you begin, ensure you have `uv`/`uvx`, `gcloud` CLI, `terraform`, `git`, and `gh` CLI (for automated CI/CD setup) installed and authenticated.
**Installing the `agent-starter-pack` CLI:**
Choose one method to get the `agent-starter-pack` command:
1. **`uvx` (Recommended for Zero-Install/Automation):** Run directly without prior installation.
```bash
uvx agent-starter-pack create ...
```
2. **Virtual Environment (`pip` or `uv`):**
```bash
pip install agent-starter-pack
```
3. **Persistent CLI Install (`pipx` or `uv tool`):** Installs globally in an isolated environment.
---
### `agent-starter-pack create` Command
Generates a new agent project directory based on a chosen template and configuration.
**Usage:**
```bash
agent-starter-pack create PROJECT_NAME [OPTIONS]
```
**Arguments:**
* `PROJECT_NAME`: Name for your new project directory and base for GCP resource naming (max 26 chars, converted to lowercase).
**Template Selection:**
* `-a, --agent`: Agent template - built-in agents (e.g., `adk`, `agentic_rag`), remote templates (`adk@gemini-fullstack`, `github.com/user/repo@branch`), or local projects (`local@./path`).
**Deployment Options:**
* `-d, --deployment-target`: Target environment (`cloud_run`, `gke`, or `agent_engine`).
* `--cicd-runner`: CI/CD runner (`google_cloud_build` or `github_actions`).
* `--region`: GCP region (default: `us-central1`).
**Data & Storage:**
* `-ds, --datastore`: Datastore type for data ingestion (`vertex_ai_search`, `vertex_ai_vector_search`, `cloud_sql`). Automatically enables data ingestion when specified.
* `--session-type`: Session storage (`in_memory`, `cloud_sql`, `agent_engine`).
**Project Creation:**
* `-o, --output-dir`: Output directory (default: current directory).
* `--agent-directory, -dir`: Agent code directory name (default: `app`).
* `--in-folder`: Create files in current directory instead of new subdirectory.
**Automation:**
* `--auto-approve`: **Skip all interactive prompts (crucial for automation).**
* `--skip-checks`: Skip GCP/Vertex AI verification checks.
* `--debug`: Enable debug logging.
**Automated Creation Example:**
```bash
uvx agent-starter-pack create my-automated-agent \
-a adk \
-d cloud_run \
--region us-central1 \
--auto-approve
```
---
### `agent-starter-pack enhance` Command
Enhance your existing project with AI agent capabilities by adding agent-starter-pack features in-place. This command supports all the same options as `create` but templates directly into the current directory instead of creating a new project directory.
**Usage:**
```bash
agent-starter-pack enhance [TEMPLATE_PATH] [OPTIONS]
```
**Key Differences from `create`:**
* Templates into current directory (equivalent to `create --in-folder`)
* `TEMPLATE_PATH` defaults to current directory (`.`)
* Project name defaults to current directory name
* Additional `--base-template` option to override template inheritance
**Enhanced Project Example:**
```bash
# Enhance current directory with agent capabilities
uvx agent-starter-pack enhance . \
--base-template adk \
-d cloud_run \
--region us-central1 \
--auto-approve
```
**Project Structure:** Expects agent code in `app/` directory (configurable via `--agent-directory`).
---
### `agent-starter-pack extract` Command
Creates a minimal, shareable version of an agent by stripping deployment scaffolding while preserving core agent logic.
**Usage:**
```bash
agent-starter-pack extract OUTPUT_PATH [OPTIONS]
```
**Key Options:**
* `OUTPUT_PATH`: Required destination directory for extracted agent
* `--source, -s`: Source project directory (default: `.`)
* `--dry-run`: Preview what would be extracted without making changes
* `--force, -f`: Overwrite output directory if it exists
**What Gets Extracted:**
- Agent directory (e.g., `app/`) with `agent.py` and custom modules
- `pyproject.toml` (scaffolding dependencies removed)
- Minimal `Makefile` and `README.md`
**What Gets Removed:**
- `deployment/`, `.github/`, `.cloudbuild/`, `frontend/`, `data_ingestion/`, `notebooks/`, `tests/`
- Scaffolding files: `fast_api_app.py`, `agent_engine_app.py`, `app_utils/`
**Example:**
```bash
# Extract current project to shareable form
uvx agent-starter-pack extract ../my-agent-share
# Preview changes first
uvx agent-starter-pack extract ../my-agent-share --dry-run
```
**Workflow:** Use `extract` to share agents, then recipients use `enhance` to add their own deployment infrastructure.
---
### `agent-starter-pack upgrade` Command
Upgrades projects to newer ASP versions using intelligent 3-way merge.
**Usage:**
```bash
agent-starter-pack upgrade [PROJECT_PATH] [OPTIONS]
```
**Key Options:**
* `PROJECT_PATH`: Project to upgrade (default: `.`)
* `--dry-run`: Preview changes without applying them
* `--auto-approve, -y`: Auto-apply non-conflicting changes without prompts
**3-Way Merge Logic:**
- **You unchanged, ASP updated** → Auto-update the file
- **You modified, ASP unchanged** → Preserve your version
- **Both changed** → Prompt to resolve conflict
- **New ASP file** → Prompt to add
- **Removed in ASP** → Prompt to remove
**Files Never Modified:**
- Agent code (`app/agent.py`, custom modules)
- Configuration files (`.env`, secrets)
**Example:**
```bash
# Preview upgrade changes
uvx agent-starter-pack upgrade --dry-run
# Apply upgrade with auto-approve for non-conflicts
uvx agent-starter-pack upgrade -y
```
**Requirements:** Requires `uvx` and project must have `asp_version` in `[tool.agent-starter-pack]` section of `pyproject.toml`.
---
### Available Agent Templates
Templates for the `create` command (via `-a` or `--agent`):
| Agent Name | Description |
| :--------------------- | :------------------------------------------- |
| `adk` | Base ReAct agent (ADK) |
| `adk_gemini_fullstack` | Production-ready fullstack research agent |
| `agentic_rag` | RAG agent for document retrieval & Q&A |
| `langgraph` | Base ReAct agent (LangGraph) |
| `adk_live` | Real-time multimodal RAG agent |
---
### Including a Data Ingestion Pipeline (for RAG agents)
For RAG agents needing custom document search, enabling this option automates loading, chunking, embedding documents with Vertex AI, and storing them in a vector database.
**How to enable:**
```bash
uvx agent-starter-pack create my-rag-agent \
-a agentic_rag \
-d cloud_run \
-i \
-ds vertex_ai_search \
--auto-approve
```
**Post-creation:** Follow your new project's `data_ingestion/README.md` and `GEMINI.md` for setup and deployment workflow.
---
### Section 3: Key Features & Configuration Options
---
This section describes features available when creating projects. For post-creation workflow and commands, see `GEMINI.md` in your generated project.
### Session Management Options
For stateful agents, the starter pack supports persistent sessions (configure at creation time):
* **`in_memory`:** Ephemeral sessions (for testing/prototyping).
* **`cloud_sql`:** Durable PostgreSQL-backed sessions.
* **`agent_engine`:** Managed sessions (for Agent Engine deployment only).
Choose with `--session-type` flag during project creation.
### Deployment Targets
* **`cloud_run`:** Serverless container platform on Cloud Run. Supports all session types.
* **`gke`:** Managed Kubernetes (GKE Autopilot). Same container-based deployment as Cloud Run with more infrastructure control.
* **`agent_engine`:** Managed Vertex AI Agent Engine. Provides automatic session management.
### CI/CD Runners
* **`google_cloud_build`:** Cloud Build triggers. Requires Cloud Build GitHub connection.
* **`github_actions`:** GitHub Actions workflows with Workload Identity Federation.
* **`skip`:** No CI/CD (manual deployments via `make deploy`).
### Monitoring & Observability (Built-in)
* **Technology:** OpenTelemetry integration for Cloud Trace and Logging.
* **Custom Tracer:** Large payloads linked to GCS (overcoming service limits).
* **Infrastructure:** Log Router sinks data to BigQuery (provisioned by Terraform).
---
### Section 4: CLI Reference for CI/CD Setup
---
### `agent-starter-pack setup-cicd`
Automates the complete CI/CD infrastructure setup for GitHub-based deployments. Intelligently detects your CI/CD runner (Google Cloud Build or GitHub Actions) and configures everything automatically.
**Usage:**
```bash
uvx agent-starter-pack setup-cicd [OPTIONS]
```
**Prerequisites:**
- Run from the project root (directory with `pyproject.toml`)
- Required tools: `gh` CLI (authenticated), `gcloud` CLI (authenticated), `terraform`
- `Owner` role on GCP projects
- GitHub token with `repo` and `workflow` scopes
**Key Options:**
* `--staging-project`, `--prod-project`: GCP project IDs (will prompt if omitted).
* `--repository-name`, `--repository-owner`: GitHub repo details (will prompt if omitted).
* `--cicd-project`: CI/CD resources project (defaults to prod project).
* `--dev-project`: Development project ID (optional).
* `--region`: GCP region (default: `us-central1`).
* `--auto-approve`: Skip all interactive prompts.
* `--local-state`: Use local Terraform state instead of GCS backend.
* `--debug`: Enable debug logging.
**What it does:**
1. Creates/connects GitHub repository
2. Sets up Terraform infrastructure with remote state
3. Configures CI/CD runner connection (Cloud Build or GitHub Actions with WIF)
4. Provisions staging/prod environments
5. Sets up local Git repository with origin remote
**Automated Example:**
```bash
uvx agent-starter-pack setup-cicd \
--staging-project your-staging-project \
--prod-project your-prod-project \
--repository-name your-repo-name \
--repository-owner your-username \
--auto-approve
```
**After setup, push to trigger pipeline:**
```bash
git add . && git commit -m "Initial commit" && git push -u origin main
```
* Note: For coding agents - ask user for required project IDs and repo details before running with `--auto-approve`.
* Note: If user prefers different git provider, refer to `deployment/README.md` for manual deployment.