diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fcac2a502..1b972d570 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,80 +1,232 @@ -# How to Contribute +# Contributing to Agent Starter Pack -We'd love to accept your patches and contributions to this sample. There are -just a few small guidelines you need to follow. +We welcome contributions to the Agent Starter Pack! This guide helps you understand the project and make effective contributions. -## Contributor License Agreement +## Quick Navigation -Contributions to this project must be accompanied by a Contributor License Agreement. You (or your employer) retain the copyright to your contribution; this simply gives us permission to use and redistribute your contributions as part of the project. Head over to [Google Developers CLA](https://cla.developers.google.com/) to see your current agreements on file or to sign a new one. +| What you're changing | Start here | +|---------------------|------------| +| Template files (Jinja templates in base_templates/) | [Template Development Workflow](#template-development-workflow) | +| CLI commands (cli/commands/) | [Code Quality](#code-quality) | +| Documentation | [Pull Request Process](#pull-request-process) | +| **AI coding agents** | See [GEMINI.md](./GEMINI.md) for comprehensive AI agent guidance | -You generally only need to submit a CLA once, so if you've already submitted one (even if it was for a different project), you probably don't need to do it again. +## Understanding ASP Architecture -## Code Reviews +Agent Starter Pack is a **template generator**, not a runtime framework. The CLI generates standalone projects that users then customize and deploy. -All submissions, including submissions by project members, require review. We -use GitHub pull requests for this purpose. Consult -[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more -information on using pull requests. +### 4-Layer Template System -## Community Guidelines +Templates are processed in order, with later layers overriding earlier ones: -This project follows [Google's Open Source Community Guidelines](https://opensource.google/conduct/). +1. **Base Templates** (`agent_starter_pack/base_templates//`) - Core Jinja scaffolding (Python, Go, more coming) +2. **Deployment Targets** (`agent_starter_pack/deployment_targets/`) - Environment-specific overrides (cloud_run, agent_engine) +3. **Frontend Types** (`agent_starter_pack/frontends/`) - UI-specific files +4. **Agent Templates** (`agent_starter_pack/agents/*/`) - Agent-specific logic and configurations -## Contributor Guide +### Key Directories -If you are new to contributing to open source, you can find helpful information in this contributor guide. +| Directory | Purpose | +|-----------|---------| +| `agent_starter_pack/base_templates/` | Jinja templates by language (see [Language-Specific Notes](#language-specific-notes)) | +| `agent_starter_pack/deployment_targets/` | Files that override/extend base for specific deployments | +| `agent_starter_pack/agents/` | Agent-specific files (adk, adk_live, langgraph, etc.) | +| `agent_starter_pack/cli/commands/` | CLI command implementations (create, setup-cicd, etc.) | +| `agent_starter_pack/cli/utils/` | Shared utilities including template processing | -You may follow these steps to contribute: +## Template Development Workflow -1. **Fork the official repository.** This will create a copy of the official repository in your own account. -2. **Sync the branches.** This will ensure that your copy of the repository is up-to-date with the latest changes from the official repository. -3. **Work on your forked repository's feature branch.** This is where you will make your changes to the code. -4. **Commit your updates on your forked repository's feature branch.** This will save your changes to your copy of the repository. -5. **Submit a pull request to the official repository's main branch.** This will request that your changes be merged into the official repository. -6. **Resolve any linting errors.** This will ensure that your changes are formatted correctly. - - For errors generated by [check-spelling](https://github.com/check-spelling/check-spelling), go to the [Job Summary](https://github.com/GoogleCloudPlatform/generative-ai/actions/workflows/spelling.yaml) to read the errors. - - Fix any spelling errors that are found. - - Forbidden Patterns are defined as regular expressions, you can copy/paste them into many IDEs to find the instances. [Example for Visual Studio Code](https://medium.com/@nikhilbaxi3/visual-studio-code-secrets-of-regular-expression-search-71723c2ecbd2). - - Add false positives to [`.github/actions/spelling/allow.txt`](.github/actions/spelling/allow.txt). Be sure to check that it's actually spelled correctly! +Template changes require a specific workflow because you're modifying Jinja templates. -Here are some additional things to keep in mind during the process: +### Recommended Process -- **Read the [Google's Open Source Community Guidelines](https://opensource.google/conduct/).** The contribution guidelines will provide you with more information about the project and how to contribute. -- **Test your changes.** Before you submit a pull request, make sure that your changes work as expected. -- **Be patient.** It may take some time for your pull request to be reviewed and merged. +1. **Generate a test instance** + ```bash + uv run agent-starter-pack create mytest -p -s -y -d cloud_run --output-dir target + ``` + Flags: `-p` prototype (no CI/CD), `-s` skip checks, `-y` auto-approve ---- +2. **Initialize git in the generated project** + ```bash + cd target/mytest && git init && git add . && git commit -m "Initial" + ``` -## For Google Employees +3. **Develop with tight feedback loops** + - Make changes directly in `target/mytest/` + - Test immediately with `make lint` and running the code + - Iterate until the change works + +4. **Backport changes to Jinja templates** + - Identify the source template file in `agent_starter_pack/` + - Apply your changes, adding Jinja conditionals if needed + - Use `{%- -%}` whitespace control carefully (see [GEMINI.md](./GEMINI.md#critical-whitespace-control-patterns)) + +5. **Validate across combinations** + ```bash + # Test your target combination + _TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents + + # Test alternate agent with same deployment + _TEST_AGENT_COMBINATION="adk_live,cloud_run" make lint-templated-agents -If you're a Google employee, as a requirement, please follow the steps outlined in the [Google Cloud Platform Generative AI repository's contributing guide](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/CONTRIBUTING.md#for-google-employees) for processes and requirements. + # Test same agent with alternate deployment + _TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents + ``` -## Code Quality Checks +### Why This Workflow? -To ensure code quality, we utilize automated checks. Before submitting a pull request, please run the following commands locally: +- Jinja templates are harder to debug than rendered code +- Generated projects give immediate feedback on syntax errors +- Backporting ensures you understand exactly what changed +- Cross-combination testing catches conditional logic bugs +### Language-Specific Notes + +The template development workflow above applies to all languages. Here are language-specific details: + +| Language | Agent(s) | Linter | Test Command | Status | +|----------|----------|--------|--------------|--------| +| Python | adk, adk_a2a, adk_live, agentic_rag, langgraph, custom_a2a | `ruff`, `ty` | `make lint` | Production | +| Go | adk_go | `golangci-lint` | `make lint` | Production | +| Java | (coming soon) | - | - | In development | +| TypeScript | (coming soon) | - | - | In development | + +**Python example:** ```bash -make install +uv run agent-starter-pack create mytest -a adk -d cloud_run -p -s -y --output-dir target ``` -This installs development dependencies, including linting tools. +**Go example:** +```bash +uv run agent-starter-pack create mytest -a adk_go -d cloud_run -p -s -y --output-dir target +``` -Then, execute the following Make command: +## Testing Your Changes + +### Local Testing ```bash +# Install dependencies (first time) +make install + +# Run linters (ruff, ty, codespell) make lint + +# Run test suite +make test ``` -This command runs the following linters to check for code style, potential errors, and type hints: +### Template Combination Testing -- **codespell**: Detects common spelling mistakes in code and documentation. -- **ruff**: A fast Python linter and formatter, it checks for errors, coding standards, and enforces style consistency. -- **ty**: Performs static type checking to catch type errors before runtime (Astral's Rust-based type checker). +Template changes can affect multiple agent/deployment combinations. Test the minimum coverage matrix: ```bash -make test +# Format: agent,deployment_target,--flag,value +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents +_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="langgraph,cloud_run" make lint-templated-agents +``` + +**Minimum coverage before PR:** +- [ ] Your target combination +- [ ] One alternate agent with same deployment target +- [ ] One alternate deployment target with same agent + +### Quick Project Generation for Testing + +```bash +# Fastest: Agent Engine prototype +uv run agent-starter-pack create test-$(date +%s) -p -s -y -d agent_engine --output-dir target + +# Cloud Run with session type +uv run agent-starter-pack create test-$(date +%s) -p -s -y -d cloud_run --session-type in_memory --output-dir target + +# Full project with CI/CD +uv run agent-starter-pack create test-$(date +%s) -s -y -d agent_engine --cicd-runner google_cloud_build --output-dir target +``` + +## Pull Request Process + +### Commit Message Format + ``` +: + + +- Why the change was needed +- What was the root cause (for fixes) +- How the fix addresses it +``` + +**Types**: `fix`, `feat`, `refactor`, `docs`, `test`, `chore` + +### PR Description Template + +```markdown +## Summary +- Key change 1 +- Key change 2 + +## Problem (for fixes) +Description of the issue, including error messages if applicable. + +## Solution +How the changes fix the problem. + +## Testing +- [ ] Tested combination: agent,deployment +- [ ] Tested alternate: agent2,deployment +- [ ] Tested alternate: agent,deployment2 +``` + +### Required Checks + +All PRs must pass: +- `make lint` - Code style and type checking +- `make test` - Unit and integration tests +- Template linting for affected combinations + +## Code Quality + +### Linting Tools + +| Tool | Purpose | +|------|---------| +| `ruff` | Python linting and formatting | +| `ty` | Static type checking (Astral's Rust-based checker) | +| `codespell` | Spelling mistakes in code and docs | + +### Template-Specific Linting + +For template files, also ensure: +- All Jinja blocks are balanced (`{% if %}` has `{% endif %}`) +- Whitespace control is correct (see [GEMINI.md](./GEMINI.md#critical-whitespace-control-patterns)) +- Generated files pass `ruff` in all combinations + +### Spelling Errors + +For errors from [check-spelling](https://github.com/check-spelling/check-spelling): +1. Check the Job Summary for specific errors +2. Fix actual spelling mistakes +3. Add false positives to `.github/actions/spelling/allow.txt` + +## Contributor License Agreement + +Contributions must be accompanied by a Contributor License Agreement. You (or your employer) retain copyright; this gives us permission to use and redistribute your contributions. + +Sign at [Google Developers CLA](https://cla.developers.google.com/). You generally only need to submit once. + +## Community Guidelines + +This project follows [Google's Open Source Community Guidelines](https://opensource.google/conduct/). + +## For Google Employees + +Follow the additional steps in the [Generative AI on Google Cloud repository's contributing guide](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/CONTRIBUTING.md#for-google-employees). + +--- -This command runs the test suite using pytest, covering both unit and integration tests: +## Additional Resources -Your pull request will also be automatically checked by these tools using GitHub Actions. Ensuring your code passes these checks locally will help expedite the review process. +- **[GEMINI.md](./GEMINI.md)** - Comprehensive guide for AI coding agents, including Jinja patterns and debugging +- **[docs/](./docs/)** - User-facing documentation for generated projects diff --git a/GEMINI.md b/GEMINI.md index debf29d2b..2ef096c08 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -1,203 +1,245 @@ -# Agent Starter Pack - Coding Agent Guide +# Agent Starter Pack - AI Coding Agent Guide + +> **Scope**: This document is for AI coding agents contributing to the **Agent Starter Pack repository itself** (the template generator). For guidance on working with **generated projects**, see [llm.txt](./llm.txt). This document provides essential guidance, architectural insights, and best practices for AI coding agents tasked with modifying the Google Cloud Agent Starter Pack. Adhering to these principles is critical for making safe, consistent, and effective changes. +--- + ## Core Principles for AI Agents 1. **Preserve and Isolate:** Your primary objective is surgical precision. Modify *only* the code segments directly related to the user's request. Preserve all surrounding code, comments, and formatting. Do not rewrite entire files or functions to make a small change. 2. **Follow Conventions:** This project relies heavily on established patterns. Before writing new code, analyze the surrounding files to understand and replicate existing conventions for naming, templating logic, and directory structure. -3. **Search Comprehensively:** A single change often requires updates in multiple places. When modifying configuration, variables, or infrastructure, you **must** search across the entire repository, including: - * `agent-starter-pack/base_template/` (the core template) - * `agent-starter-pack/deployment_targets/` (environment-specific overrides) +3. **Template-First Mindset:** ASP is a template generator. The CLI should remain lean with good defaults. Most features belong in templates, not CLI code. +4. **Search Comprehensively:** A single change often requires updates in multiple places. When modifying configuration, variables, or infrastructure, you **must** search across the entire repository, including: + * `agent_starter_pack/base_templates/` (core templates by language) + * `agent_starter_pack/deployment_targets/` (environment-specific overrides) * `.github/` and `.cloudbuild/` (CI/CD workflows) * `docs/` (user-facing documentation) +--- + ## Project Architecture Overview -### Templating Engine: Cookiecutter + Jinja2 +### 4-Layer Template System -The starter pack uses **Cookiecutter** to generate project scaffolding from templates that are heavily customized with the **Jinja2** templating language. Understanding the rendering process is key to avoiding errors. +Template processing follows this hierarchy (later layers override earlier ones): -**Multi-Phase Template Processing:** +| Layer | Directory | Purpose | +|-------|-----------|---------| +| 1. Base | `agent_starter_pack/base_templates//` | Core Jinja scaffolding (Python, Go, more coming) | +| 2. Deployment | `agent_starter_pack/deployment_targets/` | Environment overrides (cloud_run, agent_engine) | +| 3. Frontend | `agent_starter_pack/frontends/` | UI-specific files | +| 4. Agent | `agent_starter_pack/agents/*/` | Agent-specific logic and configurations | -Templates are processed in a specific order. A failure at any stage will break the project generation. +**Rule**: Always place changes in the correct layer. Check if deployment targets need corresponding updates. -1. **Cookiecutter Variable Substitution:** Simple replacement of `{{cookiecutter.variable_name}}` placeholders with values from `cookiecutter.json`. -2. **Jinja2 Logic Execution:** Conditional blocks (`{% if %}`), loops (`{% for %}`), and other logic are executed. This is the most complex and error-prone stage. -3. **File/Directory Name Templating:** File and directory names containing Jinja2 blocks are rendered. For example, `{% if cookiecutter.cicd_runner == 'github_actions' %}.github{% else %}unused_github{% endif %}`. +### Key Directory Structure + +``` +agent_starter_pack/ +├── agents/ # Agent-specific files +│ ├── adk/ # Base ADK agent (Python) +│ ├── adk_a2a/ # A2A-enabled ADK agent +│ ├── adk_go/ # Base ADK agent (Go) +│ ├── adk_live/ # Real-time multimodal agent +│ ├── agentic_rag/ # RAG agent +│ └── langgraph/ # LangGraph-based agent +├── base_templates/ # Core Jinja templates by language +│ ├── python/ # Python project template +│ │ ├── {{cookiecutter.agent_directory}}/ +│ │ ├── deployment/ +│ │ ├── tests/ +│ │ └── Makefile +│ └── go/ # Go project template +├── deployment_targets/ # Environment-specific overrides +│ ├── agent_engine/ # Agent Engine deployment +│ └── cloud_run/ # Cloud Run deployment +├── frontends/ # UI templates +└── cli/ # CLI implementation + ├── commands/ # create, setup-cicd, enhance, etc. + └── utils/ # Template processing, helpers +``` + +### When to Modify What + +| Change Type | Where to Modify | Also Check | +|-------------|-----------------|------------| +| Affects ALL generated projects | `base_templates//` | Deployment targets for conflicts | +| Deployment-specific logic | `deployment_targets//` | Base templates for shared code | +| Agent-specific feature | `agents//` | Other agents for consistency | +| New CLI flag/command | `cli/commands/` | `cli/utils/` for shared logic | +| CI/CD changes | Both `.github/` AND `.cloudbuild/` | Keep in sync | +| Documentation | `docs/` | README.md for overview changes | -### Key Directory Structures +### Template Processing Flow -- `agent-starter-pack/base_template/`: This is the **core template**. Most changes that should apply to all generated projects should start here. -- `agent-starter-pack/deployment_targets/`: Contains files that **override or are added to** the `base_template` for a specific deployment target (e.g., `cloud_run`, `gke`, `agent_engine`). If a file exists in both `base_template` and a deployment target, the latter is typically used. -- `agent-starter-pack/agents/`: Contains pre-packaged, self-contained agent examples. Each has its own `.template/templateconfig.yaml` to define its specific variables and dependencies. -- `agent-starter-pack/cli/commands`: Contains the logic for the CLI commands, such as `create` and `setup-cicd`. +1. **Variable resolution** from `cookiecutter.json` +2. **File copying** (base → deployment → frontend → agent overlays) +3. **Jinja2 rendering** of file content +4. **File/directory name rendering** (Jinja in filenames) -### CLI Commands +### Cross-File Dependencies -- `create.py`: Handles the creation of new agent projects. It orchestrates the template processing, configuration merging, and deployment target selection. -- `setup_cicd.py`: Automates the setup of the CI/CD pipeline. It interacts with `gcloud` and `gh` to configure GitHub repositories, Cloud Build triggers, and Terraform backend. +Changes often require coordinated updates: +- **Configuration**: `templateconfig.yaml` → `cookiecutter.json` → rendered templates +- **CI/CD**: `.github/workflows/` ↔ `.cloudbuild/` (must stay in sync) +- **Infrastructure**: Base terraform → deployment target overrides -### Template Processing +--- -- `template.py`: Located in `agent-starter-pack/cli/utils`, this script contains the core logic for processing the templates. It copies the base template, overlays the deployment target files, and then applies the agent-specific files. +## Template Development Workflow -## Critical Jinja Templating Rules +Template changes require a specific workflow because you're modifying Jinja templates, not regular source files. -Failure to follow these rules is the most common source of project generation errors. +> **Note:** This workflow applies to both Python and Go templates. Both use Jinja templating with the same patterns (`{{cookiecutter.*}}`, `{% if %}`, etc.). -### 1. Block Balancing -**Every opening Jinja block must have a corresponding closing block.** This is the most critical rule. +### Step-by-Step Process -- `{% if ... %}` requires `{% endif %}` -- `{% for ... %}` requires `{% endfor %}` -- `{% raw %}` requires `{% endraw %}` +#### 1. Generate a Test Instance -**Correct:** -```jinja -{% if cookiecutter.deployment_target == 'cloud_run' %} - # Cloud Run specific content -{% endif %} +```bash +uv run agent-starter-pack create mytest -p -s -y -d cloud_run --output-dir target ``` -### 2. Variable Usage -Distinguish between substitution and logic: +Flags explained: +- `-p` / `--prototype`: Minimal project (no CI/CD or Terraform) +- `-s` / `--skip-checks`: Skip GCP/Vertex AI verification +- `-y` / `--auto-approve`: Skip all confirmation prompts +- `-d`: Deployment target +- `--output-dir target`: Output to target/ (gitignored) -- **Substitution (in file content):** Use double curly braces: `{{ cookiecutter.project_name }}` -- **Logic (in `if`/`for` blocks):** Use the variable directly: `{% if cookiecutter.session_type == 'cloud_sql' %}` +#### 2. Initialize Git in the Generated Project -### 3. Whitespace Control -Jinja is sensitive to whitespace. Use hyphens to control newlines and prevent unwanted blank lines in rendered files. +```bash +cd target/mytest && git init && git add . && git commit -m "Initial" +``` -- `{%-` removes whitespace before the block. -- `-%}` removes whitespace after the block. +This creates a baseline for tracking your changes with `git diff`. -**Example:** -```jinja -{%- if cookiecutter.some_option %} -option = true -{%- endif %} -``` +#### 3. Develop with Tight Feedback Loops -## Terraform Best Practices +- Make changes directly in `target/mytest/` +- Test immediately: `make lint`, run the code, check output +- Iterate until the change works correctly +- Use `git diff` to see exactly what you changed -### Unified Service Account (`app_sa`) -The project uses a single, unified application service account (`app_sa`) across all deployment targets to simplify IAM management. +#### 4. Backport Changes to Jinja Templates -- **Do not** create target-specific service accounts (e.g., `cloud_run_sa`). -- Define roles for this account in `app_sa_roles`. -- Reference this account consistently in all Terraform and CI/CD files. +- Find the source template: `find agent_starter_pack -name "filename.py" -type f` +- Apply your changes to the template file +- Add Jinja conditionals if the change is conditional +- Use `{%- -%}` whitespace control carefully -### Resource Referencing -Use consistent and clear naming for Terraform resources. When referencing resources, especially those created conditionally or with `for_each`, ensure the reference is also correctly keyed. +#### 5. Validate Across Combinations -**Example:** -```hcl -# Creation -resource "google_service_account" "app_sa" { - for_each = local.deploy_project_ids # e.g., {"staging" = "...", "prod" = "..."} - account_id = "${var.project_name}-app" - # ... -} +```bash +# Test your target combination +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents -# Correct Reference -# In a Cloud Run module for the staging environment -service_account = google_service_account.app_sa["staging"].email +# Test alternate agent with same deployment +_TEST_AGENT_COMBINATION="adk_live,cloud_run" make lint-templated-agents + +# Test same agent with alternate deployment +_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents ``` -## CI/CD Integration (GitHub Actions & Cloud Build) +### Why This Workflow? -The project maintains parallel CI/CD implementations. **Any change to CI/CD logic must be applied to both.** +- Jinja templates are harder to debug than rendered code +- Generated projects give immediate feedback on syntax errors +- Backporting ensures you understand exactly what changed +- Cross-combination testing catches conditional logic bugs -- **GitHub Actions:** Configured in `.github/workflows/`. Uses `${{ vars.VAR_NAME }}` for repository variables. -- **Google Cloud Build:** Configured in `.cloudbuild/`. Uses `${_VAR_NAME}` for substitution variables. +### Language-Specific Notes -When adding a new variable or secret, ensure it is configured correctly for both systems in the Terraform scripts that manage them (e.g., `github_actions_variable` resource and Cloud Build trigger substitutions). +The template development workflow above applies to all languages. Here are language-specific details: -## Advanced Template System Patterns +| Language | Agent(s) | Linter | Test Command | Status | +|----------|----------|--------|--------------|--------| +| Python | adk, adk_a2a, adk_live, agentic_rag, langgraph, custom_a2a | `ruff`, `ty` | `make lint` | Production | +| Go | adk_go | `golangci-lint` | `make lint` | Production | +| Java | (coming soon) | - | - | In development | +| TypeScript | (coming soon) | - | - | In development | -### 4-Layer Architecture -Template processing follows this hierarchy (later layers override earlier ones): -1. **Base Template** (`agent-starter-pack/base_template/`) - Applied to ALL projects -2. **Deployment Targets** (`agent-starter-pack/deployment_targets/`) - Environment overrides -3. **Frontend Types** (`agent-starter-pack/frontends/`) - UI-specific files -4. **Agent Templates** (`agent-starter-pack/agents/*/`) - Individual agent logic +**Python example:** +```bash +uv run agent-starter-pack create mytest -a adk -d cloud_run -p -s -y --output-dir target +``` -**Rule**: Always place changes in the correct layer. Check if deployment targets need corresponding updates. +**Go example:** +```bash +uv run agent-starter-pack create mytest -a adk_go -d cloud_run -p -s -y --output-dir target +``` -### Template Processing Flow -1. Variable resolution from `cookiecutter.json` -2. File copying (base → overlays) -3. Jinja2 rendering of content -4. File/directory name rendering +--- -### Cross-File Dependencies -Changes often require coordinated updates: -- **Configuration**: `templateconfig.yaml` → `cookiecutter.json` → rendered templates -- **CI/CD**: `.github/workflows/` ↔ `.cloudbuild/` (must stay in sync) -- **Infrastructure**: Base terraform → deployment target overrides +## Jinja Templating Rules -### Conditional Logic Patterns -```jinja -{%- if cookiecutter.agent_name == "adk_live" %} -# Agent-specific logic -{%- elif cookiecutter.deployment_target == "cloud_run" %} -# Deployment-specific logic -{%- endif %} -``` +> **Note:** These rules apply to both Python and Go templates. Both languages use the same Jinja2 templating patterns. + +### Templating Engine: Cookiecutter + Jinja2 -### Testing Strategy -Test changes across multiple dimensions: -- Agent types (adk_live, adk, etc.) -- Deployment targets (cloud_run, agent_engine) -- Feature combinations (data_ingestion, frontend_type) -- Example command for testing the starter pack creation - from the root of the repo run: `uv run agent-starter-pack create myagent-$(date +%s) --output-dir target` +The starter pack uses **Cookiecutter** to generate project scaffolding from templates customized with **Jinja2**. Understanding the rendering process is key to avoiding errors. -### Common Pitfalls -- **Hardcoded URLs**: Use relative paths for frontend connections -- **Missing Conditionals**: Wrap agent-specific code in proper `{% if %}` blocks -- **Dependency Conflicts**: Some agents lack certain extras (e.g., adk_live + lint) +**Multi-Phase Template Processing:** -## Linting and Testing Multiple Combinations +1. **Cookiecutter Variable Substitution:** Replacement of `{{cookiecutter.variable_name}}` placeholders +2. **Jinja2 Logic Execution:** Conditional blocks (`{% if %}`), loops (`{% for %}`) +3. **File/Directory Name Templating:** Jinja2 in filenames is rendered -**IMPORTANT:** Only run linting when explicitly requested by the user. Do not proactively lint unless asked. +### Block Balancing (Critical) -### Linting System +**Every opening Jinja block must have a corresponding closing block.** -The project uses **Ruff** for linting/formatting and **ty** for type checking. Use these commands to validate template combinations: +- `{% if ... %}` requires `{% endif %}` +- `{% for ... %}` requires `{% endfor %}` +- `{% raw %}` requires `{% endraw %}` -**Linting:** -```bash -_TEST_AGENT_COMBINATION="agent,target,--param,value" make lint-templated-agents +```jinja +{% if cookiecutter.deployment_target == 'cloud_run' %} + # Cloud Run specific content +{% endif %} ``` -**Testing:** -```bash -_TEST_AGENT_COMBINATION="agent,target,--param,value" make test-templated-agents -``` +### Variable Usage -Both commands use the same `_TEST_AGENT_COMBINATION` environment variable to control which agent combination to validate. +Distinguish between substitution and logic: -### Testing Methodology +- **Substitution (in file content):** `{{ cookiecutter.project_name }}` +- **Logic (in `if`/`for` blocks):** `{% if cookiecutter.session_type == 'cloud_sql' %}` -**Critical Principle:** Template changes can affect MULTIPLE agent/deployment combinations. Test across combinations when making template modifications. +### Whitespace Control -**Common Combinations:** -```bash -# Linting examples -_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents -_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents +Jinja is sensitive to whitespace. Use hyphens to control newlines: + +- `{%-` removes whitespace before the block +- `-%}` removes whitespace after the block +- `{%- -%}` removes whitespace on both sides + +```jinja +{%- if cookiecutter.some_option %} +option = true +{%- endif %} +``` + +### Conditional Logic Patterns -# Testing examples -_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make test-templated-agents -_TEST_AGENT_COMBINATION="langgraph,agent_engine" make test-templated-agents +```jinja +{%- if cookiecutter.agent_name == "adk_live" %} +# Agent-specific logic +{%- elif cookiecutter.deployment_target == "cloud_run" %} +# Deployment-specific logic +{%- endif %} ``` -### Critical Whitespace Control Patterns +--- + +## Critical Whitespace Control Patterns Jinja2 whitespace control is the #1 source of linting failures. Understanding these patterns is essential. -#### Pattern 1: Conditional Imports with Blank Line Separation +### Pattern 1: Conditional Imports with Blank Line Separation **Problem:** Python requires blank lines to separate third-party imports from project imports. Conditional imports must handle this correctly. @@ -230,7 +272,7 @@ from {{cookiecutter.agent_directory}}.app_utils.gcs import create_bucket_if_not_ - The blank line AFTER the conditional import goes INSIDE the if block when needed - Test BOTH when condition is true AND false -#### Pattern 2: Long Import Lines +### Pattern 2: Long Import Lines **Problem:** Ruff enforces line length limits. Long import statements must be split. @@ -250,9 +292,9 @@ from app.app_utils.typing import ( ) ``` -#### Pattern 3: File End Newlines +### Pattern 3: File End Newlines -**Problem:** Ruff requires exactly ONE newline at the end of every file, no more, no less. +**Problem:** Ruff requires exactly ONE newline at the end of every file. **Wrong - No newline:** ```jinja @@ -287,9 +329,119 @@ import logging Notice `{%- endif -%}` to prevent blank line before the else block. -### Debugging Linting Failures +### Whitespace Control Cheat Sheet + +```jinja +# Remove whitespace BEFORE the tag +{%- if condition %} + +# Remove whitespace AFTER the tag +{% if condition -%} + +# Remove whitespace on BOTH sides +{%- if condition -%} + +# Typical pattern for conditional imports +{% if condition -%} +import something +{% endif %} + +# Typical pattern for conditional code blocks with blank line before +{%- if condition %} + +some_code() +{%- endif %} + +# Pattern for preventing blank line between consecutive conditionals +{%- endif -%} +{%- if next_condition %} +``` + +--- + +## Testing Strategy + +### Testing Coverage Matrix + +**Critical Principle:** Template changes can affect MULTIPLE agent/deployment combinations. Test across combinations when making template modifications. + +| Dimension | Options | +|-----------|---------| +| Agents | adk, adk_a2a, adk_go, adk_live, agentic_rag, langgraph | +| Deployments | cloud_run, agent_engine | +| Session types | in_memory, cloud_sql, agent_engine | +| Features | data_ingestion, frontend_type | + +### Minimum Coverage Before PR + +- [ ] Your target combination +- [ ] One alternate agent with same deployment target +- [ ] One alternate deployment target with same agent + +### Linting Commands + +**IMPORTANT:** Only run linting when explicitly requested by the user. Do not proactively lint unless asked. + +```bash +# Linting a specific combination +_TEST_AGENT_COMBINATION="agent,target,--param,value" make lint-templated-agents + +# Testing a specific combination +_TEST_AGENT_COMBINATION="agent,target,--param,value" make test-templated-agents +``` + +### Common Test Combinations + +```bash +# Cloud Run combinations (Python) +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents +_TEST_AGENT_COMBINATION="adk_live,cloud_run,--session-type,in_memory" make lint-templated-agents + +# Agent Engine combinations (Python) +_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="adk_live,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="langgraph,agent_engine" make lint-templated-agents + +# Go template testing +_TEST_AGENT_COMBINATION="adk_go,cloud_run" make lint-templated-agents + +# With session type variations +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,agent_engine" make lint-templated-agents +``` + +### Testing Workflow for Template Changes + +**Before committing ANY template change:** + +```bash +# 1. Test the specific combination you're working on +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents + +# 2. Test related combinations (same deployment, different agents) +_TEST_AGENT_COMBINATION="adk_live,cloud_run,--session-type,in_memory" make lint-templated-agents + +# 3. Test alternate code paths (different deployment, session types) +_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents + +# 4. If modifying deployment target files, test all agents with that target +# For agent_engine changes: +_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="adk_live,agent_engine" make lint-templated-agents +_TEST_AGENT_COMBINATION="langgraph,agent_engine" make lint-templated-agents +``` + +**Golden Rule:** After ANY template change affecting imports, conditionals, or file endings, test AT LEAST 3 combinations: +1. The target combination +2. An alternate agent with same deployment +3. An alternate deployment with same agent + +--- + +## Debugging Linting Failures + +### Step 1: Identify the Exact Error -**Step 1: Identify the exact error** ```bash # Look for the diff output in the error message --- app/fast_api_app.py @@ -303,19 +455,22 @@ Notice `{%- endif -%}` to prevent blank line before the else block. The `+` line shows what Ruff WANTS to add. In this case, it wants a blank line after `agent_engines`. -**Step 2: Find the generated file** +### Step 2: Find the Generated File + ```bash # Generated files are in target/ cat target/project-name/app/fast_api_app.py | head -30 ``` -**Step 3: Trace back to template** +### Step 3: Trace Back to Template + ```bash # Find the template source find agent_starter_pack -name "fast_api_app.py" -type f ``` -**Step 4: Check BOTH branches of conditionals** +### Step 4: Check BOTH Branches of Conditionals + - If `{% if condition %}` exists, test with condition true AND false - Use different agent combinations to toggle different conditions @@ -344,74 +499,52 @@ find agent_starter_pack -name "fast_api_app.py" -type f - Different agents trigger different code paths - Must test multiple agent types -### Testing Workflow for Template Changes +--- -**Before committing ANY template change:** +## CI/CD Integration -```bash -# 1. Test the specific combination you're working on -_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,in_memory" make lint-templated-agents +The project maintains parallel CI/CD implementations. **Any change to CI/CD logic must be applied to both.** -# 2. Test related combinations (same deployment, different agents) -_TEST_AGENT_COMBINATION="adk_live,cloud_run,--session-type,in_memory" make lint-templated-agents +- **GitHub Actions:** Configured in `.github/workflows/`. Uses `${{ vars.VAR_NAME }}` for repository variables. +- **Google Cloud Build:** Configured in `.cloudbuild/`. Uses `${_VAR_NAME}` for substitution variables. -# 3. Test alternate code paths (different deployment, session types) -_TEST_AGENT_COMBINATION="adk,cloud_run,--session-type,agent_engine" make lint-templated-agents -_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents +When adding a new variable or secret, ensure it is configured correctly for both systems in the Terraform scripts that manage them (e.g., `github_actions_variable` resource and Cloud Build trigger substitutions). -# 4. If modifying deployment target files, test all agents with that target -# For agent_engine changes: -_TEST_AGENT_COMBINATION="adk,agent_engine" make lint-templated-agents -_TEST_AGENT_COMBINATION="adk_live,agent_engine" make lint-templated-agents -_TEST_AGENT_COMBINATION="langgraph,agent_engine" make lint-templated-agents -``` +--- -### Quick Reference: Whitespace Control Cheat Sheet +## Terraform Best Practices -```jinja -# Remove whitespace BEFORE the tag -{%- if condition %} +### Unified Service Account (`app_sa`) -# Remove whitespace AFTER the tag -{% if condition -%} +The project uses a single, unified application service account (`app_sa`) across all deployment targets to simplify IAM management. -# Remove whitespace on BOTH sides -{%- if condition -%} +- **Do not** create target-specific service accounts (e.g., `cloud_run_sa`) +- Define roles for this account in `app_sa_roles` +- Reference this account consistently in all Terraform and CI/CD files -# Typical pattern for conditional imports -{% if condition -%} -import something -{% endif %} +### Resource Referencing -# Typical pattern for conditional code blocks with blank line before -{%- if condition %} +Use consistent and clear naming for Terraform resources. When referencing resources, especially those created conditionally or with `for_each`, ensure the reference is also correctly keyed. -some_code() -{%- endif %} +```hcl +# Creation +resource "google_service_account" "app_sa" { + for_each = local.deploy_project_ids # e.g., {"staging" = "...", "prod" = "..."} + account_id = "${var.project_name}-app" + # ... +} -# Pattern for preventing blank line between consecutive conditionals -{%- endif -%} -{%- if next_condition %} +# Correct Reference +# In a Cloud Run module for the staging environment +service_account = google_service_account.app_sa["staging"].email ``` -**Golden Rule:** After ANY template change affecting imports, conditionals, or file endings, test AT LEAST 3 combinations: -1. The target combination -2. An alternate agent with same deployment -3. An alternate deployment with same agent - -## File Modification Checklist - -- [ ] **Jinja Syntax:** All `{% if %}` and `{% for %}` blocks correctly closed? -- [ ] **Variable Consistency:** `cookiecutter.` variables spelled correctly? -- [ ] **Cross-Target Impact:** Base template changes checked against deployment targets? -- [ ] **CI/CD Parity:** Changes applied to both GitHub Actions and Cloud Build? -- [ ] **Multi-Agent Testing:** Tested with different agent types and configurations? +--- ## Pull Request Best Practices -When creating pull requests for this repository, follow these guidelines for clear, professional commits and PR descriptions. - ### Commit Message Format + ``` : @@ -488,17 +621,24 @@ before creating the connection. ``` ### Key Principles + - **Concise but complete**: Provide enough context for reviewers - **Problem-first**: Explain the "why" before the "what" - **Professional tone**: Avoid mentions of AI tools or assistants -### Key Tooling +--- -- **`uv` for Python:** Primary tool for dependency management and CLI execution +## File Modification Checklist -## Quick Testing Commands +- [ ] **Jinja Syntax:** All `{% if %}` and `{% for %}` blocks correctly closed? +- [ ] **Variable Consistency:** `cookiecutter.` variables spelled correctly? +- [ ] **Cross-Target Impact:** Base template changes checked against deployment targets? +- [ ] **CI/CD Parity:** Changes applied to both GitHub Actions and Cloud Build? +- [ ] **Multi-Agent Testing:** Tested with different agent types and configurations? + +--- -When developing or testing CLI changes, use these shortcuts for fast iteration: +## Quick Reference ### Fast Project Creation @@ -527,14 +667,29 @@ uv run agent-starter-pack create test-$(date +%s) -p -s -y -d cloud_run --sessio uv run agent-starter-pack create test-$(date +%s) -s -y -d agent_engine --cicd-runner google_cloud_build --output-dir target ``` -## Key Files Reference +### Key Files Reference | File | Purpose | |------|---------| | `agent_starter_pack/cli/commands/create.py` | Main create command, CLI flags, shared options | | `agent_starter_pack/cli/utils/template.py` | Template processing, `process_template()`, CI/CD runner prompt | -| `agent_starter_pack/base_template/pyproject.toml` | Generated project metadata, `[tool.agent-starter-pack]` section | -| `agent_starter_pack/base_template/Makefile` | Generated project Makefile targets | +| `agent_starter_pack/base_templates/python/pyproject.toml` | Generated project metadata, `[tool.agent-starter-pack]` section | +| `agent_starter_pack/base_templates/python/Makefile` | Generated project Makefile targets | + +### Key Tooling + +- **`uv` for Python:** Primary tool for dependency management and CLI execution + +--- + +## Common Pitfalls + +- **Hardcoded URLs**: Use relative paths for frontend connections +- **Missing Conditionals**: Wrap agent-specific code in proper `{% if %}` blocks +- **Dependency Conflicts**: Some agents lack certain extras (e.g., adk_live + lint) +- **makefile_hashes.json**: Can cause merge conflicts during active development - regenerate if needed + +--- ## Project Metadata Structure diff --git a/llm.txt b/llm.txt index 401f8106b..7d52f3fe0 100644 --- a/llm.txt +++ b/llm.txt @@ -1,5 +1,7 @@ --- -**llm.txt** documents the "Agent Starter Pack" repository, providing a source of truth on its purpose, features, and usage. +**llm.txt** - Guide for AI agents working with **generated projects** (projects created by Agent Starter Pack). + +> **Scope**: This document is for developing, testing, and deploying **generated agent projects**. For contributing to the **Agent Starter Pack repository itself** (the template generator), see [GEMINI.md](./GEMINI.md). --- ### Section 1: Project Overview