Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 77 additions & 28 deletions mintlify/tutorials/manage-projects-with-terraform.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,24 +27,29 @@ This tutorial is part of the **Bytebase Terraform Provider** series:

## What You'll Learn

Create projects to organize your databases into logical groups.
In this tutorial, you'll learn how to:

- **Organize** databases into logical project groups for better management
- **Configure** project-specific settings and policies
- **Set up** webhooks for project notifications
- **Choose** between dynamic and explicit database assignment patterns

## Prerequisites

Before starting this tutorial, ensure you have:

- **Completed Parts 1 & 2**: Environments and instances configured
- **Built-in instances**: `Test Sample Instance` and `Prod Sample Instance` from sample data
- **Terraform workspace**: From previous tutorials
- Completed [Part 1: Manage Environments with Terraform](/tutorials/manage-environments-with-terraform) and [Part 2: Manage Databases with Terraform](/tutorials/manage-databases-with-terraform)
- Access to the built-in `Test Sample Instance` and `Prod Sample Instance` from Bytebase sample data
- Your Terraform workspace from the previous tutorials

## Step 1 - Understand Current Setup

From previous tutorials, you have:
From the previous tutorials, your Bytebase workspace contains:

- **Test instance** with `hr_test` and `postgres` databases
- **Prod instance** with `hr_prod` and `postgres` databases
- **Test instance**: Contains `hr_test` and `postgres` databases
- **Prod instance**: Contains `hr_prod` and `postgres` databases

Currently, these `hr_test` and `hr_prod` databases are in the default `Sample Project`, while `postgres` databases are unassigned. You can view them in Bytebase by clicking **Databases** on the left sidebar. Let's organize them into new projects.
Currently, the `hr_test` and `hr_prod` databases are assigned to the default `Sample Project`, while the `postgres` databases remain unassigned. You can verify this by navigating to **Databases** in the left sidebar. We'll now organize these databases into dedicated projects for better management.

## Step 2 - Create Projects

Expand All @@ -56,39 +61,77 @@ Currently, these `hr_test` and `hr_prod` databases are in the default `Sample Pr
Create `3-projects.tf`:

```hcl 3-projects.tf
# Project One - All test databases
# Project One - Development/Test Environment
# Uses dynamic assignment to automatically include all test databases
resource "bytebase_project" "project_one" {
depends_on = [bytebase_instance.test]
resource_id = "project-one"
title = "Project One"

# Automatically include all databases from test instance
# Project-level configurations for development workflow
allow_modify_statement = true # Allow direct SQL modifications
auto_resolve_issue = false # Manual issue resolution for review
auto_enable_backup = false # Disable automatic backups for test data

# Dynamically include all databases from the test instance
databases = bytebase_instance.test.databases

# Configure Slack notifications for team collaboration
webhooks {
title = "Sample webhook 1"
type = "SLACK"
url = "https://webhook.site/91fcd52a-39f1-4e7b-a43a-ddf72796d6b1"
notification_types = [
"NOTIFY_ISSUE_APPROVED",
"NOTIFY_PIPELINE_ROLLOUT",
"ISSUE_CREATE",
]
}
Copy link

Copilot AI Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The webhook URL appears to be a test URL from webhook.site, which is a public service. In production documentation, consider using a placeholder URL like 'https://your-slack-webhook-url.com' or adding a note that users should replace this with their actual webhook URL to avoid potential security issues.

Copilot uses AI. Check for mistakes.
}

# Project Two - Specific production database
# Project Two - Production Environment
# Uses explicit assignment for precise control over production databases
resource "bytebase_project" "project_two" {
depends_on = [bytebase_instance.prod]
resource_id = "project-two"
title = "Project Two"

# Explicitly specify which database to include
# Explicitly specify which production databases to include
databases = [
"instances/prod-sample-instance/databases/hr_prod"
]
}
```

**Two Assignment Patterns**
<Note>
The configuration above demonstrates project-level settings including webhook notifications. For a complete list of available project configurations, refer to the [Terraform provider documentation](https://registry.terraform.io/providers/bytebase/bytebase/latest/docs/data-sources/project).

You can also configure SQL review policies at the project level. Learn more in [Part 5: Manage SQL Review Rules with Terraform](/tutorials/manage-sql-review-rules-with-terraform).
</Note>

1. **Dynamic**: `databases = bytebase_instance.test.databases`
### Database Assignment Patterns

- Automatically includes all databases from an instance
- Good for development/test environments
Choose the appropriate pattern based on your project's needs:

#### Dynamic Assignment

```hcl
databases = bytebase_instance.test.databases
```

1. **Explicit**: `databases = ["instances/.../databases/..."]`
- Manually specify each database
- Better control for production
- **Automatically includes** all databases from a specified instance
- **Best for**: Development and testing environments where databases change frequently
- **Benefit**: Self-maintaining as new databases are automatically included

#### Explicit Assignment

```hcl
databases = ["instances/prod-sample-instance/databases/hr_prod"]
```

- **Manually specify** each database by its full path
- **Best for**: Production environments requiring precise control
- **Benefit**: Granular control over which databases belong to the project

## Step 3 - Apply Configuration

Expand All @@ -99,27 +142,33 @@ terraform apply

## Step 4 - Verify Projects

1. Go to **Projects** in Bytebase.
1. Navigate to **Projects** in the Bytebase GUI.

1. You'll see three projects:
1. You should now see three projects organized as follows:

- **Sample Project** (original, now empty)
- **Sample Project** (original default project, now empty after database migration)

- **Project One** (contains `hr_test` and `postgres` which are both from `Test Sample Instance`)
- **Project One** (contains both `hr_test` and `postgres` databases from `Test Sample Instance`)

![project-1](/content/docs/tutorials/manage-projects-with-terraform/bb-project-1.webp)

- **Project Two** (contains `hr_prod` which is from `Prod Sample Instance`)
- **Project Two** (contains the `hr_prod` database from `Prod Sample Instance`)

![project-2](/content/docs/tutorials/manage-projects-with-terraform/bb-project-2.webp)

1. Click on **Project One** and navigate to **Settings** to review the configurations you've applied.

1. Click **Webhooks** and you'll see the webhook you set.

## Summary and Next Steps

You've successfully organized databases into projects! Key takeaways:
Congratulations! You've successfully organized your databases into logical projects. Here are the key concepts you've learned:

- **Projects group related databases** for better organization
- **Two assignment patterns**: dynamic (all databases) or explicit (specific databases)
- **Projects span environments**: can include both test and prod databases
- **Project Organization**: Projects serve as logical containers that group related databases for improved management and governance
- **Assignment Flexibility**: Choose between dynamic assignment (automatic inclusion) for development environments or explicit assignment (manual control) for production
- **Cross-Environment Projects**: Projects can span multiple environments, though it's often better to separate test and production databases into different projects
- **Project Configuration**: Customize project behavior with settings like statement modification permissions, automatic issue resolution, and backup policies
- **Notification Integration**: Set up webhooks to keep your team informed about project activities through Slack or other channels

<Card
title="Part 5: Manage General Settings with Terraform"
Expand Down
101 changes: 54 additions & 47 deletions mintlify/tutorials/manage-sql-review-rules-with-terraform.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,13 @@ This tutorial is part of the **Bytebase Terraform Provider** series:

## What You'll Learn

- **Configure** SQL review rules for schema standards
- **Enforce** naming conventions and structural requirements
- **Apply** different severity levels (`ERROR`, `WARNING`)
- **Target** specific environments with review policies
In this tutorial, you'll learn how to:

- **Set up** automated SQL review rules to maintain schema standards
- **Implement** naming conventions and structural requirements across your databases
- **Configure** severity levels (`ERROR`, `WARNING`) to control validation strictness
- **Apply** review policies to specific environments for targeted governance
- **Test** your SQL review rules to ensure they work as intended

## Prerequisites

Expand Down Expand Up @@ -60,9 +63,9 @@ quality and schema consistency before changes enter the approval workflow.

### Step 1 - Create Review Configuration

<Tip>
The examples below show some SQL Review rules. For a complete list of available rules and their configurations, see the above terraform provider resource documentation.
</Tip>
<Note>
The examples below demonstrate some SQL Review rules. For a comprehensive list of available rules and their configuration options, refer to the Terraform provider resource documentation linked above.
</Note>

Create `5-sql-review.tf` with the SQL review configuration:

Expand All @@ -77,42 +80,42 @@ resource "bytebase_review_config" "sample" {
title = "Sample SQL Review Config"
enabled = true

# Apply to production environment only
# Apply rules to production environment only
resources = toset([
bytebase_setting.environments.environment_setting[0].environment[1].name
])

# Rule 1: Warn about nullable columns
# Rule 1: Column nullability - warn about nullable columns to improve data integrity
rules {
type = "column.no-null"
engine = "POSTGRES"
level = "WARNING"
}

# Rule 2: Require specific columns
# Rule 2: Required audit columns - enforce standard tracking fields
rules {
type = "column.required"
engine = "POSTGRES"
level = "ERROR"
payload = "{\"list\":[\"id\",\"created_ts\",\"updated_ts\",\"creator_id\",\"updater_id\"]}"
}

# Rule 3: Tables must have primary key
# Rule 3: Primary key requirement - ensure all tables have primary keys
rules {
type = "table.require-pk"
engine = "POSTGRES"
level = "ERROR"
}

# Rule 4: Column naming convention
# Rule 4: Column naming standards - enforce snake_case naming convention
rules {
type = "naming.column"
engine = "POSTGRES"
level = "ERROR"
payload = "{\"format\":\"^[a-z]+(_[a-z]+)*$\",\"maxLength\":64}"
}

# Rule 5: Limit maximum rows in SELECT
# Rule 5: Query performance protection - limit maximum rows in SELECT statements
rules {
type = "statement.maximum-limit-value"
engine = "POSTGRES"
Expand All @@ -122,6 +125,10 @@ resource "bytebase_review_config" "sample" {
}
```

<Note>
The configuration above applies SQL Review rules specifically to the `Prod` environment. Alternatively, you can scope rules to a specific project by using `bytebase_project.sample_project.name` in the `resources` attribute.
</Note>

### Step 2 - Apply Configuration

```bash
Expand All @@ -131,81 +138,81 @@ terraform apply

### Step 3 - Verify in Bytebase

1. Go to **CI/CD > SQL Review** on the left sidebar, you should see `Sample SQL Review Config` listed. Note that it's applied to the `Prod` environment.
1. Navigate to **CI/CD > SQL Review** in the left sidebar. You should see `Sample SQL Review Config` listed, showing that it's applied to the `Prod` environment.

![sql-review-list](/content/docs/tutorials/manage-sql-review-rules-with-terraform/bb-sql-review-list.webp)

1. Click on it to view the configured rules.
1. Click on the configuration to examine the rules you've defined.

![sql-review-pg](/content/docs/tutorials/manage-sql-review-rules-with-terraform/bb-sql-review-pg.webp)

### Understanding SQL Review Rules

SQL Review rules are categorized by what they validate:
SQL Review rules are organized into categories based on what they validate:

#### 1. Column Rules

- **column.no-null**: Warns about nullable columns to encourage data integrity
- **column.required**: Enforces required columns (id, timestamps, audit fields) for consistency
- **column.no-null**: Issues warnings about nullable columns to promote data integrity and prevent null-related errors
- **column.required**: Mandates the presence of essential columns (ID, timestamps, audit fields) to ensure consistent table structure

#### 2. Table Rules

- **table.require-pk**: Ensures every table has a primary key for data integrity and performance
- **table.require-pk**: Guarantees that every table includes a primary key, which is crucial for data integrity, replication, and query performance

#### 3. Naming Rules

- **naming.column**: Enforces lowercase snake_case column names for consistency across your schema
- **naming.column**: Enforces consistent lowercase snake_case naming conventions across your database schema, improving readability and maintainability

#### 4. Statement Rules

- **statement.maximum-limit-value**: Prevents SELECT queries without proper limits to avoid performance issues
- **statement.maximum-limit-value**: Prevents potentially harmful SELECT queries by requiring appropriate LIMIT clauses, protecting against performance degradation

### Step 4 - Test SQL Review

Create a test SQL statement to see the review in action:
Let's validate that your SQL Review rules are working by testing them with some sample SQL:

1. Go to **Project Two**, and click **Database > Databases** on the left sidebar.
1. Check `hr_prod` and click **Edit Schema**.
1. Try creating a table that violates the rules:
1. Navigate to **Project Two**, then click **Database > Databases** in the left sidebar.
1. Select the `hr_prod` database and click **Edit Schema**.
1. Create a table that intentionally violates multiple rules:

```sql
-- This will trigger multiple violations
-- This table intentionally violates multiple SQL review rules
CREATE TABLE BadExample (
FirstName VARCHAR(50), -- Violates naming convention
LastName VARCHAR(50) -- Missing required columns, no primary key
FirstName VARCHAR(50), -- Violates snake_case naming convention
LastName VARCHAR(50) -- Missing required audit columns and primary key
);
```

Expected violations:

- ❌ Column naming convention (should be first_name)
- ❌ Missing required columns (id, created_ts, etc.)
- ❌ Column naming convention violation (should be `first_name`, `last_name`)
- ❌ Missing required audit columns (`id`, `created_ts`, `updated_ts`, `creator_id`, `updater_id`)
- ❌ No primary key defined

![sql-review-violations](/content/docs/tutorials/manage-sql-review-rules-with-terraform/bb-sql-review-violations.webp)

1. Even if you continue to create an issue, it will still show red SQL Review.
1. Even if you proceed to create an issue, the SQL Review violations will be prominently displayed with error indicators.

![issue-sql-review-errors](/content/docs/tutorials/manage-sql-review-rules-with-terraform/bb-issue-sql-review-errors.webp)

1. Try a compliant table:
1. Now try creating a compliant table that follows all your rules:

```sql
-- This follows all rules
-- This table complies with all SQL review rules
CREATE TABLE good_example (
id SERIAL PRIMARY KEY,
first_name VARCHAR(50) NOT NULL,
last_name VARCHAR(50) NOT NULL,
created_ts TIMESTAMP NOT NULL DEFAULT NOW(),
updated_ts TIMESTAMP NOT NULL DEFAULT NOW(),
creator_id INTEGER NOT NULL,
updater_id INTEGER NOT NULL
id SERIAL PRIMARY KEY, -- Required primary key
first_name VARCHAR(50) NOT NULL, -- Snake_case naming, not null
last_name VARCHAR(50) NOT NULL, -- Snake_case naming, not null
created_ts TIMESTAMP NOT NULL DEFAULT NOW(), -- Required audit column
updated_ts TIMESTAMP NOT NULL DEFAULT NOW(), -- Required audit column
creator_id INTEGER NOT NULL, -- Required audit column
updater_id INTEGER NOT NULL -- Required audit column
);
```

Expected result:

- ✅ All required rules pass
- ✅ All SQL review rules pass successfully

![issue-sql-review-pass](/content/docs/tutorials/manage-sql-review-rules-with-terraform/bb-issue-sql-review-pass.webp)

Expand Down Expand Up @@ -245,12 +252,12 @@ rules {
## Key Points

- **Rule Levels**:
- `ERROR`: Blocks the SQL statement from being executed
- `WARNING`: Allows execution but notifies users of potential issues
- **Engine Specific**: Rules can target specific database engines (PostgreSQL, MySQL, etc.)
- **Environment Scoped**: Apply different rules to test vs. production environments
- **Payload Format**: Complex rules use JSON payloads for configuration (e.g., regex patterns, lists)
- **Best Practice**: Start with WARNING level rules and gradually move to ERROR as teams adapt
- `ERROR`: Blocks SQL statement execution, preventing non-compliant changes from proceeding
- `WARNING`: Permits execution while alerting users to potential issues, useful for gradual policy adoption
- **Engine Specificity**: Rules can be tailored to specific database engines (PostgreSQL, MySQL, Oracle, etc.) to accommodate platform differences
- **Environment Scoping**: Apply stricter rules to production while allowing more flexibility in development and testing environments
- **Payload Configuration**: Complex rules utilize JSON payloads for detailed configuration (regex patterns, value lists, thresholds)
- **Implementation Strategy**: Begin with `WARNING` level rules to familiarize teams with standards, then gradually transition to `ERROR` level for enforcement

<Card
title="Part 6: Manage Users and Groups with Terraform"
Expand Down