Skip to content

docs: add sdl multi-file format docs #938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tianzhou merged 3 commits into from
Nov 19, 2025
Merged

docs: add sdl multi-file format docs #938

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5511fa0
docs: add sdl multi-file format docs
rebelice Nov 13, 2025
f0d71e9
chore: update
rebelice Nov 14, 2025
c135fe5
chore: update docs
rebelice Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 173 additions & 31 deletions mintlify/gitops/state-based-workflow/develop.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,33 +6,145 @@ Create SDL files that define your complete database schema in a declarative form

## Getting Started

When starting with state-based workflow:

1. **Export Current Schema** - In Bytebase database detail page, click `Export Schema` to download your current database schema
2. **Organize Schema Files** - Edit and organize the exported schema into manageable files
3. **Commit to Repository** - Add SDL files to version control
4. **Make Changes** - Update the desired state and commit
To use state-based workflow, you must start by exporting your database schema. Bytebase provides a built-in export feature that generates SDL files in the multi-file format, ready to commit to your Git repository.

### Quick Start Workflow

1. **Export** - Click `Export Schema` → `Multi-File (ZIP)` in Bytebase database detail page
2. **Extract** - Unzip the downloaded file to your Git repository
3. **Verify** - Check the exported structure follows the [multi-file format](#file-organization)
4. **Commit** - Add files to Git and push to your repository
5. **Make Changes** - Edit the SDL files to modify your schema
6. **Deploy** - Commit changes to trigger deployment via GitOps

<Tip>
The exported ZIP contains a complete, ready-to-use schema structure. You can immediately commit it to Git without any manual organization.
</Tip>

### How to Export Schema

1. **Navigate to Database Detail Page** - Go to your database in Bytebase
2. **Click "Export Schema" Button** - Find it in the action buttons at the top
3. **Choose Export Format**:
- **Multi-File (ZIP)** - Recommended for GitOps workflow
- Downloads organized schema as a ZIP file
- Each object in its own file (e.g., `schemas/public/tables/users.sql`)
- Ready to extract and commit to Git
- **Single File** - All objects in one SQL file
- Useful for quick review or legacy workflows
- Requires manual splitting if you want multi-file format

### What Gets Exported

The multi-file export automatically includes:
- **Tables** - With inline indexes, comments, and owned sequences
- **Views** - Regular views with their definitions
- **Materialized Views** - With their indexes and refresh settings
- **Functions & Procedures** - User-defined functions and stored procedures
- **Sequences** - Independent sequences consolidated per schema
- **Comments** - All object and column comments

## File Organization

Organize SDL files by schema or module:
Bytebase uses a **multi-file format** where each database object is stored in a separate file, organized by schema and object type. This structure provides clear organization and makes code reviews easier.

### Directory Structure

```
schema/
├── public.sql # Public schema objects
├── analytics.sql # Analytics schema
└── internal.sql # Internal schema
schemas/
├── public/
│ ├── tables/
│ │ ├── users.sql
│ │ ├── orders.sql
│ │ └── products.sql
│ ├── views/
│ │ ├── active_users.sql
│ │ └── order_summary.sql
│ ├── materialized_views/
│ │ └── sales_summary.sql
│ ├── functions/
│ │ └── get_user_count.sql
│ ├── procedures/
│ │ └── update_user_status.sql
│ └── sequences.sql
├── analytics/
│ ├── tables/
│ │ └── events.sql
│ └── views/
│ └── daily_stats.sql
└── internal/
└── tables/
└── audit_logs.sql
```

Or split by object type:
### Organization Rules

- **One object per file** - Each table, view, function, etc. has its own file
- **Schema-based grouping** - Objects are organized under their schema directory
- **Type-based subdirectories** - Within each schema, objects are grouped by type (`tables/`, `views/`, etc.)
- **Sequences consolidated** - Independent sequences are grouped in a single `sequences.sql` file per schema
- **Related objects bundled** - Table files include their indexes, comments, and owned sequences

### File Naming

Files are named after the database object they define:
- Tables: `schemas/{schema}/tables/{table_name}.sql`
- Views: `schemas/{schema}/views/{view_name}.sql`
- Materialized Views: `schemas/{schema}/materialized_views/{materialized_view_name}.sql`
- Functions: `schemas/{schema}/functions/{function_name}.sql`
- Procedures: `schemas/{schema}/procedures/{procedure_name}.sql`
- Sequences: `schemas/{schema}/sequences.sql` (consolidated)

### Example: Table File Content

A table file includes the table definition and all related objects:

```sql
-- schemas/public/tables/users.sql

CREATE TABLE public.users (
id INTEGER NOT NULL DEFAULT nextval('public.users_id_seq'::regclass),
username TEXT NOT NULL,
email TEXT NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT users_pkey PRIMARY KEY (id),
CONSTRAINT users_username_key UNIQUE (username),
CONSTRAINT users_email_key UNIQUE (email)
);

COMMENT ON TABLE public.users IS 'Application users';
COMMENT ON COLUMN public.users.username IS 'Unique username for login';

CREATE INDEX idx_users_email ON public.users(email);
CREATE INDEX idx_users_created_at ON public.users(created_at);

COMMENT ON INDEX idx_users_email IS 'Index for email lookups';

-- Owned sequence
CREATE SEQUENCE public.users_id_seq;
ALTER SEQUENCE public.users_id_seq OWNED BY public.users.id;
```
schema/
├── 01_tables.sql
├── 02_indexes.sql
├── 03_views.sql
├── 04_functions.sql
└── 05_sequences.sql

### Example: Materialized View File Content

Materialized view files include the view definition and its indexes:

```sql
-- schemas/public/materialized_views/sales_summary.sql

CREATE MATERIALIZED VIEW public.sales_summary AS
SELECT
DATE_TRUNC('day', created_at) AS sale_date,
COUNT(*) AS total_orders,
SUM(amount) AS total_amount
FROM public.orders
GROUP BY sale_date;

COMMENT ON MATERIALIZED VIEW public.sales_summary IS 'Daily sales aggregation';

CREATE INDEX idx_sales_summary_date ON public.sales_summary(sale_date);

COMMENT ON INDEX idx_sales_summary_date IS 'Index on sale date for fast lookups';
```

## SDL Syntax Requirements
Expand Down Expand Up @@ -163,15 +275,11 @@ SDL files support these PostgreSQL statements:
- `INSERT`, `UPDATE`, `DELETE` (use migration-based workflow)
- Transaction control (`BEGIN`, `COMMIT`)

## Complete SDL Example

```sql
-- schema/public.sql
## Complete Multi-File SDL Example

-- Sequences
CREATE SEQUENCE public.users_id_seq;
### schemas/public/tables/users.sql

-- Tables
```sql
CREATE TABLE public.users (
id INTEGER NOT NULL DEFAULT nextval('public.users_id_seq'::regclass),
username TEXT NOT NULL,
Expand All @@ -185,8 +293,21 @@ CREATE TABLE public.users (
CONSTRAINT check_status_values CHECK (status IN ('active', 'inactive', 'suspended'))
);

COMMENT ON TABLE public.users IS 'Application users';
COMMENT ON COLUMN public.users.email IS 'User email address (unique)';

CREATE INDEX idx_users_email ON public.users(email);
CREATE INDEX idx_users_created_at ON public.users(created_at);
CREATE INDEX idx_users_status ON public.users(status) WHERE status != 'inactive';

-- Owned sequence for id column
CREATE SEQUENCE public.users_id_seq;
ALTER SEQUENCE public.users_id_seq OWNED BY public.users.id;
```

### schemas/public/tables/user_profiles.sql

```sql
CREATE TABLE public.user_profiles (
user_id INTEGER NOT NULL,
bio TEXT,
Expand All @@ -195,24 +316,45 @@ CREATE TABLE public.user_profiles (
CONSTRAINT fk_user_profiles_users FOREIGN KEY (user_id) REFERENCES public.users(id) ON DELETE CASCADE
);

-- Indexes
CREATE INDEX idx_users_email ON public.users(email);
CREATE INDEX idx_users_created_at ON public.users(created_at);
CREATE INDEX idx_users_status ON public.users(status) WHERE status != 'inactive';
COMMENT ON TABLE public.user_profiles IS 'Extended user profile information';
```

-- Views
### schemas/public/views/active_users.sql

```sql
CREATE VIEW public.active_users AS
SELECT id, username, email, created_at
FROM public.users
WHERE status = 'active';

-- Functions
COMMENT ON VIEW public.active_users IS 'View of all active users';
```

### schemas/public/functions/get_user_count.sql

```sql
CREATE FUNCTION public.get_user_count()
RETURNS INTEGER
LANGUAGE sql
AS $$
SELECT COUNT(*) FROM public.users;
$$;

COMMENT ON FUNCTION public.get_user_count() IS 'Returns total count of users';
```

### schemas/public/sequences.sql

```sql
-- Independent sequences (not owned by any table column)
CREATE SEQUENCE public.custom_id_seq
START WITH 1000
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;

COMMENT ON SEQUENCE public.custom_id_seq IS 'Custom sequence for special IDs';
```

---
Expand Down