Skip to content
This repository was archived by the owner on Sep 4, 2025. It is now read-only.
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .datarc.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@
"theme": "mountain",
"reporter": "ink"
}
}
}
20 changes: 10 additions & 10 deletions .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,33 +19,33 @@
"@typescript-eslint/no-floating-promises": "error",
"@typescript-eslint/no-misused-promises": "error",
"@typescript-eslint/await-thenable": "error",

// Promise plugin rules
"promise/catch-or-return": "error",
"promise/no-return-wrap": "error",
"promise/param-names": "error",
"promise/always-return": "error",

// Require await in async functions
"require-await": "error",

// Additional async/await rules
"no-async-promise-executor": "error",
"no-await-in-loop": "warn",
"no-return-await": "error",
"prefer-promise-reject-errors": "error",

// General best practices
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": ["error", { "argsIgnorePattern": "^_" }],
"@typescript-eslint/no-unused-vars": [
"error",
{ "argsIgnorePattern": "^_" }
],
"no-console": "off",
"semi": ["error", "always"],
"quotes": ["error", "single", { "avoidEscape": true }]
},
"plugins": [
"@typescript-eslint",
"promise"
],
"plugins": ["@typescript-eslint", "promise"],
"overrides": [
{
"files": ["*.js"],
Expand All @@ -55,4 +55,4 @@
}
}
]
}
}
36 changes: 29 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ D.A.T.A. (Database Automation, Testing, and Alignment) is a CLI tool for managin
## Common Development Commands

### Running Tests

```bash
# Run all Vitest tests
npm test
Expand All @@ -29,6 +30,7 @@ npm run test:coverage
```

### Database Migration Workflow

```bash
# 1. Generate migration from SQL changes
npm run migrate:generate # or: data db migrate generate --name <name>
Expand All @@ -47,6 +49,7 @@ npm run migrate:rollback # or: data db migrate rollback --prod
```

### Building and Compiling

```bash
# Compile SQL sources into migration
data db compile
Expand All @@ -58,24 +61,28 @@ data db compile --deploy-functions
## Architecture

### Code Organization Rules

- **One Class Per File**: Each file must contain exactly one class. The filename must match the class name.
- **Self-Documenting Names**: Each artifact should describe its contents based on its filename.
- **No Multi-Class Files**: If a file contains multiple classes, it must be refactored immediately.

### Command Class Hierarchy

- **Command** (src/lib/Command.js): Base class with event emission and logging
- **SupabaseCommand**: Commands using Supabase API
- **DatabaseCommand**: Direct database access commands
- **TestCommand**: Testing-related commands

All commands follow an event-driven pattern:

```javascript
command.emit('progress', { message: 'Processing...' });
command.emit('success', { message: 'Complete!' });
command.emit('error', { message: 'Failed', error });
command.emit("progress", { message: "Processing..." });
command.emit("success", { message: "Complete!" });
command.emit("error", { message: "Failed", error });
```

### Directory Structure

- **src/commands/**: Command implementations organized by domain (db/, functions/, test/)
- **src/lib/**: Core libraries and base classes
- **src/reporters/**: Output formatters (CliReporter)
Expand All @@ -85,13 +92,17 @@ command.emit('error', { message: 'Failed', error });
- **functions/**: Supabase Edge Functions

### Path Configuration

Paths can be configured via:

1. Command-line options: `--sql-dir`, `--tests-dir`, `--migrations-dir`
2. Environment variables: `data_SQL_DIR`, `data_TESTS_DIR`, `data_MIGRATIONS_DIR`
3. Configuration file: `.datarc.json`

### Configuration System

Configuration is loaded from `.datarc.json` with the following structure:

```json
{
"test": {
Expand All @@ -110,19 +121,22 @@ Configuration is loaded from `.datarc.json` with the following structure:
## Important Patterns

### Production Safety

- All production commands require explicit `--prod` flag
- Destructive operations require typed confirmation
- Commands wrap operations in transactions where supported
- Process management includes zombie prevention and cleanup

### Error Handling

- Custom error types in `src/lib/dataError/`
- Commands should emit error events before throwing
- Process exit codes are handled by CliReporter

### Testing Strategy
- Unit tests use Vitest (test/*.test.js)
- Database tests use pgTAP (tests/*.sql)

- Unit tests use Vitest (test/\*.test.js)
- Database tests use pgTAP (tests/\*.sql)
- Test commands support multiple output formats (console, JUnit, JSON)
- Coverage enforcement configurable via .datarc.json

Expand Down Expand Up @@ -151,26 +165,31 @@ data_MIGRATIONS_DIR=./migrations
## Development Notes

### Adding New Commands

1. Extend appropriate base class (Command, DatabaseCommand, etc.)
2. Implement `performExecute()` method
3. Emit appropriate events for progress tracking
4. Register in src/index.js with commander

### Working with Migrations

- Migrations include metadata.json with tracking info
- Use MigrationMetadata class for parsing/validation
- Test migrations run in isolated schemas (@data.tests.*)
- Test migrations run in isolated schemas (@data.tests.\*)
- Production migrations require double confirmation

### Edge Functions Integration

- Functions can be deployed with migrations via `--deploy-functions`
- Validation happens before deployment
- Production deployments require import maps unless `--skip-import-map`

## Troubleshooting

### Compile Command Issues

If `data db compile` exits with no error output:

- Ensure SQL source directory exists (default: ./sql)
- Use `--sql-dir` and `--migrations-dir` to specify custom paths
- The compile command now properly displays errors for missing directories
Expand All @@ -192,12 +211,14 @@ npm run postinstall # Manually re-install git hooks if needed
```

#### Git Pre-commit Hook

- Automatically runs ESLint on staged JavaScript files
- Prevents commits with linting errors
- Checks for floating promises and async issues
- Bypass with `git commit --no-verify` (use sparingly!)

#### ESLint Rules Enforced

- `require-await`: Async functions must use await
- `promise/catch-or-return`: Promises must be handled
- `promise/always-return`: Promise chains must return values
Expand All @@ -206,6 +227,7 @@ npm run postinstall # Manually re-install git hooks if needed
For TypeScript projects, use `@typescript-eslint/no-floating-promises` to catch unawaited async calls.

### Recent Fixes

- Fixed error handling in CompileCommand constructor to properly display errors
- Added `isProd` property to start event emissions
- Fixed MigrationCompiler config property naming (sqlDir vs rootDir)
- Fixed MigrationCompiler config property naming (sqlDir vs rootDir)
78 changes: 39 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ DATA generates deterministic migrations from your SQL source files and blocks un
```

> [!warning]- **Work in Progress**
> This project is actively in pre-release, ***use at your own risk!***
> This project is actively in pre-release, **_use at your own risk!_**
> **See** [/issues/README.md](the issues README) to get a sense of current progress.

## How It Works
Expand Down Expand Up @@ -63,10 +63,10 @@ $ data align production
# 2. Organize your SQL in named directories
/sql/
extensions/ # PostgreSQL extensions
tables/ # Table definitions
tables/ # Table definitions
policies/ # RLS policies
# etc.

# 3. The DATA workflow
data automate # Generate migration plan
data test # Run your test suite
Expand All @@ -80,30 +80,30 @@ data align production --confirm # Deploy to prod (with confirmation)
DATA **blocks production deployments** unless:

- ✅ **Clean repo** (no uncommitted/untracked files)
- ✅ **Up-to-date** (not behind origin/main)
- ✅ **Up-to-date** (not behind origin/main)
- ✅ **Correct branch** (main, configurable)
- ✅ **Tests passing** (100% required, configurable)

Large changes require typing confirmation. *"Proceeding without corrections would be... illogical."*
Large changes require typing confirmation. _"Proceeding without corrections would be... illogical."_

## Core Commands

| Command | Purpose |
|---------|---------|
| `data status` | Show current state vs environments |
| `data automate` | Generate migration plan from SQL |
| `data test` | Run test suite (required for prod) |
| `data promote` | Tag the tested release |
| `data align <env>` | Deploy to environment (🔐 **gated**) |
| `data rollback --to-tag <tag>` | Revert to any previous tag |
| `data analyze` | Detect drift between repo and DB |
| Command | Purpose |
| ------------------------------ | ------------------------------------ |
| `data status` | Show current state vs environments |
| `data automate` | Generate migration plan from SQL |
| `data test` | Run test suite (required for prod) |
| `data promote` | Tag the tested release |
| `data align <env>` | Deploy to environment (🔐 **gated**) |
| `data rollback --to-tag <tag>` | Revert to any previous tag |
| `data analyze` | Detect drift between repo and DB |

## Git-First Deployments

Every deployment creates an immutable git tag. Rollbacks are exact and boring:

```bash
# Deploy creates tags automatically
# Deploy creates tags automatically
data align production
# → Creates: data/prod/2025.241.1430

Expand All @@ -118,13 +118,13 @@ data status

## Why DATA vs Others?

| Feature | DATA | Flyway | Liquibase | Supabase CLI |
|---------|------|--------|-----------|--------------|
| **Golden SQL** | ✅ Git-native | ❌ Hand-written migrations | ❌ Changelog format | ❌ Hand-written |
| **Deterministic** | ✅ Pure git diff | ⚠️ DB introspection | ⚠️ DB introspection | ❌ Manual |
| **Production gates** | ✅ Non-negotiable | ⚠️ Optional | ⚠️ Optional | ❌ None |
| **Rollback** | ✅ Tag-based | ⚠️ Down scripts | ⚠️ Manual tags | ❌ Manual |
| **Personality** | 🖖 Lt. Commander Data | 😐 | 😐 | 😐 |
| Feature | DATA | Flyway | Liquibase | Supabase CLI |
| -------------------- | --------------------- | -------------------------- | ------------------- | --------------- |
| **Golden SQL** | ✅ Git-native | ❌ Hand-written migrations | ❌ Changelog format | ❌ Hand-written |
| **Deterministic** | ✅ Pure git diff | ⚠️ DB introspection | ⚠️ DB introspection | ❌ Manual |
| **Production gates** | ✅ Non-negotiable | ⚠️ Optional | ⚠️ Optional | ❌ None |
| **Rollback** | ✅ Tag-based | ⚠️ Down scripts | ⚠️ Manual tags | ❌ Manual |
| **Personality** | 🖖 Lt. Commander Data | 😐 | 😐 | 😐 |

## Example: Safety Gates in Action

Expand All @@ -134,26 +134,26 @@ $ data align production

🔴 RED ALERT: Working directory not clean
Modified: sql/tables/users.sql

Commander, your working directory contains uncommitted changes.
Probability of catastrophic failure: 87.3%

Recommended action: git commit or git stash

# After fixing
$ data align production

✅ All safety checks passed
- Repository: clean ✅
- Branch: main (approved) ✅
- Branch: main (approved) ✅
- Tests: 147/147 passing (100%) ✅

Migration preview:
+ CREATE TABLE crew_evaluations
+ ALTER TABLE users ADD COLUMN shore_leave_balance

Type 'ENGAGE' to proceed: ENGAGE

Deployment successful. "Make it so" achieved.
```

Expand All @@ -168,7 +168,7 @@ The `/sql/` directory is where your "golden sql" files should live. You must sor
```
/sql/
extensions/ # PostgreSQL extensions
tables/ # Tables and relationships
tables/ # Tables and relationships
functions/ # Stored procedures
policies/ # RLS policies
indexes/ # Performance indexes
Expand All @@ -191,7 +191,7 @@ The `/sql/` directory is where your "golden sql" files should live. You must sor
"minimum_coverage": 95,
"enforce": true
},
"personality": "android" // android | quiet | tng
"personality": "android" // android | quiet | tng
}
```

Expand All @@ -204,8 +204,8 @@ The `/sql/` directory is where your "golden sql" files should live. You must sor
data automate
data test
data promote
- name: Deploy to Production

- name: Deploy to Production
if: github.ref == 'refs/heads/main'
run: data align production --confirm
```
Expand All @@ -230,25 +230,25 @@ npm install -g @starfleet/data

## Troubleshooting

| Problem | Solution |
|---------|----------|
| "Working directory not clean" | `git commit` or `git stash` |
| "Behind origin/main" | `git pull origin main` |
| "Tests failing" | Fix tests, DATA won't deploy broken code |
| Problem | Solution |
| ----------------------------- | ---------------------------------------- |
| "Working directory not clean" | `git commit` or `git stash` |
| "Behind origin/main" | `git pull origin main` |
| "Tests failing" | Fix tests, DATA won't deploy broken code |

## The Philosophy

**Golden SQL is truth. Git is memory. Tests are trust.**

DATA enforces bulletproof deployments through non-negotiable safety gates. This isn't about restricting developers—it's about giving them confidence. When DATA approves your deployment, you can sleep soundly.

*"In my observations of human behavior, I have noticed that engineers sleep better when their deployments cannot accidentally destroy everything."* — Lt. Commander Data
_"In my observations of human behavior, I have noticed that engineers sleep better when their deployments cannot accidentally destroy everything."_ — Lt. Commander Data

---

**Live long and prosper.** 🖖

*"Spot has been fed. Database operations may proceed."*
_"Spot has been fed. Database operations may proceed."_

## Contributing

Expand Down
Loading