Skip to content

Conversation

@productdevbook
Copy link
Owner

@productdevbook productdevbook commented Aug 18, 2025

Summary

This PR introduces an automatic database migration system for our Nitro/Nuxt application using Drizzle ORM. The system automatically runs database migrations on application startup, eliminating the need for manual migration execution in both development and production environments.

⚠️ Note: While this implementation works functionally, we're exploring better approaches for idempotent migrations. See issue #20 for research on cleaner solutions that don't require manual modification of Drizzle-generated files.

Key Features

Automatic Migration Execution: Runs migrations via Nitro plugin on startup
Environment-Aware: Different behavior for development vs production
Silent Mode: Reduces console noise in development environment
Idempotent Migrations: Safe to run multiple times without errors
PostgreSQL Notice Suppression: Clean output without unnecessary NOTICE messages
Production Build Support: Automatically copies migration files to build output

Technical Implementation

1. Nitro Plugin (server/plugins/01.database-migrate.ts)

  • Automatically executes on application startup
  • Environment detection for development vs production paths
  • Silent mode in development to reduce console clutter
  • PostgreSQL connection with onnotice handler for suppressing notices
  • Configurable via AUTO_MIGRATE environment variable (defaults to enabled)

2. Build Integration (nuxt.config.ts)

  • Nitro compiled hook to copy migration files to .output/server/migrations
  • Ensures migration files are available in production builds
  • Recursive directory copying with proper error handling

3. Idempotent Migration SQL

  • Custom Migration Modifications: All migration files have been modified to handle existing database objects
  • Enum types wrapped in DO $$ ... IF NOT EXISTS blocks
  • Table creation uses CREATE TABLE IF NOT EXISTS
  • Foreign key constraints check for existence before adding
  • Prevents "already exists" errors when running migrations multiple times

🔍 Research Needed: The manual modification approach works but isn't ideal. We're seeking better patterns - see #20 for discussion on cleaner approaches.

Environment Variables

  • AUTO_MIGRATE=false - Disable automatic migrations (default: enabled)
  • DATABASE_URL - PostgreSQL connection string (required)

Path Resolution

Environment Migration Path Description
Development ./server/database/migrations Relative from project root
Production ../migrations Relative from plugin location in build

Migration Behavior

  • Development: Silent mode enabled, minimal console output
  • Production: Normal logging for deployment visibility
  • Error Handling: Detailed error reporting with PostgreSQL error codes
  • Connection Management: Proper connection cleanup after migration

Known Limitations & Future Improvements

This current approach requires manual modification of Drizzle-generated migration files to make them idempotent. While functional, we acknowledge this isn't the cleanest solution. We're actively researching better approaches in #20.

Alternative approaches being considered:

  1. Proper migration state tracking instead of forced idempotency
  2. Custom Drizzle configuration for automatic idempotent SQL generation
  3. Database state comparison and differential migration application
  4. Enhanced migration lock/state management mechanisms

Feedback Requested

🔍 Community Input Welcome: Please share your thoughts on this auto-migration approach:

  1. Developer Experience: Does this improve or complicate your development workflow?
  2. Production Safety: Are there concerns about automatic migrations in production?
  3. Configuration: Should we add more configuration options (e.g., migration timeout, retry logic)?
  4. Docker/Container Support: How does this work in your containerized environments?
  5. Migration Strategy: Would you prefer opt-in vs opt-out behavior?
  6. Error Recovery: What should happen when migrations fail in production?
  7. Better Patterns: Do you know cleaner approaches for this problem? (Also discuss in Research: Better approach for idempotent migrations in auto-migration system #20)

Testing

  • Tested in development mode with pnpm dev
  • Tested in production mode with pnpm build && pnpm preview
  • Verified idempotent behavior by running migrations multiple times
  • Confirmed migration files are copied to build output
  • Tested PostgreSQL notice suppression
  • Verified path resolution in both environments

Migration Impact

This change consolidates multiple migration files into a single idempotent migration (0000_jazzy_jackpot.sql) that includes all schema changes with proper existence checks. The migration history has been reset to provide a clean foundation for the auto-migration system.

Breaking Changes

None. This is an additive feature that can be disabled via environment variable.

Related Issues

This commit introduces an automatic database migration system that runs on application startup, eliminating the need for manual migration execution.

Key Features:
- Automatic migration execution via Nitro plugin
- Environment-aware path resolution for dev/production
- Silent mode in development to reduce noise
- Idempotent migrations that handle "already exists" errors
- PostgreSQL notice suppression for cleaner output
- Build-time migration file copying for production deployments

Technical Implementation:
- Added server/plugins/01.database-migrate.ts Nitro plugin
- Integrated Nitro build hook to copy migration files to .output
- Modified migration SQL files to use IF NOT EXISTS patterns
- Added AUTO_MIGRATE environment variable for control (default: enabled)

The plugin automatically detects development vs production environments and adjusts paths accordingly. In development, it runs silently to avoid console clutter. All migration operations are idempotent, preventing errors when running multiple times.

Custom migration modifications were made to handle PostgreSQL enum types, table creation, and foreign key constraints with proper existence checks.
@productdevbook
Copy link
Owner Author

Replaced by #27 with a better approach using Drizzle's built-in migration tracking instead of manual IF NOT EXISTS modifications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants