Skip to content

[FEATURE] Implement Sequelize's migration system to manage database schema changesΒ #107

@nanotaboada

Description

@nanotaboada

Problem

The application currently uses a pre-seeded SQLite database file (storage/players-sqlite3.db) that is copied into the Docker container and mounted as a persistent volume. While this approach works for a Proof of Concept, it presents several challenges for production-ready development:

  1. Manual Schema Changes: Any database schema modification requires manually editing the SQLite file or recreating it from scratch, which is error-prone and not version-controlled.
  2. No Version Control for Schema: The database structure is embedded in a binary file (.db), making it impossible to track schema changes through git history.
  3. Team Collaboration Issues: Multiple developers cannot easily sync database schema changes. Each dev must manually replicate schema modifications.
  4. No Rollback Capability: There's no built-in way to revert schema changes if an update causes issues.
  5. Deployment Complexity: Applying schema updates to existing production databases requires custom scripts and manual intervention.
  6. Testing Challenges: Integration tests rely on the pre-seeded database state, making it harder to test schema changes independently.

Current Approach:

  • Pre-seeded storage/players-sqlite3.db file committed to repository
  • Docker entrypoint copies database from /app/hold/ to persistent volume /storage/ on first run
  • Schema defined implicitly in Sequelize model (src/models/player-model.ts)
  • No automated way to create, update, or rollback schema changes

Problems for Future Development:

  • Adding new columns/tables requires manual SQL or full DB recreation
  • No audit trail for schema evolution
  • Cannot easily support multiple environments (dev, staging, prod) with different schema states
  • Difficult to coordinate schema changes with code deployments

Proposed Solution

Implement Sequelize's migration system to manage database schema changes in a version-controlled, automated, and reversible manner. Migrations provide:

  1. Version-Controlled Schema: Each migration is a timestamped file in git, providing full schema history
  2. Automated Application: Run npx sequelize-cli db:migrate to apply pending migrations
  3. Rollback Support: Use db:migrate:undo to revert changes safely
  4. Team Synchronization: Developers pull migration files and run migrations locally to sync schemas
  5. CI/CD Integration: Migrations can run automatically as part of deployment pipelines
  6. Environment Parity: Ensure dev, staging, and prod databases stay in sync

Benefits:

  • βœ… Track schema changes in git with commit messages
  • βœ… Apply schema updates consistently across environments
  • βœ… Rollback problematic changes without data loss
  • βœ… Generate initial schema from existing models
  • βœ… Support future features (adding columns, indexes, foreign keys)
  • βœ… Enable database seeding as separate step from schema creation

Migration Workflow:

  1. Developer creates migration: npx sequelize-cli migration:generate --name add-player-nationality
  2. Developer writes up() and down() functions in migration file
  3. Migration committed to git and pushed
  4. Other developers pull and run: npx sequelize-cli db:migrate
  5. Production deployment runs migrations automatically

Suggested Approach

1. Install Sequelize CLI and Dependencies

npm install --save-dev sequelize-cli
npm install --save-dev @types/node

Note: sequelize is already installed as a dependency.

2. Create .sequelizerc Configuration File

File: .sequelizerc (root directory)

This tells Sequelize CLI where to find migrations, models, and config files:

const path = require('path');

module.exports = {
    'config': path.resolve('config', 'database.json'),
    'models-path': path.resolve('src', 'models'),
    'seeders-path': path.resolve('database', 'seeders'),
    'migrations-path': path.resolve('database', 'migrations'),
};

Directory Structure:

database/
β”œβ”€β”€ migrations/        # Migration files (timestamped)
└── seeders/          # Seed data files (optional)
config/
└── database.json     # Database configuration for each environment

3. Create Database Configuration File

File: config/database.json

{
  "development": {
    "dialect": "sqlite",
    "storage": "./storage/players-sqlite3.db",
    "logging": console.log
  },
  "test": {
    "dialect": "sqlite",
    "storage": ":memory:",
    "logging": false
  },
  "production": {
    "dialect": "sqlite",
    "storage": "/storage/players-sqlite3.db",
    "logging": false
  }
}

Explanation:

  • development: Local SQLite file in storage/ directory
  • test: In-memory database for fast test execution
  • production: Docker volume mount path (/storage/ in container)

Alternative: Use TypeScript config file (config/database.ts) and export dynamic configuration:

import path from 'node:path';

const storagePath = process.env.STORAGE_PATH ?? path.join(process.cwd(), 'storage', 'players-sqlite3.db');

export default {
    development: {
        dialect: 'sqlite',
        storage: storagePath,
        logging: console.log,
    },
    // ... other environments
};

4. Generate Initial Migration from Existing Schema

Create a migration that reflects the current Player model schema:

npx sequelize-cli migration:generate --name create-players-table

File: database/migrations/YYYYMMDDHHMMSS-create-players-table.js

'use strict';

module.exports = {
    async up(queryInterface, Sequelize) {
        await queryInterface.createTable('players', {
            id: {
                type: Sequelize.INTEGER,
                autoIncrement: true,
                primaryKey: true,
                allowNull: false,
            },
            firstName: {
                type: Sequelize.STRING,
                allowNull: false,
            },
            middleName: {
                type: Sequelize.STRING,
                allowNull: true,
            },
            lastName: {
                type: Sequelize.STRING,
                allowNull: false,
            },
            dateOfBirth: {
                type: Sequelize.DATE,
                allowNull: true,
            },
            squadNumber: {
                type: Sequelize.INTEGER,
                allowNull: false,
                unique: true,
            },
            position: {
                type: Sequelize.STRING,
                allowNull: false,
            },
            abbrPosition: {
                type: Sequelize.STRING,
                allowNull: true,
            },
            team: {
                type: Sequelize.STRING,
                allowNull: true,
            },
            league: {
                type: Sequelize.STRING,
                allowNull: true,
            },
            starting11: {
                type: Sequelize.BOOLEAN,
                allowNull: true,
            },
        });

        // Add index on squadNumber for faster lookups
        await queryInterface.addIndex('players', ['squadNumber'], {
            unique: true,
            name: 'players_squad_number_unique',
        });
    },

    async down(queryInterface, Sequelize) {
        await queryInterface.dropTable('players');
    },
};

Key Points:

  • up() creates the table and indexes
  • down() reverts changes (drops table)
  • Sequelize CLI tracks which migrations have run in a SequelizeMeta table

5. Create Seed Data Migration (Optional)

Generate a seeder to populate initial player data:

npx sequelize-cli seed:generate --name seed-initial-players

File: database/seeders/YYYYMMDDHHMMSS-seed-initial-players.js

'use strict';

module.exports = {
    async up(queryInterface, Sequelize) {
        await queryInterface.bulkInsert('players', [
            {
                id: 1,
                firstName: 'Emiliano',
                middleName: 'Viviano',
                lastName: 'MartΓ­nez',
                dateOfBirth: '1992-08-02',
                squadNumber: 23,
                position: 'Goalkeeper',
                abbrPosition: 'GK',
                team: 'Aston Villa',
                league: 'Premier League',
                starting11: true,
            },
            // ... more players
        ], {});
    },

    async down(queryInterface, Sequelize) {
        await queryInterface.bulkDelete('players', null, {});
    },
};

Run Seeds:

npx sequelize-cli db:seed:all

6. Update npm Scripts

Add migration commands to package.json:

"scripts": {
    "db:migrate": "sequelize-cli db:migrate",
    "db:migrate:undo": "sequelize-cli db:migrate:undo",
    "db:migrate:undo:all": "sequelize-cli db:migrate:undo:all",
    "db:seed": "sequelize-cli db:seed:all",
    "db:seed:undo": "sequelize-cli db:seed:undo:all",
    "db:create": "sequelize-cli db:create",
    "db:drop": "sequelize-cli db:drop",
    "db:reset": "npm run db:migrate:undo:all && npm run db:migrate && npm run db:seed"
}

Usage:

  • npm run db:migrate - Apply pending migrations
  • npm run db:migrate:undo - Revert last migration
  • npm run db:seed - Populate seed data
  • npm run db:reset - Reset and rebuild database

7. Update Docker Entrypoint Script

Modify scripts/entrypoint.sh to run migrations instead of copying pre-seeded DB:

#!/bin/sh
set -e

echo "βœ” Executing entrypoint script..."

VOLUME_STORAGE_PATH="/storage/players-sqlite3.db"

echo "βœ” Starting container..."

# Check if database exists, create if not
if [ ! -f "$VOLUME_STORAGE_PATH" ]; then
  echo "⚠️ No existing database file found in volume."
  echo "Running migrations to create schema..."
  NODE_ENV=production npx sequelize-cli db:migrate
  echo "βœ” Database initialized at $VOLUME_STORAGE_PATH"
else
  echo "βœ” Existing database file found."
  echo "Running pending migrations..."
  NODE_ENV=production npx sequelize-cli db:migrate
fi

echo "βœ” Ready!"
echo "πŸš€ Launching app..."
exec "$@"

Key Changes:

  • Remove pre-seeded database copy logic
  • Run db:migrate on every container start (safe: applies only pending migrations)
  • Use NODE_ENV=production to use correct database config

8. Update Dockerfile to Include Migration Files

Modify Dockerfile to copy migration files and CLI:

# Stage 1: Builder
FROM node:krypton-alpine AS builder

WORKDIR /app

COPY package.json package-lock.json tsconfig.json ./
RUN npm ci

COPY src/ ./src/
COPY database/ ./database/        # Add migrations and seeders
COPY config/ ./config/            # Add database config
COPY .sequelizerc ./              # Add Sequelize CLI config

RUN npm run build && \
    npm run swagger:docs && \
    npm prune --omit=dev

# Stage 2: Runtime
FROM node:krypton-alpine AS runtime

# ... existing runtime setup ...

# Copy migrations and config for runtime execution
COPY --from=builder     /app/database/              ./database/
COPY --from=builder     /app/config/                ./config/
COPY --from=builder     /app/.sequelizerc           ./.sequelizerc

# ... rest of Dockerfile ...

Important: Keep sequelize-cli as a regular dependency (not dev-only) or install it globally in the runtime image:

RUN npm install -g sequelize-cli

9. Remove Pre-Seeded Database from Repository

After migration setup is complete and tested:

# Remove pre-seeded database (keep directory)
git rm storage/players-sqlite3.db
echo "players-sqlite3.db" >> storage/.gitignore
git add storage/.gitignore
git commit -m "chore: migrate to Sequelize migrations, remove pre-seeded DB"

Note: Keep storage/ directory with a .gitkeep file for local development:

touch storage/.gitkeep
git add storage/.gitkeep

10. Update Documentation

File: README.md

Add section on database setup:

## Database Setup

This project uses Sequelize migrations to manage the database schema.

### Initial Setup (First Time)

```bash
# Run migrations to create schema
npm run db:migrate

# (Optional) Seed initial data
npm run db:seed

Creating New Migrations

# Generate migration file
npx sequelize-cli migration:generate --name add-nationality-column

# Edit migration file in database/migrations/
# Run migration
npm run db:migrate

Rollback Migrations

# Undo last migration
npm run db:migrate:undo

# Undo all migrations
npm run db:migrate:undo:all

Docker

Migrations run automatically on container startup via entrypoint.sh.

Acceptance Criteria

  1. βœ… Sequelize CLI Installed: sequelize-cli and @types/node installed as dev dependencies
  2. βœ… Configuration Files Created: .sequelizerc and config/database.json exist with correct paths
  3. βœ… Initial Migration Created: Migration file exists in database/migrations/ that creates players table matching current schema
  4. βœ… Migration Scripts Added: npm scripts (db:migrate, db:migrate:undo, db:seed) added to package.json
  5. βœ… Migrations Run Successfully: npm run db:migrate creates database from scratch without errors
  6. βœ… Rollback Works: npm run db:migrate:undo successfully reverts the last migration
  7. βœ… Docker Integration: Dockerfile and entrypoint script updated to run migrations on container startup
  8. βœ… Local Development Works: Developers can clone repo, run npm run db:migrate, and start app without manual DB setup
  9. βœ… CI/CD Compatible: Migrations run successfully in GitHub Actions workflow
  10. βœ… Documentation Updated: README.md includes clear instructions for migration usage
  11. βœ… Pre-Seeded DB Removed: storage/players-sqlite3.db removed from git (with proper .gitignore)
  12. βœ… Seed Data Optional: Seed data extracted to separate seeder file (optional step)

References

Sequelize Documentation

Related Issues

Migration Best Practices

TypeScript Integration

Testing with Migrations

  • Integration tests can use :memory: database with migrations applied before each test suite
  • Ensures tests always run against correct schema
  • Example pattern:
    beforeAll(async () => {
        await sequelize.sync({ force: true }); // Or run migrations
    });

CI/CD Integration Examples

  • Run migrations in GitHub Actions before running tests

  • Use separate database configs for test environment

  • Example workflow step:

    - name: Run Database Migrations
      run: npm run db:migrate
      env:
        NODE_ENV: test

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestplanningEnables automatic issue planning with CodeRabbitpriority highImportant for production readiness. Schedule for current milestone.typescriptPull requests that update TypeScript code

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions