v3 Migration Guide

arunavo4 · arunavo4 · commit 2140f754362d · 2025-07-17T12:18:23.000+05:30
diff --git a/MIGRATION_GUIDE.md b/MIGRATION_GUIDE.md
@@ -1,6 +1,162 @@
-# Drizzle Kit Migration Guide
+# Migration Guide
 
-This project now uses Drizzle Kit for database migrations, providing better schema management and migration tracking.
+This guide covers database migrations and version upgrades for Gitea Mirror.
+
+## Version 3.0 Migration Guide
+
+### Overview of v3 Changes
+
+Version 3.0 introduces significant security improvements and authentication changes:
+- **Token Encryption**: All GitHub and Gitea tokens are now encrypted in the database
+- **Better Auth**: Complete authentication system overhaul with session-based auth
+- **SSO/OIDC Support**: Enterprise authentication options
+- **Enhanced Security**: Improved error handling and security practices
+
+### Breaking Changes in v3
+
+#### 1. Authentication System Overhaul
+- Users now log in with **email** instead of username
+- Session-based authentication replaces JWT tokens
+- New auth endpoints: `/api/auth/[...all]` instead of `/api/auth/login`
+- Password reset may be required for existing users
+
+#### 2. Token Encryption
+- All stored GitHub and Gitea tokens are encrypted using AES-256-GCM
+- Requires encryption secret configuration
+- Existing unencrypted tokens must be migrated
+
+#### 3. Environment Variables
+**Required changes:**
+- `JWT_SECRET` → `BETTER_AUTH_SECRET` (backward compatible)
+- New: `BETTER_AUTH_URL` (required)
+- New: `ENCRYPTION_SECRET` (recommended)
+
+#### 4. Database Schema Updates
+New tables added:
+- `sessions` - User session management
+- `accounts` - Authentication accounts
+- `verification_tokens` - Email verification
+- `oauth_applications` - OAuth app registrations
+- `sso_providers` - SSO configuration
+
+### Migration Steps from v2 to v3
+
+**⚠️ IMPORTANT: Backup your database before upgrading!**
+
+```bash
+cp data/gitea-mirror.db data/gitea-mirror.db.backup
+```
+
+#### Automated Migration (Docker Compose)
+
+For Docker Compose users, v3 migration is **fully automated**:
+
+1. **Update your docker-compose.yml** to use v3:
+```yaml
+services:
+  gitea-mirror:
+    image: ghcr.io/raylabshq/gitea-mirror:v3
+```
+
+2. **Pull and restart the container**:
+```bash
+docker compose pull
+docker compose down
+docker compose up -d
+```
+
+**That's it!** The container will automatically:
+- ✅ Generate BETTER_AUTH_SECRET (from existing JWT_SECRET if available)
+- ✅ Generate ENCRYPTION_SECRET for token encryption
+- ✅ Create Better Auth database tables
+- ✅ Migrate existing users to Better Auth system
+- ✅ Encrypt all stored GitHub/Gitea tokens
+- ✅ Apply all necessary database migrations
+
+#### Manual Migration (Non-Docker)
+
+#### Step 1: Update Environment Variables
+Add to your `.env` file:
+```bash
+# Set your application URL (required)
+BETTER_AUTH_URL=http://localhost:4321  # or your production URL
+
+# Optional: These will be auto-generated if not provided
+# BETTER_AUTH_SECRET=your-existing-jwt-secret  # Will use existing JWT_SECRET
+# ENCRYPTION_SECRET=your-48-character-secret   # Will be auto-generated
+```
+
+#### Step 2: Stop the Application
+```bash
+# Stop your running instance
+pkill -f "bun run start" # or your process manager command
+```
+
+#### Step 3: Update to v3
+```bash
+# Pull latest changes
+git pull origin v3
+
+# Install dependencies
+bun install
+```
+
+#### Step 4: Run Migrations
+```bash
+# Option 1: Automatic migration on startup
+bun run build
+bun run start  # Migrations run automatically
+
+# Option 2: Manual migration
+bun run migrate:better-auth      # Migrate users to Better Auth
+bun run migrate:encrypt-tokens   # Encrypt stored tokens
+```
+
+### Post-Migration Tasks
+
+1. **All users must log in again** - Sessions are invalidated
+2. **Users log in with email** - Not username anymore
+3. **Check token encryption** - Verify GitHub/Gitea connections still work
+4. **Update API integrations** - Switch to new auth endpoints
+
+### Troubleshooting v3 Migration
+
+#### Users Can't Log In
+- Ensure they're using email, not username
+- They may need to reset password if migration failed
+- Check Better Auth migration logs
+
+#### Token Decryption Errors
+- Verify ENCRYPTION_SECRET is set correctly
+- Re-run token encryption migration
+- Users may need to re-enter tokens
+
+#### Database Errors
+- Ensure all migrations completed
+- Check disk space for new tables
+- Review migration logs in console
+
+### Rollback Procedure
+If migration fails:
+```bash
+# Stop application
+pkill -f "bun run start"
+
+# Restore database backup
+cp data/gitea-mirror.db.backup data/gitea-mirror.db
+
+# Checkout previous version
+git checkout v2.22.0
+
+# Restart with old version
+bun run start
+```
+
+---
+
+## Drizzle Kit Migration Guide
+
+This project uses Drizzle Kit for database migrations, providing better schema management and migration tracking.
 
 ## Overview
 
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -29,6 +29,9 @@ services:
       - HOST=0.0.0.0
       - PORT=4321
       - BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET:-your-secret-key-change-this-in-production}
+      - BETTER_AUTH_URL=${BETTER_AUTH_URL:-http://localhost:4321}
+      # Optional: ENCRYPTION_SECRET will be auto-generated if not provided
+      # - ENCRYPTION_SECRET=${ENCRYPTION_SECRET:-}
       # GitHub/Gitea Mirror Config
       - GITHUB_USERNAME=${GITHUB_USERNAME:-}
       - GITHUB_TOKEN=${GITHUB_TOKEN:-}
diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh
@@ -88,6 +88,32 @@ if [ "$BETTER_AUTH_SECRET" = "your-secret-key-change-this-in-production" ] || [
   echo "BETTER_AUTH_SECRET has been set to a secure random value"
 fi
 
+# Generate a secure ENCRYPTION_SECRET if one isn't provided
+ENCRYPTION_SECRET_FILE="/app/data/.encryption_secret"
+
+if [ -z "$ENCRYPTION_SECRET" ]; then
+  # Check if we have a previously generated secret
+  if [ -f "$ENCRYPTION_SECRET_FILE" ]; then
+    echo "Using previously generated ENCRYPTION_SECRET"
+    export ENCRYPTION_SECRET=$(cat "$ENCRYPTION_SECRET_FILE")
+  else
+    echo "Generating a secure random ENCRYPTION_SECRET"
+    # Generate a 48-character secret for encryption
+    if command -v openssl >/dev/null 2>&1; then
+      GENERATED_ENCRYPTION_SECRET=$(openssl rand -base64 36)
+    else
+      # Fallback to using /dev/urandom if openssl is not available
+      echo "OpenSSL not found, using fallback method for encryption secret generation"
+      GENERATED_ENCRYPTION_SECRET=$(head -c 36 /dev/urandom | base64 | tr -d '\n' | head -c 48)
+    fi
+    export ENCRYPTION_SECRET="$GENERATED_ENCRYPTION_SECRET"
+    # Save the secret to a file for persistence across container restarts
+    echo "$GENERATED_ENCRYPTION_SECRET" > "$ENCRYPTION_SECRET_FILE"
+    chmod 600 "$ENCRYPTION_SECRET_FILE"
+  fi
+  echo "ENCRYPTION_SECRET has been set to a secure random value"
+fi
+
 
 
 # Skip dependency installation entirely for pre-built images
@@ -256,6 +282,69 @@ else
   else
     echo "Warning: Could not find mirror_jobs table update script."
   fi
+
+  # Run v3 migrations if needed
+  echo "Checking for v3 migrations..."
+  
+  # Check if we need to run Better Auth migration (check if accounts table exists)
+  if ! sqlite3 /app/data/gitea-mirror.db "SELECT name FROM sqlite_master WHERE type='table' AND name='accounts';" | grep -q accounts; then
+    echo "🔄 v3 Migration: Creating Better Auth tables..."
+    # Create Better Auth tables
+    sqlite3 /app/data/gitea-mirror.db <<EOF
+    CREATE TABLE IF NOT EXISTS accounts (
+      id TEXT PRIMARY KEY,
+      userId TEXT NOT NULL,
+      accountId TEXT NOT NULL,
+      providerId TEXT NOT NULL,
+      accessToken TEXT,
+      refreshToken TEXT,
+      expiresAt INTEGER,
+      password TEXT,
+      createdAt INTEGER NOT NULL,
+      updatedAt INTEGER NOT NULL,
+      FOREIGN KEY (userId) REFERENCES users(id)
+    );
+    
+    CREATE TABLE IF NOT EXISTS sessions (
+      id TEXT PRIMARY KEY,
+      userId TEXT NOT NULL,
+      token TEXT NOT NULL,
+      expiresAt INTEGER NOT NULL,
+      createdAt INTEGER NOT NULL,
+      updatedAt INTEGER NOT NULL,
+      FOREIGN KEY (userId) REFERENCES users(id)
+    );
+    
+    CREATE TABLE IF NOT EXISTS verification_tokens (
+      id TEXT PRIMARY KEY,
+      identifier TEXT NOT NULL,
+      token TEXT NOT NULL,
+      expires INTEGER NOT NULL
+    );
+    
+    CREATE INDEX IF NOT EXISTS idx_accounts_userId ON accounts(userId);
+    CREATE INDEX IF NOT EXISTS idx_sessions_token ON sessions(token);
+    CREATE INDEX IF NOT EXISTS idx_verification_identifier_token ON verification_tokens(identifier, token);
+EOF
+  fi
+  
+  # Run Better Auth user migration
+  if [ -f "dist/scripts/migrate-better-auth.js" ]; then
+    echo "🔄 v3 Migration: Migrating users to Better Auth..."
+    bun dist/scripts/migrate-better-auth.js
+  elif [ -f "scripts/migrate-better-auth.ts" ]; then
+    echo "🔄 v3 Migration: Migrating users to Better Auth..."
+    bun scripts/migrate-better-auth.ts
+  fi
+  
+  # Run token encryption migration
+  if [ -f "dist/scripts/migrate-tokens-encryption.js" ]; then
+    echo "🔄 v3 Migration: Encrypting stored tokens..."
+    bun dist/scripts/migrate-tokens-encryption.js
+  elif [ -f "scripts/migrate-tokens-encryption.ts" ]; then
+    echo "🔄 v3 Migration: Encrypting stored tokens..."
+    bun scripts/migrate-tokens-encryption.ts
+  fi
 fi
 
 # Extract version from package.json and set as environment variable
diff --git a/scripts/migrate-better-auth.ts b/scripts/migrate-better-auth.ts
@@ -0,0 +1,76 @@
+#!/usr/bin/env bun
+import { db } from "../src/lib/db";
+import { users, accounts } from "../src/lib/db/schema";
+import { eq } from "drizzle-orm";
+import bcrypt from "bcryptjs";
+
+console.log("🔄 Starting Better Auth migration...");
+
+async function migrateToBetterAuth() {
+  try {
+    // Check if migration is needed
+    const existingAccounts = await db.select().from(accounts).limit(1);
+    if (existingAccounts.length > 0) {
+      console.log("✓ Better Auth migration already completed");
+      return;
+    }
+
+    // Get all users with password hashes
+    const allUsers = await db.select().from(users);
+    
+    if (allUsers.length === 0) {
+      console.log("ℹ️  No users to migrate");
+      return;
+    }
+
+    console.log(`📊 Found ${allUsers.length} users to migrate`);
+
+    // Migrate each user
+    for (const user of allUsers) {
+      try {
+        // Skip users without passwords (shouldn't happen but be safe)
+        if (!user.password) {
+          console.log(`⚠️  Skipping user ${user.email} - no password hash found`);
+          continue;
+        }
+
+        // Create Better Auth account entry
+        await db.insert(accounts).values({
+          id: crypto.randomUUID(),
+          userId: user.id,
+          accountId: user.email, // Use email as account ID
+          providerId: "credential", // Better Auth credential provider
+          accessToken: null,
+          refreshToken: null,
+          expiresAt: null,
+          password: user.password, // Move password hash to accounts table
+          createdAt: new Date(),
+          updatedAt: new Date()
+        });
+
+        // Remove password from users table (Better Auth manages it now)
+        await db.update(users)
+          .set({ password: null })
+          .where(eq(users.id, user.id));
+
+        console.log(`✓ Migrated user: ${user.email}`);
+      } catch (error) {
+        console.error(`❌ Failed to migrate user ${user.email}:`, error);
+        // Continue with other users even if one fails
+      }
+    }
+
+    console.log("✅ Better Auth migration completed successfully");
+    
+    // Verify migration
+    const migratedAccounts = await db.select().from(accounts);
+    console.log(`📊 Total accounts after migration: ${migratedAccounts.length}`);
+    
+  } catch (error) {
+    console.error("❌ Better Auth migration failed:", error);
+    process.exit(1);
+  }
+}
+
+// Run migration
+migrateToBetterAuth();
