GW-Migrate

A user-friendly web application for migrating emails between Google Workspace domains.

⚠️ IMPORTANT WARNINGS ⚠️

🚨 INTERNAL DEVELOPMENT TOOL - This software is currently in active development and should be considered ALPHA/BETA quality.

🚨 USE AT YOUR OWN RISK - This tool directly manipulates email data in Google Workspace accounts. Users assume ALL RESPONSIBILITY for:

• Data loss or corruption
• Service interruptions
• Security implications
• Compliance with organizational policies
• Google API usage and quota limits

🚨 NO WARRANTIES - This software is provided "AS IS" without warranty of any kind, either express or implied.

🚨 BACKUP YOUR DATA - Always backup your emails before running any migration operations.

🚨 TEST THOROUGHLY - Test with a small subset of emails before attempting large-scale migrations.

Development Status

• ✅ Core functionality implemented
• ✅ Background processing system
• ✅ Authentication pairs management
• ⚠️ Testing and validation in progress
• ⚠️ Production hardening ongoing

Suitable For

• Internal company migrations
• Technical users comfortable with email systems
• Development and testing environments
• Users who can troubleshoot issues independently

Features

• Simple Web Interface: Easy-to-use Bootstrap UI suitable for non-technical users
• OAuth2 Authentication: Secure Google Workspace authentication for both domains
• Email Selection: Browse and select specific emails to migrate, or migrate entire inbox
• Large Inbox Support: No limits - can migrate thousands of emails from any size inbox
• Real-time Progress: Live migration progress tracking with detailed logs
• Smart Labeling: Automatically creates Gmail labels to identify migrated emails
• Email Tracking: Adds custom headers with source email IDs for easy identification
• Error Handling: Comprehensive error handling and recovery
• Migration History: Track all migration jobs and their status

Prerequisites

1. Google Cloud Project with Gmail API enabled
2. OAuth2 Credentials configured for both domains
3. Node.js (v16 or higher)
4. SQLite (automatically included)

Setup Instructions

1. Google Cloud Configuration

1. Go to Google Cloud Console
2. Create a new project or select existing one
3. Enable the Gmail API:
   • Go to APIs & Services > Library
   • Search for "Gmail API" and enable it
4. Create OAuth2 credentials:
   • Go to APIs & Services > Credentials
   • Click "Create Credentials" > "OAuth 2.0 Client IDs"
   • Set application type to "Web application"
   • Add authorized redirect URI: http://localhost:3000/auth/google/callback
   • Save the Client ID and Client Secret

2. Application Setup

1. Clone/Download this project

2. Install dependencies:

   npm install

3. Configure environment variables:

   cp .env.example .env

   Edit .env file with your values:

   GOOGLE_CLIENT_ID=your_google_client_id_here
   GOOGLE_CLIENT_SECRET=your_google_client_secret_here
   GOOGLE_REDIRECT_URI=http://localhost:3000/auth/google/callback
   SESSION_SECRET=your_random_session_secret_here
   PORT=3000
   NODE_ENV=development
   DB_PATH=./data/migrations.db

4. Start the application:

   # Development mode (with auto-restart)
   npm run dev
   
   # Production mode
   npm start

5. Access the application: Open http://localhost:3000 in your web browser

How to Use

Step 1: Authentication

1. Click "Connect Source Domain" to authenticate your source account
2. Log in with your source Google Workspace account
3. Click "Connect Destination Domain" to authenticate your destination account
4. Log in with your destination Google Workspace account

Step 2: Start Migration

1. Once both accounts are connected, click "Start New Migration"
2. The system will create a new migration job

Step 3: Select Emails

1. Browse emails: View emails from the source account with search filters
2. Individual selection: Select specific emails you want to migrate
3. Entire inbox: Click "Migrate Entire Inbox" to migrate all emails at once
4. Start migration: Click "Start Migration" to begin the process

Step 4: Monitor Progress

1. View real-time migration progress
2. See detailed logs and status updates
3. Check for any failed migrations
4. Note the Gmail label created for tracking

Step 5: Review Results

1. View migration history in the dashboard
2. Check migration statistics and labels created
3. Review any errors or issues
4. Find migrated emails in Gmail using the migration labels

Gmail Label System

The tool automatically creates Gmail labels to help you track migrated emails:

Label Structure

• Parent Label: Email Migration/
• Source Label: Email Migration/Migrated from [source-email]

Label Features

• Visual Identification: Blue color coding for easy recognition
• Source Tracking: Each label shows which source account emails came from
• Custom Headers: Adds tracking headers to emails with original message IDs
• Easy Organization: All migration labels grouped under "Email Migration" folder

Finding Migrated Emails

1. In the Inbox: All migrated emails automatically appear in your Inbox
2. By Labels: Look for labels starting with "Email Migration/"
3. Filter by Source: Click on the migration label to see all emails from that source
4. Advanced Tracking: Each email contains custom headers with original source information

Email Placement

• Inbox: All migrated emails are automatically placed in the Inbox
• Labels: Migration labels are applied in addition to Inbox placement
• Organization: You can move emails to other folders while keeping the migration labels

Technical Details

Architecture

• Backend: Node.js with Express.js
• Database: SQLite for migration tracking
• Authentication: Google OAuth2
• Frontend: Bootstrap 5 with EJS templates
• API: Gmail API v1

Database Schema

• users: Store OAuth tokens for authenticated users
• migration_jobs: Track migration job status and progress
• email_migrations: Individual email migration records
• migration_logs: Detailed operation logs

Security Features

• Secure OAuth2 flow with refresh tokens
• Session-based authentication
• Domain validation (ensures users use correct email domains)
• Input validation and sanitization
• Error logging without exposing sensitive data

API Endpoints

Authentication

• GET /auth/source - Start OAuth for source domain
• GET /auth/destination - Start OAuth for destination domain
• GET /auth/google/callback - OAuth callback handler
• GET /auth/logout - Logout user

Migration

• GET /migration/dashboard - View migration jobs
• POST /migration/start - Create new migration job
• GET /migration/select-emails/:jobId - Email selection page
• GET /migration/progress/:jobId - Migration progress page

API

• GET /api/emails/:userEmail - List emails from Gmail
• POST /api/migrate - Start email migration
• GET /api/progress/:jobId - Get migration progress

Troubleshooting

Common Issues

1. "OAuth Error" or "Invalid Credentials"

   • Check your Google Cloud Console configuration
   • Verify Client ID and Secret in .env file
   • Ensure redirect URI matches exactly

2. "Domain Mismatch" Error

   • Ensure you're using the correct email domains
   • Source: @source-domain.com, Destination: @destination-domain.com

3. "Failed to Load Emails"

   • Check Gmail API is enabled in Google Cloud Console
   • Verify user has granted necessary permissions
   • Check network connectivity

4. Migration Failures

   • Review migration logs in the progress page
   • Check quota limits in Google Cloud Console
   • Verify both accounts have necessary permissions

Debug Mode

Set NODE_ENV=development in .env file for detailed error logs.

Limitations

• Gmail API has rate limits (250 quota units per user per second)
• Large migrations may take considerable time
• Requires active internet connection
• Both Google Workspace accounts must have API access enabled

Production Deployment

For production deployment:

1. Set NODE_ENV=production in environment variables
2. Use a reverse proxy (nginx) for SSL termination
3. Use a process manager (PM2) for application monitoring
4. Configure proper backup for SQLite database
5. Set up monitoring and logging
6. Use HTTPS for all OAuth redirects

Contributing

This is an open source project. Contributions are welcome!

1. Fork the repository
2. Create a feature branch
3. Make your changes with proper testing
4. Submit a pull request with detailed description
5. Follow the existing code style and patterns

Support & Disclaimer

⚠️ Community Support Only - This is an open source project provided without professional support or warranties.

Self-Service Support:

1. Check the troubleshooting section above
2. Review application logs for detailed error information
3. Verify Google Cloud Console configuration
4. Test with a small number of emails first
5. Search existing GitHub issues
6. Create new GitHub issues for bugs or feature requests

Enterprise Support: Not available. Users are responsible for their own implementations.

License

MIT License - See LICENSE file for details.

This software is provided "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Acknowledgments

• Built with Node.js, Express, and Gmail API
• UI components from Bootstrap 5
• Background processing powered by Bull and Redis
• Authentication via Google OAuth2