🐍 Advanced Snake Game AI - Complete Documentation

📋 Table of Contents

• Quick Start
• Google Colab Setup
• Local Development Setup
• Command Reference
• Configuration Guide
• Training Examples
• Model Testing
• Advanced Usage
• Troubleshooting

---

🚀 Quick Start

For Impatient Users (5 seconds to AI Snake!)

# Run this single line for instant demo
python snake_ai.py

Google Colab (One-Click Setup)

# In Colab cell 1:
!wget https://raw.githubusercontent.com/your-repo/snake_ai.py
exec(open('snake_ai.py').read())

# In Colab cell 2:
quick_train_demo(difficulty=GameDifficulty.MEDIUM, render=False, max_episodes=500)

---

🔧 Google Colab Setup

Method 1: Automatic Setup (Recommended)

# Cell 1: Auto-setup and run
import subprocess
import sys

# Install required packages
packages = [
    'pygame==2.5.2', 'torch', 'matplotlib>=3.7.0', 
    'seaborn>=0.12.0', 'tensorboard', 'pyyaml', 'pandas'
]

for package in packages:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', package])

# Setup virtual display for rendering
!apt-get update -qq
!apt-get install -y -qq xvfb
import os
os.environ['DISPLAY'] = ':99'
!Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &

print("✅ Setup complete!")

# Cell 2: Load and run the AI
# Paste the entire snake_ai.py code here, then run:
quick_train_demo(difficulty=GameDifficulty.MEDIUM, render=False)

Method 2: Manual Cell-by-Cell Setup

# Cell 1: Install dependencies
!pip install pygame torch matplotlib seaborn tensorboard pyyaml pandas numpy

# Cell 2: Setup display (for rendering)
!apt-get update -qq > /dev/null 2>&1
!apt-get install -y -qq xvfb > /dev/null 2>&1
import os
os.environ['DISPLAY'] = ':99'
!Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &

# Cell 3: Import and initialize
# Paste the snake_ai.py code here

# Cell 4: Quick demo
trainer, results, test_results = quick_train_demo(
    difficulty=GameDifficulty.EASY,
    render=False,  # Set to True if you want to see the game
    max_episodes=200
)

Method 3: From GitHub (Future)

# Cell 1: Clone and setup
!git clone https://github.com/your-username/advanced-snake-ai.git
%cd advanced-snake-ai
!pip install -r requirements.txt
!python setup_colab.py

# Cell 2: Run
!python snake_ai.py --mode demo --difficulty medium --episodes 500

---

💻 Local Development Setup

Prerequisites

• Python 3.8+
• pip or conda

Installation Steps

Option 1: Quick Install

# Clone repository (when available)
git clone https://github.com/your-username/advanced-snake-ai.git
cd advanced-snake-ai

# Install dependencies
pip install -r requirements.txt

# Run demo
python snake_ai.py

Option 2: Manual Setup

# Create virtual environment (recommended)
python -m venv snake_ai_env

# Activate environment
# Windows:
snake_ai_env\Scripts\activate
# macOS/Linux:
source snake_ai_env/bin/activate

# Install packages
pip install pygame torch torchvision matplotlib seaborn tensorboard pyyaml pandas numpy

# Save the snake_ai.py file and run
python snake_ai.py

Option 3: Conda Setup

# Create conda environment
conda create -n snake_ai python=3.9
conda activate snake_ai

# Install PyTorch (choose your platform)
conda install pytorch torchvision torchaudio -c pytorch

# Install other packages
pip install pygame matplotlib seaborn tensorboard pyyaml pandas

# Run
python snake_ai.py

---

📚 Command Reference

Basic Commands

Demo Mode (Default)

# Quick demo with default settings
python snake_ai.py

# Demo with specific difficulty
python snake_ai.py --mode demo --difficulty hard --episodes 1000

# Demo with rendering (see the game)
python snake_ai.py --mode demo --difficulty medium --render --episodes 500

Training Mode

# Full training session
python snake_ai.py --mode train --difficulty medium --episodes 10000

# Training with custom name
python snake_ai.py --mode train --experiment-name "my_experiment" --episodes 5000

# Training with rendering (slower but fun to watch)
python snake_ai.py --mode train --render --episodes 2000

Testing Mode

# Test a trained model
python snake_ai.py --mode test --model-path "models/best_model.pth"

# Test with specific difficulty
python snake_ai.py --mode test --model-path "models/best_model.pth" --difficulty hard

Advanced Commands

Environment-Specific

# For Google Colab
python snake_ai.py --colab --mode demo --episodes 300

# With specific experiment name
python snake_ai.py --experiment-name "colab_test_$(date +%s)" --episodes 1000

Difficulty Levels

# Easy (slow speed, no obstacles)
python snake_ai.py --difficulty easy --episodes 2000

# Medium (normal speed, no obstacles) - Default
python snake_ai.py --difficulty medium --episodes 5000

# Hard (fast speed, 3 obstacles)
python snake_ai.py --difficulty hard --episodes 10000

# Extreme (very fast, 5 obstacles, advanced AI)
python snake_ai.py --difficulty extreme --episodes 20000

Complete Command Options

python snake_ai.py \
  --mode [train|test|demo] \
  --difficulty [easy|medium|hard|extreme] \
  --episodes NUMBER \
  --render \
  --model-path PATH \
  --experiment-name NAME \
  --colab

---

⚙️ Configuration Guide

Programmatic Configuration

Game Configuration

from dataclasses import asdict

# Custom game setup
game_config = GameConfig(
    width=800,              # Game window width
    height=600,             # Game window height
    block_size=20,          # Size of each game block
    speed=15,               # Game speed (FPS)
    enable_obstacles=True,  # Add obstacles
    num_obstacles=3,        # Number of obstacles
    food_value=1,           # Points per food
    max_steps_multiplier=100, # Timeout = steps * snake_length
    colors={
        'background': (20, 20, 30),
        'snake_head': (100, 255, 100),
        'snake_body': (50, 200, 50),
        'food': (255, 100, 100),
        # ... more colors
    }
)

Training Configuration

# Advanced training setup
training_config = TrainingConfig(
    # Network Architecture
    network_type=NetworkType.RAINBOW_DQN,  # DQN, DOUBLE_DQN, DUELING_DQN, RAINBOW_DQN
    hidden_sizes=[512, 256, 128],          # Neural network layer sizes
    activation="relu",                      # relu, leaky_relu, elu, gelu
    dropout_rate=0.1,                      # Dropout for regularization
    
    # Learning Parameters
    learning_rate=1e-4,                    # Adam learning rate
    weight_decay=1e-5,                     # L2 regularization
    gamma=0.99,                            # Discount factor
    
    # Exploration
    epsilon_start=1.0,                     # Initial exploration rate
    epsilon_end=0.01,                      # Final exploration rate
    epsilon_decay_steps=50000,             # Steps to decay epsilon
    
    # Memory and Training
    memory_size=100000,                    # Replay buffer size
    batch_size=64,                         # Training batch size
    use_per=True,                          # Use Prioritized Experience Replay
    
    # Training Schedule
    max_episodes=10000,                    # Maximum episodes
    warmup_steps=1000,                     # Steps before training starts
    train_frequency=4,                     # Train every N steps
    target_update_frequency=1000,          # Update target network frequency
    
    # Monitoring
    use_tensorboard=True,                  # Enable TensorBoard logging
    save_frequency=2000,                   # Save model every N episodes
    log_frequency=100,                     # Log progress every N episodes
)

Configuration Presets

Easy Mode

game_config, training_config = create_configs_from_difficulty(GameDifficulty.EASY)
# - Slower game speed
# - No obstacles
# - 5000 episodes max
# - Higher learning rate

Medium Mode (Recommended)

game_config, training_config = create_configs_from_difficulty(GameDifficulty.MEDIUM)
# - Normal game speed
# - No obstacles  
# - 10000 episodes max
# - Balanced parameters

Hard Mode

game_config, training_config = create_configs_from_difficulty(GameDifficulty.HARD)
# - Faster game speed
# - 3 obstacles
# - 20000 episodes max
# - Rainbow DQN architecture

Extreme Mode

game_config, training_config = create_configs_from_difficulty(GameDifficulty.EXTREME)
# - Very fast game speed
# - 5 obstacles
# - 50000 episodes max
# - Large network with all features

---

🎯 Training Examples

Example 1: Quick Demo Training

# 5-minute demo training
trainer, results, test_results = quick_train_demo(
    difficulty=GameDifficulty.MEDIUM,
    render=False,  # True to watch training
    max_episodes=500
)

print(f"Best Score: {results['best_score']}")
print(f"Test Performance: {test_results['mean_score']:.2f}")

Example 2: Full Training Session

# Create custom configuration
game_config = GameConfig(width=600, height=600, speed=20)
training_config = TrainingConfig(
    network_type=NetworkType.DUELING_DQN,
    max_episodes=10000,
    use_tensorboard=True
)

# Initialize trainer
trainer = AdvancedSnakeTrainer(
    game_config=game_config,
    training_config=training_config,
    experiment_name="my_snake_experiment"
)

# Train the model
results = trainer.train()

# Test the trained model
test_results = trainer.test_agent(num_episodes=10, render=True)

trainer.close()

Example 3: Advanced Training with Monitoring

# Advanced setup with all features
trainer = AdvancedSnakeTrainer(
    game_config=GameConfig(
        width=800, height=600, enable_obstacles=True, num_obstacles=4
    ),
    training_config=TrainingConfig(
        network_type=NetworkType.RAINBOW_DQN,
        hidden_sizes=[1024, 512, 256],
        max_episodes=25000,
        use_per=True,
        use_tensorboard=True,
        patience=5000  # Early stopping
    ),
    experiment_name="advanced_rainbow_training"
)

# Train with monitoring
try:
    results = trainer.train()
    
    # View results in TensorBoard
    print(f"View training progress: tensorboard --logdir {trainer.experiment_dir}/tensorboard")
    
    # Test multiple models
    best_score_results = trainer.test_agent(
        str(trainer.experiment_dir / "best_model_best_score.pth")
    )
    best_mean_results = trainer.test_agent(
        str(trainer.experiment_dir / "best_model_best_mean_score.pth")
    )
    
finally:
    trainer.close()

---

🧪 Model Testing

Basic Testing

# Test with default best model
trainer = AdvancedSnakeTrainer(GameConfig(), TrainingConfig())
results = trainer.test_agent(
    model_path="models/best_model.pth",
    num_episodes=10,
    render=True
)

Advanced Testing

# Comprehensive evaluation
test_results = trainer.test_agent(
    model_path="experiments/my_experiment/best_model_best_score.pth",
    num_episodes=50,
    render=True,
    record_video=False  # Future feature
)

print("Test Results:")
print(f"Mean Score: {test_results['mean_score']:.2f} ± {test_results['std_score']:.2f}")
print(f"Max Score: {test_results['max_score']}")
print(f"Success Rate: {sum(1 for s in test_results['scores'] if s > 0) / len(test_results['scores']) * 100:.1f}%")
print(f"Death Causes: {test_results['death_causes']}")

Batch Testing (Multiple Models)

import glob

# Test all models in directory
model_paths = glob.glob("experiments/*/best_model_*.pth")

for model_path in model_paths:
    print(f"\nTesting: {model_path}")
    results = trainer.test_agent(model_path, num_episodes=20, render=False)
    print(f"Performance: {results['mean_score']:.2f} ± {results['std_score']:.2f}")

---

🔬 Advanced Usage

Custom Network Architecture

# Create custom DQN variant
class CustomDQN(nn.Module):
    def __init__(self, input_size=17, output_size=3):
        super().__init__()
        # Your custom architecture here
        pass
    
    def forward(self, x):
        # Your forward pass here
        pass

# Use custom network (modify the code)
# trainer.agent.q_network = CustomDQN().to(trainer.agent.device)

Custom Reward Function

# Modify reward in AdvancedSnakeGame.step() method
def calculate_custom_reward(self, prev_distance, current_distance, done, info):
    reward = 0
    
    # Distance-based reward
    distance_improvement = prev_distance - current_distance
    reward += distance_improvement * 5
    
    # Food reward
    if self.head == self.food:
        reward += 100 + len(self.snake) * 5  # Bonus for longer snake
    
    # Survival bonus
    if not done:
        reward += 2
    
    # Death penalty
    if done:
        if info.get('death_cause') == 'wall':
            reward -= 50
        elif info.get('death_cause') == 'self':
            reward -= 100
        else:
            reward -= 25
    
    return reward

Experiment Tracking

# Track multiple experiments
experiments = {}

for difficulty in [GameDifficulty.EASY, GameDifficulty.MEDIUM, GameDifficulty.HARD]:
    print(f"Training {difficulty.name} difficulty...")
    
    game_config, training_config = create_configs_from_difficulty(difficulty)
    training_config.max_episodes = 2000
    
    trainer = AdvancedSnakeTrainer(
        game_config, training_config,
        experiment_name=f"comparison_{difficulty.name.lower()}"
    )
    
    results = trainer.train()
    test_results = trainer.test_agent(num_episodes=20, render=False)
    
    experiments[difficulty.name] = {
        'training_results': results,
        'test_results': test_results
    }
    
    trainer.close()

# Compare results
for name, data in experiments.items():
    print(f"{name}: Best={data['training_results']['best_score']}, "
          f"Test={data['test_results']['mean_score']:.2f}")

Hyperparameter Tuning

# Grid search example
import itertools

learning_rates = [1e-3, 1e-4, 1e-5]
batch_sizes = [32, 64, 128]
network_types = [NetworkType.DQN, NetworkType.DUELING_DQN]

best_score = 0
best_config = None

for lr, batch_size, net_type in itertools.product(learning_rates, batch_sizes, network_types):
    print(f"Testing: LR={lr}, Batch={batch_size}, Network={net_type.name}")
    
    training_config = TrainingConfig(
        learning_rate=lr,
        batch_size=batch_size,
        network_type=net_type,
        max_episodes=1000  # Shorter for grid search
    )
    
    trainer = AdvancedSnakeTrainer(
        GameConfig(), training_config,
        experiment_name=f"gridsearch_lr{lr}_bs{batch_size}_{net_type.name}"
    )
    
    results = trainer.train()
    
    if results['best_score'] > best_score:
        best_score = results['best_score']
        best_config = (lr, batch_size, net_type)
    
    trainer.close()

print(f"Best configuration: LR={best_config[0]}, Batch={best_config[1]}, "
      f"Network={best_config[2].name}, Score={best_score}")

---

🔧 Troubleshooting

Common Issues

1. Import Errors

# Error: ModuleNotFoundError: No module named 'pygame'
pip install pygame torch matplotlib seaborn

# Error: No module named 'tensorboard'
pip install tensorboard

# Error: CUDA out of memory
# Solution: Reduce batch size or use CPU
training_config.batch_size = 32  # Instead of 64

2. Display Issues (Linux/Colab)

# Error: pygame.error: No available video device
# Solution: Setup virtual display
export DISPLAY=:99
Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &

# In Python:
import os
os.environ['DISPLAY'] = ':99'

3. Performance Issues

# Slow training - solutions:
training_config.render = False  # Disable rendering
training_config.batch_size = 32  # Smaller batch
training_config.train_frequency = 8  # Train less frequently

# Out of memory - solutions:
training_config.memory_size = 10000  # Smaller replay buffer
training_config.hidden_sizes = [256, 128]  # Smaller network

4. Model Not Learning

# Possible solutions:
training_config.learning_rate = 1e-3  # Higher learning rate
training_config.epsilon_decay_steps = 20000  # Slower exploration decay
training_config.warmup_steps = 500  # Shorter warmup
training_config.gamma = 0.95  # Lower discount factor

Debug Mode

# Enable debug logging
import logging
logging.getLogger().setLevel(logging.DEBUG)

# Print network architecture
print(trainer.agent.q_network)

# Monitor training metrics
trainer.training_config.log_frequency = 10  # Log every 10 episodes

# Check GPU usage
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"Current device: {trainer.agent.device}")

File Structure

project/
├── snake_ai.py                 # Main code file
├── requirements.txt             # Dependencies
├── snake_ai_models/            # Default model directory
│   ├── experiment_name/
│   │   ├── config.yaml
│   │   ├── training.log
│   │   ├── best_model_best_score.pth
│   │   ├── metrics.json
│   │   ├── training_progress.png
│   │   └── tensorboard/
│   └── ...
└── logs/                       # Log files

---

📊 Performance Benchmarks

Expected Performance by Difficulty

Difficulty	Episodes	Expected Best Score	Training Time	GPU Required
Easy	2,000	15-25	5-10 min	No
Medium	5,000	10-20	15-30 min	Recommended
Hard	10,000	8-15	1-2 hours	Yes
Extreme	25,000	5-12	3-6 hours	Yes

Hardware Recommendations

Setup	CPU	RAM	GPU	Training Speed
Minimum	2+ cores	4GB	None	1x
Recommended	4+ cores	8GB	GTX 1060+	3-5x
Optimal	8+ cores	16GB	RTX 3060+	8-10x

---

🎯 Next Steps

1. Start with Easy Demo: python snake_ai.py --difficulty easy --episodes 500
2. Try Medium Training: python snake_ai.py --mode train --difficulty medium
3. Experiment with Hard: python snake_ai.py --difficulty hard --render
4. Monitor with TensorBoard: tensorboard --logdir snake_ai_models/
5. Custom Experiments: Modify configurations and try different approaches

---

📞 Support & Resources

• Documentation: This file
• Issues: Check console output and logs
• Performance: Use TensorBoard for monitoring
• Customization: Modify GameConfig and TrainingConfig
• Advanced: Implement custom networks and reward functions

Happy Training! 🐍🤖