Merge pull request #25 from codegen-sh/codegen-cg-18634-add-benchmark-comparison-and-visualization

Author: jayhack

BENCHMARKING.md

@@ -2,6 +2,30 @@
 
 This document provides comprehensive instructions for running and understanding Grainchain sandbox provider benchmarks.
 
+## 📊 New: Advanced Analysis & Comparison Tools
+
+Grainchain now includes powerful analysis capabilities to help you make data-driven decisions about provider selection and performance optimization.
+
+### Quick Start with Analysis
+
+```bash
+# Run benchmarks and analyze results
+grainchain benchmark --provider all
+grainchain analysis compare --provider1 local --provider2 e2b --chart
+grainchain analysis report --format html --include-charts
+```
+
+### Key Analysis Features
+
+- **Provider Comparison**: Compare performance metrics between providers
+- **Trend Analysis**: Track performance changes over time
+- **Regression Detection**: Automatically detect performance degradations
+- **Interactive Dashboards**: Visualize data with charts and graphs
+- **Comprehensive Reports**: Generate detailed analysis reports
+- **Provider Recommendations**: Get data-driven provider suggestions
+
+For detailed analysis documentation, see the [Analysis Guide](docs/analysis_guide.md).
+
 ## 🚀 Quick Start
 
 ### 5-Minute Benchmark Walkthrough

benchmarks/README.md

@@ -2,6 +2,44 @@
 
 This directory contains benchmarking infrastructure for testing and comparing different sandbox providers in the Grainchain project.
 
+## 📊 New: Benchmark Analysis & Comparison
+
+Grainchain now includes powerful analysis tools to compare providers, analyze trends, and generate comprehensive reports.
+
+### Quick Analysis Commands
+
+```bash
+# Compare two providers
+grainchain analysis compare --provider1 local --provider2 e2b --chart
+
+# Analyze performance trends
+grainchain analysis trends --provider local --days 30 --interactive
+
+# Generate comprehensive report
+grainchain analysis report --format html --include-charts
+
+# Detect performance regressions
+grainchain analysis regressions --threshold 0.1
+
+# Get provider recommendations
+grainchain analysis recommend --use-case reliability
+
+# Create performance dashboard
+grainchain analysis dashboard --interactive
+```
+
+### Analysis Features
+
+- **Provider Comparison**: Compare performance between different providers
+- **Trend Analysis**: Analyze performance trends over time with statistical insights
+- **Regression Detection**: Automatically detect performance regressions
+- **Visualization**: Generate charts, graphs, and interactive dashboards
+- **Comprehensive Reports**: Create detailed reports in HTML, Markdown, or PDF
+- **Provider Recommendations**: Get data-driven provider recommendations
+- **Configuration Management**: Customize analysis settings and preferences
+
+For detailed analysis documentation, see [Analysis Guide](../docs/analysis_guide.md).
+
 ## Overview
 
 The benchmark suite evaluates sandbox providers across multiple scenarios:

benchmarks/analysis/__init__.py

@@ -0,0 +1,24 @@
+"""
+Grainchain Benchmark Analysis Module
+
+This module provides tools for analyzing, comparing, and visualizing
+benchmark results across different providers, time periods, and metrics.
+"""
+
+from .comparator import BenchmarkComparator
+from .data_parser import BenchmarkDataParser
+from .models import BenchmarkResult, ComparisonResult, ProviderMetrics
+from .reporter import BenchmarkReporter
+from .visualizer import BenchmarkVisualizer
+
+__all__ = [
+    "BenchmarkDataParser",
+    "BenchmarkComparator",
+    "BenchmarkVisualizer",
+    "BenchmarkReporter",
+    "BenchmarkResult",
+    "ProviderMetrics",
+    "ComparisonResult",
+]
+
+__version__ = "1.0.0"