Skip to content

πŸ“¦ Use GitHub Releases for benchmark data storage instead of committing to repoΒ #2074

@ImBIOS

Description

@ImBIOS

Problem

Currently, benchmark result data is stored directly in the repository under docs/results/ with large JSON files (14-20KB each) for different runtime versions:

docs/results/
β”œβ”€β”€ bun-1.1.json (19KB)
β”œβ”€β”€ bun-1.2.json (19KB)  
β”œβ”€β”€ deno-2.json (17KB)
β”œβ”€β”€ node-14.json (14KB)
β”œβ”€β”€ node-20.json (20KB)
β”œβ”€β”€ node-22.json (20KB)
└── ... (12+ more files)

Issues with current approach:

  • Repository bloat - ~300KB+ of benchmark data files
  • Git history pollution - Large binary-like files create noisy commits
  • Clone performance - Repository gets heavier over time
  • Poor data management - No proper versioning or tagging of benchmark data
  • Scalability concerns - Linear growth as we add more runtimes

Proposed Solution: GitHub Releases

Move benchmark data storage to GitHub Releases using runtime version tags for better organization.

Benefits

  • Clean repository - Keep code separate from data
  • Runtime-based versioning - Tag releases by the runtime versions being benchmarked
  • API access - GitHub Releases API for programmatic data access
  • Historical preservation - All benchmark data accessible without repo bloat
  • Faster clones - Significantly smaller repository size
  • Better searchability - Easy to find benchmarks for specific runtime versions
  • Professional approach - Follows GitHub best practices for data distribution

Implementation Plan

  1. Update CI workflow:

    • Create releases instead of committing files
    • Use runtime version tags (e.g., node-24.0.0, bun-1.2.0, deno-2.0.0)
    • Attach corresponding JSON files and SVG as release assets
  2. Update web dashboard:

    • Fetch data from GitHub Releases API instead of static files
    • Implement caching for better performance
    • Add runtime version selector for comparisons
  3. Migration strategy:

    • Create initial releases for existing benchmark data
    • Update documentation and README

Example Release Structure

Release: node-24.0.0
Tag: node-24.0.0
Title: "Node.js 24.0.0 Benchmark Results"
Assets: node-24.json, preview.svg

Release: bun-1.2.0  
Tag: bun-1.2.0
Title: "Bun 1.2.0 Benchmark Results"
Assets: bun-1.2.json, preview.svg

Release: deno-2.0.0
Tag: deno-2.0.0
Title: "Deno 2.0.0 Benchmark Results"
Assets: deno-2.json, preview.svg

API Usage Example

// Fetch Node.js 24 benchmark data
const nodeResponse = await fetch('https://api.github.com/repos/ImBIOS/typescript-runtime-type-benchmarks/releases/tags/node-24.0.0');
const nodeRelease = await nodeResponse.json();

// Fetch Bun 1.2 benchmark data
const bunResponse = await fetch('https://api.github.com/repos/ImBIOS/typescript-runtime-type-benchmarks/releases/tags/bun-1.2.0');
const bunRelease = await bunResponse.json();

// List all available runtime versions
const allReleases = await fetch('https://api.github.com/repos/ImBIOS/typescript-runtime-type-benchmarks/releases');

Benefits Summary

  • βœ… Cleaner repository structure
  • βœ… Runtime version-based organization for easy discovery
  • βœ… Improved clone performance
  • βœ… Easy searchability by runtime version
  • βœ… Professional data distribution approach
  • βœ… Historical data access with clear runtime versioning
  • βœ… Scalable for future runtime versions

This approach makes it intuitive to find benchmark results for specific runtime versions and aligns with best practices for data-heavy projects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions