Skip to content

Latest commit

 

History

History
1020 lines (725 loc) · 33.6 KB

File metadata and controls

1020 lines (725 loc) · 33.6 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.7.0] - 2025-12-11

Performance

Major Performance Optimizations - Target <20x Overhead

  • Lazy Stack Capture: Removed CaptureStack() from hot path

    • Before: ~329ns per write (full stack capture)
    • After: ~127ns per write (PC only, lazy full capture on race)
    • 2.6x faster hot path
  • Lock-Free VarState: Replaced mutex with atomic operations

    • Hot-path fields now use atomic.Uint64, atomic.Int64, atomic.Uintptr
    • Mutex only for complex operations (VectorClock promotion)
    • 10x faster VarState access
  • Sharded Shadow Memory: 256 shards instead of single sync.Map

    • Reduces contention under concurrent load
    • Cache-line padding prevents false sharing
    • 12.5% faster concurrent access
  • VectorClock Pooling: sync.Pool for VectorClock reuse

    • Before: 1,009 allocations per benchmark
    • After: 3 allocations per benchmark
    • 99.7% reduction in allocations
    • 3,387x faster VectorClock creation

[0.6.0] - 2025-12-11

Fixed

Unsynchronized Access Detection - COMPLETE

  • GoStart/GoEnd instrumentation: Proper VectorClock inheritance from parent to child goroutines
    • Parent's clock is snapshotted at go func() statement
    • Child inherits parent's clock, establishing happens-before edge
    • Fixes false negatives for x = 42; go func() { _ = x }() patterns
  • SmartTrack TOCTOU race fix: Added CompareAndSwapExclusiveWriter() for atomic ownership claim
    • Two goroutines could both see exclusiveWriter=0 and take fast path
    • CAS ensures only one goroutine claims ownership, other falls through to HB check
    • Fixes ~7% intermittent false negative rate
  • Spawn Context FIFO ordering: Changed from sync.Map to slice+mutex
    • sync.Map.Range() iterates non-deterministically
    • Slice ensures first spawn matches first child (correct clock inheritance)
  • sync.Map reassignment race: Changed Init()/Reset() to clear maps via Range+Delete
    • Reassigning contexts = sync.Map{} races with goroutines still accessing the map
    • Fixes panics during test suite runs

Test Reliability: 70% -> 100%

  • All 4 previously skipped tests now pass consistently
  • 20/20 test suite runs passing

Added

Go Race Test Suite - 100% Coverage

  • 359/359 test scenarios ported from official Go race detector test suite
    • 355 scenarios from Go 1.21 test suite
    • 4 scenarios from Go 1.25.3 (non-inline comparison tests)
  • 355 tests passing (98.88% pass rate)
  • 4 tests skipped (known detector limitations - unsync access tracking)
  • 21 test category files organized by pattern type:
    • Basic, Mutex, Channel, Sync primitives
    • Memory, Patterns, Lifecycle, Atomic
    • Issues, Reflect, Advanced, String
    • Complex, Append, Defer/Panic
    • TypeAssert, Method, Range, Final
    • Compare (Go 1.25+ non-inline comparisons)

Test Patterns Covered:

  • Write-write races, read-write races
  • Mutex synchronization (same/different mutexes)
  • Channel synchronization
  • WaitGroup, Cond, Once, Pool
  • Struct fields, slices, maps, arrays
  • Closures, defer, panic/recover
  • Type assertions, interfaces
  • Atomic operations simulation
  • Range loops, method receivers

[0.5.2] - 2025-12-10

Fixed

  • Corrected goid offsets for all Go versions: The gobuf.ret field was removed in Go 1.25, changing the goid offset:
    • Go 1.23: offset 160 (gobuf=56 bytes, 7 fields including ret)
    • Go 1.24: offset 160 (gobuf=56 bytes, 7 fields including ret)
    • Go 1.25: offset 152 (gobuf=48 bytes, 6 fields, ret removed)
  • Fixed goid_go124.go offset from incorrect 152 to correct 160
  • Updated release workflow to use Go 1.25 (matches go.mod 1.24+ requirement)
  • Fixed .golangci.yml pattern for goid_go*.go files

[0.5.1] - 2025-12-10

Fixed

  • Version-specific goid files: Replaced single goid_fast.go with version-specific:
    • goid_go123.go - Go 1.23 support
    • goid_go124.go - Go 1.24 support
    • goid_go125.go - Go 1.25 support
  • Note: This release had incorrect offset for Go 1.24 (fixed in v0.5.2)

[0.5.0] - 2025-12-10

Performance Breakthrough: ~2200x Faster Goroutine ID Extraction!

This release introduces native assembly implementation for goroutine ID extraction, eliminating the runtime.Stack parsing overhead on supported platforms.

Added

Assembly-Optimized GID Extraction

  • goid_amd64.s: Native assembly for x86-64 using TLS access
    • MOVQ (TLS), R14 to get g pointer from Thread Local Storage
    • Zero allocations, ~1.73 ns/op
  • goid_arm64.s: Native assembly for ARM64 using dedicated g register
    • MOVD g, R0 to read g pointer from R28 register
    • Zero allocations, ~2 ns/op
  • goid_go123.go / goid_go124.go / goid_go125.go: Version-specific Go wrappers
    • Reads goid at correct offset for each Go version
    • Go 1.23: offset 160, Go 1.24/1.25: offset 152
    • Automatic fallback to runtime.Stack on nil g pointer
  • goid_fallback.go: Fallback for unsupported platforms
    • Used on Go <1.23, Go >=1.26, or non-amd64/arm64 architectures
    • Uses runtime.Stack parsing (~3987 ns/op)

Performance

Method Time Allocations Speedup
Assembly (amd64) 1.73 ns/op 0 B/op ~2200x
Assembly (arm64) ~2 ns/op 0 B/op ~2000x
runtime.Stack 3987 ns/op 64 B/op baseline

Technical Details

Build Constraints:

//go:build go1.23 && !go1.24 && (amd64 || arm64)  // for Go 1.23
//go:build go1.24 && !go1.25 && (amd64 || arm64)  // for Go 1.24
//go:build go1.25 && !go1.26 && (amd64 || arm64)  // for Go 1.25

g struct goid offset calculation:

Go 1.24/1.25 (gobuf=48 bytes):
  goid at offset 152

Go 1.23 (gobuf=56 bytes):
  goid at offset 160

Changed

  • goid_generic.go: Refactored to provide common utilities
    • getGoroutineID() - main entry point
    • getGoroutineIDSlow() - runtime.Stack fallback
    • parseGID() - optimized byte parsing (no regex, no allocations)
  • .golangci.yml: Added exclusion for govet unsafeptr check on goid files
  • scripts/pre-release-check.sh: Added -unsafeptr=false flag for go vet

Removed

  • No external dependencies for goroutine ID extraction
  • outrigdev/goid dependency was evaluated but rejected in favor of own implementation

Why Own Implementation?

Strategic decision: Zero external dependencies for Go runtime proposal!

  • If Go team accepts this detector into runtime, it's their responsibility to maintain GID extraction
  • External dependency would complicate acceptance into official Go toolchain
  • Our implementation is smaller and tailored specifically for race detector needs

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.5.1

[0.4.10] - 2025-12-10

Fix: Run go mod tidy in src directory

This release fixes the missing go.sum entries for racedetector.

Fixed

  • "missing go.sum entry for module providing package"
    • After adding racedetector require to src/go.mod, go.sum was outdated
    • Now running go mod tidy in src/ directory to update go.sum

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.10

[0.4.9] - 2025-12-10

Fix: Add racedetector dependency to instrumented go.mod

This release fixes the missing racedetector dependency in the instrumented module.

Fixed

  • "no required module provides package github.com/kolkov/racedetector/race"
    • Instrumented code imports racedetector/race, but src/go.mod (copy of original) didn't have this dependency
    • Now appending require github.com/kolkov/racedetector VERSION to src/go.mod

Changed

  • Added runtime.Version constant for consistent version management
  • instrumentTestSources() now adds racedetector require to src/go.mod
  • ModFileOverlay() now uses runtime.Version constant

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.9

[0.4.8] - 2025-12-10

Hotfix: Copy go.mod to instrumented sources

This hotfix ensures the instrumented source directory is a valid Go module.

Fixed

  • go mod tidy fails: "reading src/go.mod: no such file or directory"
    • The replace MODULE => ./src directive requires ./src to be a valid module
    • Now copying original go.mod to workspace/src/ directory
    • This allows go mod tidy to resolve the replace directive correctly

Changed

  • instrumentTestSources() now copies go.mod to srcDir before go.sum

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.8

[0.4.7] - 2025-12-10

Feature: Internal Module Import Resolution

This release fixes internal import resolution for multi-package modules.

Fixed

  • Issue #10: Internal module imports not rewritten during instrumentation
    • Imports like github.com/MODULE/subpackage now resolve correctly
    • Added replace MODULE => ./src directive to instrumented go.mod
    • This allows internal packages to find each other in the temp workspace

Changed

  • ModFileOverlay() now adds replace directive for original module

    • Reads module name from original go.mod via new getModuleName() function
    • Skips adding replace for racedetector's own module (avoids conflicts)
  • New function getModuleName() - extracts module path from go.mod

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.7

[0.4.6] - 2025-12-10

Hotfix: Ultra-Conservative Identifier Handling

This hotfix takes an ultra-conservative approach to identifier instrumentation to fix remaining edge cases.

Fixed

  • Issue #9 (final): Comprehensive identifier filtering
    • IsValidationError (func) - functions from other files had nil Obj
    • commandBufferMarker (type) - types from other packages had nil Obj
    • generic type ID without instantiation - generic type parameters
    • undefined: val - scope issues with identifiers

Changed

  • Ultra-conservative Ident handling in shouldInstrument()

    • ONLY instrument identifiers with ident.Obj != nil && ident.Obj.Kind == ast.Var
    • Skip ALL identifiers where we cannot confirm they are variables
    • This eliminates false positives from functions, types, generics, etc.
  • New AST handlers in extractReads()

    • *ast.TypeAssertExpr - skip Type, only walk into X
    • *ast.FuncLit - skip anonymous function bodies (separate scope)

Limitations

  • Only local variables (same file) are instrumented
  • Package-level variables from other files may be missed
  • This is a trade-off for correctness over coverage
  • Full type-checking integration needed for complete coverage

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.6

[0.4.5] - 2025-12-10

Hotfix: CompositeLit, CallExpr and Generic Functions

This hotfix properly handles composite literals, function calls, and generic function instantiations.

Fixed

  • Issue #9 (final fix): Type expressions in composite literals

    • Surface (type) is not an expression - type names in Type{...} were being instrumented
    • Added CompositeLit handling in extractReads() - skip Type, only walk into Elts
  • Function calls and type conversions

    • Skip Fun part of CallExpr (could be function, method, or type conversion)
    • Only instrument the arguments
  • Generic function instantiation

    • cannot use generic function without instantiation error
    • Skip IndexListExpr entirely (generic type params like Func[T, U])

Changed

  • extractReads() now handles additional AST node types:
    • *ast.CompositeLit - skip Type, walk into Elts only
    • *ast.CallExpr - skip Fun, walk into Args only
    • *ast.IndexListExpr - skip entirely (generic instantiation)

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.5

[0.4.4] - 2025-12-10

Hotfix: Conservative SelectorExpr & Struct Literals

This hotfix takes a more conservative approach to instrumentation, fixing remaining edge cases that caused compilation errors.

Fixed

  • Issue #9 (continued): More SelectorExpr edge cases

    • Method values: hub.GetAdapter, result.String
    • Return value fields: GetGlobal().Hub
    • Package-qualified identifiers with non-stdlib packages
    • Undefined identifiers in complex expressions
  • Struct literal field names

    • Field names in struct literals (Point{X: 1, Y: 2}) were incorrectly treated as variables
    • Added KeyValueExpr handling in extractReads() to skip field names
    • Only the value expressions are now instrumented

Changed

  • Skip ALL SelectorExpr in shouldInstrument()

    • Without full type information, too many non-addressable cases exist
    • This is a trade-off: may miss some struct field race conditions
    • Safer than breaking compilation on valid code
  • Removed isLikelyPackageName() - no longer needed with conservative approach

  • Added KeyValueExpr handling in extractReads()

    • Key (field name) is skipped
    • Value expression is still walked for instrumentation

Limitations

  • Struct field access via dot notation (obj.Field) is not instrumented
  • This reduces detection coverage but ensures correctness
  • Full type-checking integration planned for future release

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.4

[0.4.3] - 2025-12-10

Hotfix: Package Functions & Map Index Support

This hotfix resolves remaining instrumentation errors with package function calls and map index expressions.

Fixed

  • Package function calls (os.ReadFile, strconv.Atoi, filepath.Join, etc.)

    • Added isLikelyPackageName() heuristic to detect standard library packages
    • SelectorExpr now properly filtered when X is a package identifier
  • Map index expressions (map[key])

    • Cannot take address of map element in Go
    • All IndexExpr now skipped (conservative approach without type info)
    • This may miss some slice/array race conditions, but avoids compilation errors

Changed

  • shouldInstrument() now handles additional expression types:

    • *ast.SelectorExpr - skip package.Function patterns
    • *ast.IndexExpr - skip all index expressions (maps not addressable)
  • New function isLikelyPackageName() - heuristic for common stdlib packages

Limitations

  • IndexExpr on slices/arrays is now skipped (false negatives possible)
  • Custom package names not in stdlib list may still cause issues
  • Full type-checking would be needed for 100% accuracy

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.3

[0.4.2] - 2025-12-10

Hotfix: Function References & Built-in Support

This hotfix resolves instrumentation errors when code contains function references, built-in functions, or type conversions.

Fixed

  • Issue #9: Instrumentation breaks function references (#9)
    • Root cause: shouldInstrument() didn't filter functions, types, packages, or built-ins
    • Fix: Added comprehensive filtering for all non-addressable expressions
    • Errors fixed:
      • cannot take address of <function> (value of type func(...))
      • make (built-in) must be called
      • string (type) is not an expression

Changed

  • isBuiltinIdent() now includes all Go built-in functions and types:

    • Built-in functions: make, new, len, cap, append, copy, delete, close, panic, recover, print, println, complex, real, imag, clear, min, max
    • Built-in types: int, int8-64, uint, uint8-64, float32, float64, complex64, complex128, bool, byte, rune, string, error, any, comparable, uintptr
  • shouldInstrument() now checks ast.Obj.Kind to skip:

    • ast.Fun - function identifiers
    • ast.Typ - type identifiers
    • ast.Pkg - package identifiers

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.2

[0.4.1] - 2025-12-10

Hotfix: CI Environment Support

This hotfix resolves racedetector test failures in GitHub Actions and other CI environments.

Fixed

  • Issue #8: racedetector test fails in CI (#8)
    • Root cause: ModFileOverlay() returned empty string in "published mode" (when installed via go install)
    • Fix: Now always creates go.mod in workspace, using published package version in CI
    • Error: directory prefix . does not contain main module is now resolved

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.1

[0.4.0] - 2025-12-09

racedetector test Command Release

This release introduces the test command - a drop-in replacement for go test -race that works with CGO_ENABLED=0.

Added

  • racedetector test command - Run tests with race detection without CGO
    • All go test flags supported: -v, -run, -bench, -cover, -coverprofile, -timeout, -count, -parallel, -tags, -ldflags, etc.
    • Recursive package patterns: ./..., ./pkg/..., ./internal/...
    • Test files (_test.go) properly instrumented alongside source files
    • Works exactly like go test -race but without CGO requirement

Fixed

  • Critical: IncDecStmt instrumentation - counter++ and counter-- operations now properly instrumented

    • Both READ and WRITE operations are detected for increment/decrement
    • Code inside anonymous functions (go func() {...}()) now instrumented correctly
    • This fix enables detection of races that were previously missed
  • findProjectRoot() improvement - No longer confuses user's go.mod with racedetector's

    • Uses internal/race/api directory marker for accurate detection
    • Added executable path fallback for installed binaries
    • Fixes "module does not contain package" errors

Usage

# Basic usage
racedetector test ./...

# With flags
racedetector test -v -run TestFoo ./pkg/...
racedetector test -cover -coverprofile=coverage.out ./...
racedetector test -bench . -benchmem ./...

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.4.0

[0.3.2] - 2025-12-01

Go 1.24+ Requirement & Replace Directive Fix

This hotfix release upgrades the minimum Go version to 1.24 and fixes handling of replace directives in go.mod files.

Changed

  • BREAKING: Minimum Go version is now 1.24 (was 1.21)
    • Go 1.24 provides significant performance improvements (Swiss Tables maps, improved sync.Map)
    • Better runtime performance benefits race detection workloads
    • Users requiring Go 1.21 support can use v0.3.1

Fixed

  • Issue #6: Replace directives not working (#6)
    • replace directives from original project's go.mod are now properly copied to instrumented code
    • Relative paths (e.g., ../locallib) are automatically converted to absolute paths
    • Supports all replace directive formats (with/without version specifiers)

Added

  • New dependency: golang.org/x/mod v0.30.0
    • Official Go module for parsing go.mod files
    • Enables proper handling of replace directives
  • New functions in runtime package:
    • findOriginalGoMod() - Locates project's go.mod file
    • extractReplaceDirectives() - Parses and converts replace directives
    • isLocalPath() - Detects local filesystem paths

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.3.2

Note: Requires Go 1.24 or higher.

Upgrade from v0.3.1

If your project uses replace directives in go.mod, they will now work automatically. No code changes required.


[0.3.1] - 2025-12-01

Documentation Hotfix

Quick fix for documentation errors discovered after v0.3.0 release.

Fixed

  • INSTALLATION.md: Updated binary download instructions (binaries available since v0.2.0)
  • INSTALLATION.md: Fixed branch reference (mastermain)
  • USAGE_GUIDE.md: Corrected test command status (Planned for v0.4.0, not production-ready)
  • USAGE_GUIDE.md: Fixed license reference (MIT License, not BSD 3-Clause)

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.3.1

[0.3.0] - 2025-11-28

Advanced Performance Optimizations Release

This release delivers major performance improvements through sparse-aware algorithms and memory compression techniques, achieving up to 43x faster vector clock operations and 8x memory reduction.

Development Time: 1 day (research + implementation)

Added

P0: Sampling-Based Detection

  • New RACEDETECTOR_SAMPLE_RATE environment variable (0-100%)
  • Probabilistic sampling for CI/CD workflows
  • Based on TSAN's trace_pos approach
  • Zero-overhead when disabled (single branch)
  • Near-zero overhead when enabled (~5ns atomic increment)

P1: Enhanced Read-Shared (4 Inline Slots)

  • 4 inline readEpochs[4] slots in VarState
  • Avoids 256KB VectorClock allocation for 2-4 readers
  • Delayed promotion to full VectorClock
  • Reduces memory pressure for common read patterns

P1: Sparse-Aware Vector Clocks

  • Track maxTID to avoid iterating 65,536 elements
  • 655x speedup for typical programs (~100 goroutines)
  • New CopyFrom() method for efficient in-place copying
  • Join: 11.48 ns/op (was ~500ns)
  • LessOrEqual: 14.80 ns/op (was ~300ns)
  • Get/Set: 0.31-0.37 ns/op
  • Increment: 0.66 ns/op
  • Zero allocations for all operations

P1: Compressed Shadow Memory (8-Byte Alignment)

  • Address compression to 8-byte boundaries
  • Up to 8x memory reduction for sequential accesses
  • Configurable via SetAddressCompression()
  • Load (hit): 2.46 ns/op
  • LoadOrStore (hit): 3.88 ns/op
  • Concurrent: 8.81 ns/op
  • Zero allocations

Changed

  • VectorClock: Changed from array type to struct with clocks and maxTID fields
  • VectorClock.Clone(): Now uses sparse-aware copying (only copies up to maxTID)
  • VectorClock.Join(): Optimized to iterate only up to max(vc1.maxTID, vc2.maxTID)
  • VectorClock.LessOrEqual(): Optimized for sparse clocks
  • CASBasedShadow: Added address compression support

Performance

Metric v0.2.0 v0.3.0 Improvement
VectorClock Join ~500ns 11.48ns 43x faster
VectorClock LessOrEqual ~300ns 14.80ns 20x faster
Shadow Load (hit) ~10ns 2.46ns 4x faster
Shadow LoadOrStore (hit) ~10ns 3.88ns 2.5x faster
Memory (sequential) 100% ~12.5% 8x reduction

Statistics

  • Tests: 670+ passing (100% pass rate)
  • Linter: 0 issues in production code
  • Coverage: 86.3% (>70% requirement)
  • Quality Gates: All passed

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.3.0

Upgrade from v0.2.0

100% backward compatible - no code changes required!

New optional features:

  • Set RACEDETECTOR_SAMPLE_RATE=50 for 50% sampling in CI
  • Call SetAddressCompression(true) for memory savings

Acknowledgments

Research Papers:

  • "Dynamic Race Detection with O(1) Samples" (PLDI 2024) - Sampling approach
  • ThreadSanitizer trace_pos design - Atomic counter sampling
  • Sparse vector clock optimizations from academic literature

[0.2.0] - 2025-11-20

🎉 Production-Ready Release - Performance + Hardening!

This release consolidates performance optimizations + production hardening into ONE production-ready release, delivering 99% overhead reduction and enterprise-grade reliability.

Why consolidated? First impression matters - v0.2.0 is now a complete, production-ready race detector suitable for real-world deployment.

Added

Performance Optimizations (Tasks 1-3):

Task 1: CAS-Based Shadow Memory

  • Lock-free atomic CAS array replacing sync.Map
  • 81.4% faster (2.07ns vs 11.12ns)
  • Zero allocations (0 B/op, was 48-128 B/op)
  • 34-56% memory savings vs sync.Map
  • <0.01% collision rate

Task 2: BigFoot Static Coalescing

  • Static analysis to coalesce consecutive memory operations at AST level
  • Based on "Effective Race Detection for Event-Driven Programs" (PLDI 2017)
  • 90% barrier reduction (10 barriers → 1 barrier)
  • Works on: struct field access, array indexing, slice iteration

Task 3: SmartTrack Ownership Tracking

  • Exclusive writer tracking to skip happens-before checks
  • Based on "SmartTrack: Efficient Predictive Race Detection" (PLDI 2020)
  • 10-20% HB check reduction
  • Single-writer fast path: 30-50% faster

Production Hardening (Tasks 4-6):

Task 4: Increase MaxThreads and ClockBits

  • TIDBits: 8 → 16 (256 → 65,536 goroutines, 256× improvement)
  • ClockBits: 24 → 48 (16M → 281T operations, 16M× improvement)
  • Epoch: uint32 → uint64 (16-bit TID + 48-bit clock)
  • VectorClock.MaxThreads: 256 → 65,536
  • Supports large-scale applications and long-running servers

Task 5: Overflow Detection with Warnings

  • Atomic overflow detection flags (tidOverflowDetected, clockOverflowDetected)
  • 90% warning thresholds (MaxTIDWarning, MaxClockWarning)
  • Periodic checks every 10K operations
  • Clamping to prevent wraparound (production-safe)
  • Clear error messages with actionable advice

Task 6: Stack Depot for Complete Race Reports

  • New package: internal/race/stackdepot/
  • ThreadSanitizer v2 stack depot approach
  • Stack deduplication using FNV-1a hash
  • VarState stores writeStackHash and readStackHash
  • Complete race reports with both current and previous stacks
  • 8 frames per stack (ThreadSanitizer standard)

Changed

  • VarState size: 40 → 48 → 64 bytes
    • Task 4: 40 → 48 bytes (Epoch uint32 → uint64)
    • Task 6: 48 → 64 bytes (+ 2 × uint64 stack hashes)
  • Epoch type: uint32 → uint64 (16-bit TID + 48-bit clock)
  • VectorClock.MaxThreads: 256 → 65,536
  • RaceContext.TID: uint8 → uint16

Performance

Combined Impact:

  • Hot path overhead: 15-22% → 2-5% (74× speedup!)
  • Consecutive operations: 100× overhead → 1× overhead (99% reduction!)
  • Memory allocations: 48-128 B/op → 0 B/op (100% reduction!)
  • Barrier reduction: 90% fewer barriers (BigFoot)
  • Single-writer fast path: 30-50% faster (SmartTrack)

Scalability:

  • Max goroutines: 256 → 65,536 (256× improvement)
  • Max operations: 16M → 281T (16M× improvement)
  • Overflow detection: Silent failures → Early warnings ✅

Debugging:

  • Previous stack trace: "" → Full stack traces ✅
  • Stack deduplication: FNV-1a hash (memory efficient)
  • Stack depth: 8 frames (ThreadSanitizer standard)

Statistics

  • Tests: 670+ passing (100% pass rate)
  • Linter: 0 issues in production code
  • Code additions: ~500 lines (6 tasks)
  • New package: stackdepot (200+ lines)
  • Files modified: 30+ (core + tests)

Validation

All 6 tasks complete:

  1. ✅ CAS-based shadow memory (81.4% faster, 0 allocs)
  2. ✅ BigFoot static coalescing (90% barrier reduction)
  3. ✅ SmartTrack ownership tracking (10-20% HB reduction)
  4. ✅ Increase limits (65K goroutines, 281T ops)
  5. ✅ Overflow detection (warnings at 90% threshold)
  6. ✅ Stack depot (complete race reports)

Production-Ready Status:

  • ✅ Performance: 2-5% overhead (competitive with ThreadSanitizer)
  • ✅ Scalability: 65K+ goroutines, 281T operations
  • ✅ Reliability: Overflow detection with early warnings
  • ✅ Debuggability: Complete race reports with both stacks
  • ✅ Quality: 670+ tests, 0 linter issues, 100% pass rate

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.2.0

Upgrade from v0.1.0

100% backward compatible - no code changes required!

Acknowledgments

Research Papers:

  • Lock-Free Data Structures (Herlihy & Shavit) - CAS-based shadow memory
  • "Effective Race Detection for Event-Driven Programs" (PLDI 2017) - BigFoot coalescing
  • "SmartTrack: Efficient Predictive Race Detection" (PLDI 2020) - Ownership tracking
  • ThreadSanitizer v2 Design - Stack depot architecture

Community Feedback:

  • Dmitry Vyukov (dvyukov) - Identified MVP limitations in issue #6508

[0.1.0] - 2025-11-19

🎉 First Production Release - Race Detector WORKS!

This is the first working release of the Pure-Go Race Detector! The detector successfully catches real data races in concurrent Go code without requiring CGO.

Added

Phase 6A - Standalone Tool (Complete):

  • AST-based code instrumentation with race call insertion ✅
  • Automatic race package import injection ✅
  • race.Init() insertion at program start ✅
  • racedetector build command (drop-in replacement for go build) ✅
  • racedetector run command (drop-in replacement for go run) ✅
  • Cross-platform support (Linux, macOS, Windows) ✅
  • Dogfooding demo (tool tests itself) ✅

Phase 2 - Refinement (Complete):

  • Smart instrumentation filtering (skips constants, built-ins, literals, blank identifier)
  • Professional error messages with file:line:column
  • Verbose mode (-v) with instrumentation statistics
  • 5-15% overhead reduction through smart filtering

Runtime Implementation:

  • FastTrack algorithm (PLDI 2009) ✅
  • Detector with write/read race detection ✅
  • Shadow memory tracking ✅
  • Vector clocks for happens-before relationships ✅
  • Adaptive epoch ↔ vector clock switching (260x memory savings) ✅
  • Sync primitive tracking (Mutex, Channel, WaitGroup) ✅
  • Production-quality race reports with stack traces ✅
  • Race deduplication (no report spam) ✅

Documentation:

  • Comprehensive installation guide (INSTALLATION.md)
  • Complete usage guide (USAGE_GUIDE.md)
  • Example programs (mutex_protected, channel_sync)
  • Security policy (SECURITY.md)
  • Contributing guidelines (CONTRIBUTING.md)
  • Code of Conduct (CODE_OF_CONDUCT.md)

Changed

  • BREAKING: Upgraded from v0.1.0-alpha (proof-of-concept) to v0.1.0 (production-ready)
  • AST instrumentation now actually inserts race detection calls (was TODO)
  • Error messages now include file:line:column and suggestions
  • Build command now supports -v flag for verbose statistics

Fixed

  • AST instrumentation now correctly handles := (DEFINE token)
  • Constants, literals, and built-ins no longer instrumented (reduces false positives)
  • Blank identifier _ no longer instrumented (prevents "cannot use _ as value" errors)
  • Race detection calls now properly inserted BEFORE memory accesses

Performance

  • Hot path overhead: 15-22%
  • Instrumentation overhead: 5-15% reduction through smart filtering
  • Real-world workloads: 368-809ns/op
  • Zero allocations on hot paths
  • Scalability: 1000+ goroutines tested

Statistics

  • Production Code: 22,653 lines (22,273 + 380 from Phase 2)
  • Test Code: 970+ lines (850 + 120 from Phase 2)
  • Documentation: 25,601 lines
  • Total: 49,224+ lines (code + docs)
  • Tests: 70+ tests passing (100% pass rate)
  • Coverage: 45-92% across packages

Validation

Simple Race Test:

$ racedetector build examples/dogfooding/simple_race.go
$ ./simple_race
==================
WARNING: DATA RACE
Write at 0x000000c00000a0b8 by goroutine 4:
  ...
Previous Write at 0x000000c00000a0b8 by goroutine 3:
  ...
==================

DETECTOR WORKS! Successfully detects real data races!

Known Limitations

This release provides a fully functional race detector, but some advanced features are planned for future versions:

  • Stack traces show current access only (previous access placeholder)
  • Limited to memory access races (sync primitive races in progress)
  • Performance overhead higher than Go's official race detector (15-22% vs 5-10%)

Installation

go install github.com/kolkov/racedetector/cmd/racedetector@v0.1.0

Usage

# Build with race detection
racedetector build main.go

# Build with verbose statistics
racedetector build -v main.go

# Run with race detection
racedetector run main.go

Project Timeline

  • Path A: November 19, 2025 (target: December 15, 2025)
  • Phase 2: November 19, 2025
  • Path B: January 31, 2026 (planned)
  • Go Proposal: Q2 2026 (planned)

Acknowledgments

Based on the FastTrack algorithm (Flanagan & Freund, PLDI 2009). Inspired by Go team's ThreadSanitizer integration and community feature requests.


[0.1.0-alpha] - 2025-11-19

Note

This version was superseded by v0.1.0 on the same day after AST instrumentation was completed. The alpha was a proof-of-concept; v0.1.0 is the first production release.

Added

Path A - Standalone Tool (COMPLETE):

  • AST-based code instrumentation
  • Automatic race package import injection
  • race.Init() insertion at program start
  • racedetector build command (drop-in replacement for go build)
  • racedetector run command (drop-in replacement for go run)
  • Cross-platform support (Linux, macOS, Windows)
  • Dogfooding demo (tool tests itself)
  • Comprehensive documentation (Installation, Usage, Examples)
  • Example programs (mutex_protected, channel_sync)

Runtime Implementation:

  • FastTrack algorithm (PLDI 2009)
  • Detector with write/read race detection
  • Shadow memory tracking
  • Vector clocks for happens-before relationships
  • Adaptive epoch ↔ vector clock switching (260x memory savings)
  • Sync primitive tracking (Mutex, Channel, WaitGroup)
  • Production-quality race reports with stack traces

Performance

  • Hot path overhead: 15-22%
  • Real-world workloads: 368-809ns/op
  • Zero allocations on hot paths
  • Scalability: 1000+ goroutines tested

Statistics

  • 22,273 lines of Go code (production + tests + examples)
  • 25,601 lines of documentation
  • 67+ tests passing (100% pass rate)
  • 45-86% test coverage

Project Timeline

  • Path A: November 19, 2025 (target: December 15, 2025)
  • Path B: January 31, 2026 (planned)
  • Go Proposal: Q2 2026 (planned)