Skip to content

Conversation

julienrbrt
Copy link
Member

@julienrbrt julienrbrt commented Oct 6, 2025

Overview

Use sync/atomic` instead of mutexes.

Copy link
Contributor

Summary of Changes

Hello @julienrbrt, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a performance enhancement by refactoring critical state management within the block execution and submission components. It replaces traditional mutex-based synchronization with Go's sync/atomic package for specific variables. This change aims to reduce contention and improve the efficiency of concurrent access to these frequently updated values, leading to better overall system performance without compromising data integrity.

Highlights

  • Performance Optimization: Replaced sync.RWMutex with sync/atomic primitives for managing lastState in the Executor component and daIncludedHeight in the Submitter component to improve concurrency performance.
  • Executor State Management: The lastState field in the Executor struct was changed from a types.State protected by a sync.RWMutex to an *atomic.Pointer[types.State], allowing for lock-free updates and reads.
  • Submitter DA Height Management: The daIncludedHeight field in the Submitter struct was changed from a uint64 protected by a sync.RWMutex to an *atomic.Uint64, enabling efficient atomic operations for this counter.
  • API Updates: The GetLastState/SetLastState methods in Executor and GetDAIncludedHeight/SetDAIncludedHeight methods in Submitter were updated to utilize the Load and Store methods of the new atomic types.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

github-actions bot commented Oct 6, 2025

The latest Buf updates on your PR. Results from workflow CI and Release / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedOct 6, 2025, 3:53 PM

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: Performance Optimization using sync/atomic

Overview

This PR replaces mutex-based synchronization with atomic operations for two hot-path state variables. This is a solid performance optimization that reduces lock contention.


Strengths

  1. Appropriate Use of Atomics: Replacing RWMutex with atomic.Pointer and atomic.Uint64 is correct here since:

    • lastState and daIncludedHeight are accessed frequently (hot path)
    • Operations are simple load/store without complex logic under lock
    • No composite operations that require mutex protection
  2. Proper Nil Handling: Good defensive programming in GetLastState():

    if state == nil {
        return types.State{}
    }

    This prevents panics if called before initialization.

  3. Consistent Pattern: Both components (Executor and Submitter) follow the same refactoring pattern, maintaining code consistency.

  4. Test Coverage Updated: Test files properly updated to use atomic.Uint64 and atomic.Pointer.


⚠️ Potential Issues & Concerns

1. Memory Allocation on Every Store (Minor Performance Concern)

Location: block/internal/executing/executor.go:165

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Takes address of parameter
}

Issue: Taking the address of a parameter (&state) forces the state to be heap-allocated on every call. Since types.State contains:

  • time.Time (struct with int64 + pointer)
  • []byte slices (AppHash, LastResultsHash)
  • Several uint64 fields

This creates a new heap allocation per update, which may cause GC pressure in high-throughput scenarios.

Recommendation: Since State is already being copied when passed by value, consider pre-allocating or using a different pattern if this becomes a bottleneck. However, this is likely acceptable for the current use case where state updates are relatively infrequent.

2. Race Condition in GetLastState (Critical - Potential Bug)

Location: block/internal/executing/executor.go:154-161

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // ⚠️ Returns dereferenced value
}

Issue: While the atomic load is safe, the subsequent dereference and return creates a shallow copy of the State struct. The types.State struct contains slice fields (AppHash []byte, LastResultsHash Hash which is []byte):

type State struct {
    // ... other fields
    LastResultsHash Hash    // type Hash []byte
    AppHash         []byte
}

Problem: When you return *state, Go copies the struct, but slices are copied by reference. This means:

  1. Thread A calls GetLastState() and gets a copy with slice references
  2. Thread B calls SetLastState() with a new state (different slices)
  3. Thread A's slices are still valid BUT if the underlying arrays are shared or modified elsewhere, there could be subtle bugs

Impact: Low in this codebase since slices appear to be treated as immutable, but it violates the principle of returning truly independent copies.

Recommendation:

  • Document that returned State slices should be treated as read-only, OR
  • Consider deep-copying slices if mutation is possible:
func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    stateCopy := *state
    // Deep copy slices if needed
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    return stateCopy
}

3. Missing Race Detector Test (Test Coverage Gap)

The PR doesn't include explicit race condition tests. While the existing tests should catch obvious races if run with -race, consider adding:

# Verify no races introduced
go test -race ./block/internal/executing/...
go test -race ./block/internal/submitting/...

🔒 Security Considerations

  • No security concerns identified
  • Atomic operations are memory-safe
  • No exposure of sensitive data

🚀 Performance Analysis

Expected Improvements:

  • Reduced lock contention on state reads (previously required RLock)
  • Better cache performance (atomics use less memory than mutex)
  • No syscalls for simple loads (vs potential scheduler interaction with mutexes)

Benchmarking Recommendation: Consider adding a benchmark to quantify improvements:

func BenchmarkGetLastState(b *testing.B) {
    // Setup executor...
    b.RunParallel(func(pb *testing.PB) {
        for pb.Next() {
            _ = exec.GetLastState()
        }
    })
}

📋 Action Items

Before Merge:

  1. ✅ Run tests with race detector: make test -race
  2. ⚠️ Decide on slice handling in GetLastState() - document immutability assumption or add deep copy
  3. 📝 Consider adding a comment about heap allocation trade-off in SetLastState()

Optional:

  • Add concurrent access benchmark to validate performance gains
  • Document the concurrency guarantees in the function comments

Summary

This is a well-executed optimization that appropriately uses atomics over mutexes. The main concern is the slice aliasing in GetLastState() which should be addressed through either documentation or defensive copying. Once the slice handling is clarified, this PR should be safe to merge.

Recommendation: ✅ Approve with minor fixes


🤖 Generated with Claude Code

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a performance optimization that replaces sync.RWMutex with sync/atomic for managing state in Executor and Submitter. The changes are well-implemented and correctly use atomic operations for uint64 and struct pointers. I have one suggestion for Executor.GetLastState to improve thread safety by preventing potential data races that could occur if callers modify the returned state object. This is a pre-existing condition, but worth addressing during this refactoring.

Copy link

codecov bot commented Oct 6, 2025

Codecov Report

❌ Patch coverage is 80.00000% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.17%. Comparing base (b646e66) to head (6cac29f).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
block/internal/executing/executor.go 55.00% 8 Missing and 1 partial ⚠️
block/internal/syncing/syncer.go 88.88% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2735      +/-   ##
==========================================
- Coverage   62.28%   62.17%   -0.12%     
==========================================
  Files          79       79              
  Lines        8490     8497       +7     
==========================================
- Hits         5288     5283       -5     
- Misses       2711     2721      +10     
- Partials      491      493       +2     
Flag Coverage Δ
combined 62.17% <80.00%> (-0.12%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review - PR #2735: Use sync/atomic instead of mutexes

Overall, this is a solid performance optimization that replaces mutex-based synchronization with atomic operations for simple state getters/setters. The changes are well-implemented and follow Go best practices.

✅ Strengths

  1. Appropriate Use Case: Both lastState in Executor and daIncludedHeight in Submitter are perfect candidates for atomic operations - they're simple reads/writes that don't require coordination with other state.

  2. Correct Implementation:

    • atomic.Uint64 for daIncludedHeight is straightforward and correct
    • atomic.Pointer[types.State] for lastState properly handles the struct state
    • Nil check in GetLastState() prevents panic on uninitialized state
  3. Performance Benefits:

    • Eliminates mutex contention on hot paths (these getters/setters are likely called frequently)
    • Lock-free reads are significantly faster than mutex locks, especially under concurrent load
    • Reduced goroutine scheduling overhead
  4. Test Coverage: All test files properly updated with atomic.Uint64 initialization

🔍 Potential Issues & Recommendations

1. Memory Allocation in SetLastState() (Minor)

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Stores pointer to parameter
}

Issue: This stores a pointer to the state parameter. If the caller modifies the state variable after calling SetLastState, it could lead to data races since GetLastState dereferences the pointer.

Recommendation: Create a copy to ensure isolation:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state
    e.lastState.Store(&stateCopy)
}

Location: block/internal/executing/executor.go:164

2. Race Condition in GetLastState() (Low Severity)

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // ⚠️ Dereferences pointer
}

Issue: While the atomic load is safe, dereferencing the pointer creates a copy of the entire types.State struct. Between the load and the dereference, the underlying memory could theoretically be modified if SetLastState is called concurrently. However, this is only a concern if Go's GC or another goroutine could modify the memory.

Analysis: In practice, this is likely safe because:

  • Once stored, the pointer isn't modified (new calls to SetLastState create new pointers)
  • The struct copy happens atomically in Go

Recommendation: Consider this acceptable, but document the thread-safety contract. Alternatively, return a pointer to avoid the copy overhead:

func (e *Executor) GetLastState() types.State {
    if state := e.lastState.Load(); state != nil {
        return *state
    }
    return types.State{}
}

3. Initialization Consideration

The nil check in GetLastState() handles uninitialized state well. However, ensure that SetLastState() is always called during initialization before any concurrent access begins. Review initializeState() to confirm this.

Location: block/internal/executing/executor.go:177-199

4. Test Coverage Gap

While tests were updated correctly, there don't appear to be any race detection tests or concurrent access tests for these specific methods.

Recommendation: Consider adding a test like:

func TestExecutor_ConcurrentStateAccess(t *testing.T) {
    // Run with -race flag
    exec := &Executor{lastState: &atomic.Pointer[types.State]{}}
    
    var wg sync.WaitGroup
    for i := 0; i < 100; i++ {
        wg.Add(2)
        go func() {
            defer wg.Done()
            exec.SetLastState(types.State{LastBlockHeight: uint64(i)})
        }()
        go func() {
            defer wg.Done()
            _ = exec.GetLastState()
        }()
    }
    wg.Wait()
}

🎯 Performance Impact

Expected Improvements:

  • Read-heavy workloads: 10-100x faster (lock-free reads vs. mutex)
  • Write-heavy workloads: 2-5x faster (atomic CAS vs. mutex)
  • Reduced latency: No goroutine blocking/scheduling overhead

🔒 Security

No security concerns identified. The atomic operations maintain the same thread-safety guarantees as the previous mutex implementation.

📋 Minor Nits

  • Consider adding a comment explaining why atomics are used instead of mutexes for future maintainers
  • The PR description could mention the performance motivation more explicitly

✅ Verdict

Approve with minor suggestion: The SetLastState parameter aliasing issue should be addressed to prevent potential data races. Otherwise, this is a well-executed optimization.

Suggested action: Make a copy in SetLastState() before storing, then this is ready to merge.


Review performed by Claude Code following ev-node coding standards

Copy link
Contributor

claude bot commented Oct 6, 2025

PR Review: perf(block): use sync/atomic instead of mutexes

Summary

This PR refactors two critical components (Executor and Submitter) to use sync/atomic operations instead of mutex-protected primitives for better performance. The changes are well-structured and follow Go best practices for lock-free programming.

✅ Code Quality & Best Practices

Strengths:

  • Clean conversion from mutex-protected fields to atomic operations
  • Proper nil-checking in GetLastState() to handle uninitialized state
  • Consistent pattern applied across both components
  • Test updates reflect the architectural changes appropriately
  • Follows the existing codebase patterns and zero-dependency core package approach

Observations:

  • The change from lastState types.State + RWMutex to *atomic.Pointer[types.State] is semantically correct
  • The change from daIncludedHeight uint64 + RWMutex to *atomic.Uint64 is straightforward and safe

🔍 Potential Issues

1. Critical: Pointer Escape and State Mutation ⚠️

In block/internal/executing/executor.go:152-159, GetLastState() returns a dereferenced copy:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // Returns a copy
}

However, SetLastState() stores a pointer to the argument:

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Stores pointer to function parameter
}

Problem: The parameter state is allocated on the stack. Taking its address and storing it in an atomic pointer can lead to the pointer escaping to the heap, but more critically, if the caller still holds a reference to the original types.State value and mutates it, this could create a data race.

Current usage in executor.go:201 and executor.go:604:

e.SetLastState(state)  // 'state' is a local variable

This is actually safe in the current usage because:

  1. The state variable is created locally and not reused after calling SetLastState()
  2. The copy-on-return in GetLastState() prevents external mutation

However, this pattern is fragile and could break if future code modifies state after calling SetLastState().

Recommendation: For better safety and clarity, allocate explicitly:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state  // Explicit copy
    e.lastState.Store(&stateCopy)
}

2. Memory Ordering Concerns

The types.State struct contains non-atomic fields including:

  • time.Time (which contains a pointer internally)
  • Hash (byte slice, which is a header + pointer)
  • Embedded Version struct

While atomic.Pointer ensures atomicity for the pointer itself, it doesn't guarantee visibility of the pointed-to value's internal state across goroutines without additional synchronization.

Current safety: The code is safe because:

  1. GetLastState() returns a copy (not a pointer), preventing shared mutable state
  2. SetLastState() creates a new allocation each time
  3. The types.State values are effectively immutable after creation

Risk: If future changes modify GetLastState() to return a pointer (for performance), or if types.State fields are mutated after creation, you could have data races.

3. Performance Consideration

The nil-check in GetLastState() adds a branch on every read:

state := e.lastState.Load()
if state == nil {
    return types.State{}
}
return *state

Impact: Minor. The nil case should only happen during initialization before initializeState() completes. Modern CPUs handle this well with branch prediction.

Suggestion: If you want to optimize further, ensure lastState is initialized immediately in NewExecutor() to avoid the nil check:

func NewExecutor(...) (*Executor, error) {
    // ... validation ...
    
    e := &Executor{
        // ... other fields ...
        lastState: &atomic.Pointer[types.State]{},
        // ...
    }
    // Initialize with zero value
    e.lastState.Store(&types.State{})
    return e, nil
}

Then simplify GetLastState():

func (e *Executor) GetLastState() types.State {
    return *e.lastState.Load()  // No nil check needed
}

🔒 Security Concerns

None identified. The changes don't introduce security vulnerabilities. The atomic operations maintain data integrity, and there's no exposure of sensitive data.

⚡ Performance Considerations

Improvements:

  • ✅ Eliminates lock contention on GetLastState() reads (previously used RWMutex.RLock())
  • ✅ Eliminates lock contention on GetDAIncludedHeight() reads
  • ✅ Both fields are read frequently (e.g., in executor.go:221, executor.go:294, executor.go:432, executor.go:521)

Trade-offs:

  • SetLastState() now allocates a new types.State on the heap each time (vs. copying into a mutex-protected field)
  • For a struct with ~100 bytes, this is negligible compared to lock acquisition costs
  • Modern allocators handle small, short-lived allocations efficiently

Overall: Significant net performance improvement, especially under read-heavy workloads.

🧪 Test Coverage

Coverage Assessment:

  • ✅ Test updates correctly reflect the structural changes
  • ✅ Tests in submitter_test.go properly initialize atomic.Uint64
  • ⚠️ Missing: No new tests verify concurrent access patterns or race conditions

Recommendation: Add a race detector test:

func TestExecutor_ConcurrentStateAccess(t *testing.T) {
    // Test concurrent GetLastState/SetLastState calls
    // Run with: go test -race
}

📋 Additional Suggestions

  1. Consistency Check: The Syncer component (block/internal/syncing/syncer.go:152-155) still uses RWMutex for lastState. Consider applying the same optimization there for consistency.

  2. Documentation: Add a comment explaining why GetLastState() returns a copy:

    // GetLastState returns a copy of the current state.
    // The copy ensures the returned state is safe from concurrent modification.
    func (e *Executor) GetLastState() types.State {
  3. Benchmark: Before merging, run benchmarks comparing mutex vs atomic performance:

    go test -bench=. -benchmem ./block/internal/...

✅ Conclusion

This is a well-executed performance optimization with clear benefits. The primary concern is the pointer storage pattern in SetLastState(), which is currently safe but could be made more robust. The changes follow Go's atomic programming idioms correctly.

Recommendation: Approve with minor suggestions

The code is ready to merge after addressing the SetLastState() pointer storage pattern for better long-term safety, though it's functionally correct as-is.

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: Performance Optimization with sync/atomic

Summary

This PR replaces mutex-based synchronization with sync/atomic for two simple shared state variables, which is a good performance optimization for read-heavy concurrent access patterns.


✅ Strengths

  1. Appropriate use case: Both lastState and daIncludedHeight are good candidates for atomic operations:

    • Simple getter/setter patterns
    • Read-heavy workloads (multiple GetLastState() calls per write)
    • No complex critical sections requiring multiple operations
  2. Clean implementation: The changes are minimal and focused, affecting only the necessary code paths

  3. Test coverage maintained: All test files properly updated with the new atomic types


⚠️ Issues & Concerns

1. Critical: Potential Data Race with State.AppHash (block/internal/executing/executor.go:154-159)

The types.State struct contains AppHash []byte, which is a reference type. When using atomic.Pointer, the struct copy is atomic, but the slice itself can still be modified concurrently, leading to data races.

Current code:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // ⚠️ Returns copy with shared AppHash slice
}

Problem scenario:

  1. Thread A calls GetLastState() and gets a State copy
  2. Thread B calls SetLastState() with new state containing modified AppHash
  3. Thread A's AppHash slice could be concurrently accessed/modified

Recommended fix:
Either deep-copy the slice in GetLastState():

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    stateCopy := *state
    if state.AppHash != nil {
        stateCopy.AppHash = make([]byte, len(state.AppHash))
        copy(stateCopy.AppHash, state.AppHash)
    }
    return stateCopy
}

Or document that State.AppHash must be treated as immutable after creation.

2. Bug: SetLastState creates pointer to local variable (block/internal/executing/executor.go:163-165)

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Pointer to parameter
}

Problem: The parameter state is a local variable. While Go's escape analysis will move it to the heap, this creates unnecessary allocations on every call. More importantly, if a caller passes the same State object multiple times with modifications, this could lead to unexpected behavior.

Recommended fix:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state  // Explicit copy
    e.lastState.Store(&stateCopy)
}

Or better yet, change the signature to accept a pointer:

func (e *Executor) SetLastState(state *types.State) {
    e.lastState.Store(state)
}

3. Performance consideration: Mutex might still be faster

For the lastState case, the types.State struct is 96+ bytes (considering all fields). According to types/state.go:17-36:

  • Version (2 uint64s = 16 bytes)
  • ChainID (string = 16 bytes)
  • Integers (4 × 8 = 32 bytes)
  • Time (24 bytes)
  • Hash (8+ bytes)
  • AppHash slice (8 bytes pointer + capacity)

Atomic pointer operations require copying the entire struct plus heap allocation for each write, while RWMutex allows multiple concurrent readers without copying.

Recommendation: Profile both approaches under realistic load. For large structs with read-heavy patterns, RWMutex may actually be faster due to:

  • No allocation on reads
  • No copying on reads
  • Better CPU cache behavior

🔍 Security & Best Practices

  1. No security concerns - Changes don't introduce security vulnerabilities
  2. Nil handling: Good defensive programming with the nil check in GetLastState()
  3. Follows project conventions: Changes align with Go best practices for atomic operations

📝 Testing Recommendations

  1. Add race detector tests: Per CLAUDE.md, run:

    go test -race ./block/internal/executing/...
    go test -race ./block/internal/submitting/...
  2. Add concurrent access test: Verify correctness under heavy concurrent load:

    func TestExecutor_ConcurrentStateAccess(t *testing.T) {
        // Test concurrent GetLastState/SetLastState calls
    }
  3. Benchmark comparison: Add benchmarks comparing mutex vs atomic approaches:

    func BenchmarkGetLastState_Atomic(b *testing.B)
    func BenchmarkGetLastState_Mutex(b *testing.B)

📊 Verdict

CHANGES REQUESTED - The optimization is conceptually sound, but needs fixes for:

  1. AppHash slice data race protection
  2. SetLastState parameter handling
  3. Performance validation via benchmarks

Once these issues are addressed, this will be a solid performance improvement for the daIncludedHeight case (simple uint64) and potentially for lastState if the slice concern is resolved.

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: Performance Optimization with sync/atomic

Thank you for this performance optimization PR! The change from mutexes to atomic operations is generally a good improvement for simple read/write operations. Here's my detailed review:

Strengths

  1. Performance Improvement: Using atomic.Pointer[T] and atomic.Uint64 eliminates lock contention for simple get/set operations, which should improve performance in high-concurrency scenarios.

  2. Consistent Pattern: The PR applies the same optimization pattern to both executor.go and submitter.go, maintaining consistency across the codebase.

  3. Test Coverage: The tests have been updated to reflect the new atomic-based implementation, ensuring the changes are properly tested.

  4. Correct Atomic Usage: Both atomic.Pointer[types.State] and atomic.Uint64 are used appropriately for their respective data types.

⚠️ Potential Issues & Concerns

1. Critical Bug: Race Condition in SetLastState (executor.go:165)

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ❌ DANGEROUS: storing pointer to stack variable
}

Problem: The state parameter is passed by value. Taking its address (&state) creates a pointer to a stack-allocated variable that will be invalid once the function returns. This can lead to:

  • Accessing freed memory
  • Data corruption
  • Undefined behavior

Solution: Store a copy of the state on the heap:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state
    e.lastState.Store(&stateCopy)
}

2. Inconsistency with Syncer Implementation

Looking at block/internal/syncing/syncer.go, the Syncer component still uses sync.RWMutex for lastState (lines 50-51, 153-163) while using atomics for daHeight. This creates an inconsistency:

  • Executor: Uses atomic.Pointer[types.State] (this PR)
  • Syncer: Uses sync.RWMutex for lastState
  • Submitter: Uses atomic.Uint64 for daIncludedHeight (this PR)

Recommendation: Either update Syncer to use atomics as well, or document why different synchronization mechanisms are used in different components.

3. Memory Ordering Concerns

The types.State struct contains a time.Time field which itself contains unexported fields. While atomic operations provide memory ordering guarantees, the struct copy semantics might not be fully safe without additional synchronization for complex field types.

Current State struct (types/state.go):

type State struct {
    ChainID         string
    InitialHeight   uint64
    LastBlockHeight uint64
    LastBlockTime   time.Time  // Contains internal mutex/monotonic clock
    DAHeight        uint64
    AppHash         Hash
    // ...
}

Since time.Time contains a monotonic clock reading (as of Go 1.9+), copying it should be safe, but this warrants verification with race detector testing.

📋 Recommendations

  1. Fix the critical bug in SetLastState immediately - this is a correctness issue that could cause crashes or data corruption.

  2. Run with race detector: Execute tests with -race flag to catch any remaining concurrency issues:

    make test-race  # or similar command from your Makefile
    go test -race ./block/internal/executing/...
    go test -race ./block/internal/submitting/...
  3. Consider updating Syncer for consistency, or document the rationale for different approaches.

  4. Benchmark the change: While atomics should be faster, it would be valuable to see actual benchmark numbers:

    go test -bench=. -benchmem ./block/internal/executing/...
    go test -bench=. -benchmem ./block/internal/submitting/...
  5. Add a comment explaining why atomics were chosen over mutexes for these specific fields (e.g., "Using atomics for lock-free reads in hot paths").

🔒 Security Considerations

No security concerns identified. The atomic operations maintain thread-safety for the affected fields.

🧪 Testing

The test updates look correct. Consider adding a specific concurrency test to verify the atomic operations work correctly under high contention:

func TestExecutor_ConcurrentStateAccess(t *testing.T) {
    // Test concurrent GetLastState/SetLastState calls
}

📊 Performance Considerations

  • ✅ Reduced lock contention for read-heavy workloads
  • ✅ No blocking readers when writers update state
  • ⚠️ Atomic pointer operations are still slightly more expensive than direct field access
  • ✅ For daIncludedHeight (uint64), atomic operations are very efficient

Verdict

This is a good optimization, but the critical bug in SetLastState must be fixed before merging. Once that's addressed and race detector tests pass, this PR will be a solid performance improvement.


Review completed following ev-node coding standards and best practices

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: Performance Optimization with sync/atomic

Summary

This PR replaces mutex-based synchronization with sync/atomic for state management in the Executor, Submitter, and Syncer components. Overall, this is a positive change that reduces lock contention and improves performance. The implementation is mostly sound, but there are critical issues that need to be addressed.


🚨 Critical Issues

1. Data Race in GetLastState() - BLOCKER

Location: executor.go:154-165, syncer.go:151-162

Issue: The GetLastState() method has a critical data race:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }

    stateCopy := *state  // 🚨 Race condition here!
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)

    return stateCopy
}

Problem: After loading the pointer, another goroutine could call SetLastState() and store a new pointer. When you dereference *state, you're reading from memory that could be concurrently modified, causing a data race.

Evidence: The PR adds -race flag to test-cover target, which suggests race detection is now being run. This implementation will fail race detection.

Fix: You need to ensure the entire copy operation is atomic:

func (e *Executor) GetLastState() types.State {
    for {
        state := e.lastState.Load()
        if state == nil {
            return types.State{}
        }

        stateCopy := *state
        stateCopy.AppHash = bytes.Clone(state.AppHash)
        stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)

        // Verify pointer didn't change during copy
        if e.lastState.Load() == state {
            return stateCopy
        }
        // Retry if it changed
    }
}

Alternative approach: Store the entire State by value in a single atomic operation using sync/atomic.Value, though this requires careful handling of the slice fields.


2. Potential Memory Leak in SetLastState()

Location: executor.go:168-170, syncer.go:165-167

Issue:

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // 🚨 Storing pointer to parameter!
}

Problem: You're storing a pointer to the function parameter state. If the caller modifies the types.State struct after calling SetLastState(), it will cause concurrent modification issues. Additionally, the old state is never freed, potentially causing memory accumulation.

Fix: Create a heap-allocated copy:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    e.lastState.Store(&stateCopy)
}

⚠️ High Priority Issues

3. Inconsistent Atomic Pattern

Location: syncer.go:171-177

Issue: The Syncer uses atomic.LoadUint64/StoreUint64 for daHeight but atomic.Pointer for lastState. This inconsistency is confusing.

Recommendation: For consistency, use atomic.Uint64 for daHeight like in Submitter:

daHeight *atomic.Uint64

Then use daHeight.Load() and daHeight.Store() instead of the package-level functions.


4. Missing Race Detection in Regular Tests

Location: scripts/test.mk:39

Issue: Only test-cover runs with -race flag. Regular make test doesn't enable race detection.

Recommendation: Add race detection to the main test target as well, or create a dedicated test-race target that developers can run before submitting PRs.


✅ Positive Aspects

  1. Performance Win: Replacing mutexes with atomics for simple read/write operations is the right approach and will significantly reduce lock contention in high-throughput scenarios.

  2. Correct Usage in Submitter: The daIncludedHeight atomic implementation is clean and correct.

  3. Proper Test Updates: Test files were correctly updated to use atomic.Uint64 instead of mutex-based initialization.

  4. Defensive Copying: The attempt to clone byte slices (AppHash, LastResultsHash) in GetLastState() shows good awareness of shared state issues, though the implementation needs fixing (see issue ABCI interface #1).


🔍 Minor Issues

5. Nil Check May Be Unnecessary

If you initialize lastState properly in initializeState() before any goroutines access it, the nil check in GetLastState() may be redundant. However, keeping it is defensive and fine.


📊 Test Coverage

Missing Tests: I don't see specific tests for concurrent access patterns to GetLastState()/SetLastState(). Given this is a concurrency-focused change, you should add:

  1. A test that calls GetLastState() and SetLastState() concurrently from multiple goroutines
  2. Run with -race to verify no data races
  3. Test with high contention to verify the performance improvement

Example:

func TestExecutor_ConcurrentStateAccess(t *testing.T) {
    // Setup executor...
    
    var wg sync.WaitGroup
    for i := 0; i < 100; i++ {
        wg.Add(2)
        go func(i int) {
            defer wg.Done()
            state := types.State{LastBlockHeight: uint64(i)}
            exec.SetLastState(state)
        }(i)
        go func() {
            defer wg.Done()
            _ = exec.GetLastState()
        }()
    }
    wg.Wait()
}

🛡️ Security Considerations

No security vulnerabilities identified. The change is internal to state management and doesn't expose new attack surfaces.


🎯 Performance Considerations

Expected Benefits:

  • Reduced lock contention in read-heavy workloads
  • Better cache locality with atomic operations
  • Lower latency for state queries

Potential Concerns:

  • The retry loop I suggested in issue ABCI interface #1 could add latency under extreme write contention (though this should be rare)
  • Memory usage might increase slightly if old states aren't GC'd promptly, but Go's GC should handle this

📋 Recommendations

Before merging:

  1. ✅ Fix the data race in GetLastState() (Critical)
  2. ✅ Fix the memory issue in SetLastState() (Critical)
  3. ✅ Add concurrent access tests with -race enabled
  4. ⚠️ Consider making daHeight usage consistent in Syncer
  5. ⚠️ Run make test-cover with the -race flag to verify no races

Code Style: Follows Go conventions and project patterns well. The changes align with the project's modular architecture.


Conclusion

This is a valuable performance optimization, but the current implementation has critical data races that will cause issues in production. Please address issues #1 and #2 before merging. Once fixed, this will be a solid improvement to the codebase.

Great work on identifying this optimization opportunity! 🚀

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: perf(block): use sync/atomic instead of mutexes

This is a solid performance optimization that appropriately replaces mutex-based synchronization with atomic operations. Overall the changes are well-executed, but I've identified some critical issues that should be addressed.


🔴 Critical Issues

1. Data Race: Slice Aliasing in GetLastState()

Location: block/internal/executing/executor.go:154-164 and block/internal/syncing/syncer.go:149-159

Problem: The types.State struct contains slice fields (AppHash []byte, LastResultsHash Hash). When GetLastState() returns *state, it creates a shallow copy where the slices are copied by reference, not value.

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // ⚠️ AppHash and LastResultsHash are shared with internal state
}

Impact: If a caller mutates the returned AppHash or LastResultsHash slices, it will modify the internally stored state, creating a data race and violating the expected immutability.

Fix Required:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    
    stateCopy := *state
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    return stateCopy
}

Apply the same fix to syncer.go:149-159.

2. Heap Allocation: Pointer to Function Parameter

Location: block/internal/executing/executor.go:167 and block/internal/syncing/syncer.go:165

Problem: SetLastState() stores a pointer to the function parameter:

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Forces heap allocation of parameter
}

While this works (Go's escape analysis moves state to the heap), it:

  • Creates unnecessary heap allocations on every call
  • Could lead to unexpected behavior if callers reuse the same State variable

Recommended Fix:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state
    e.lastState.Store(&stateCopy)
}

This makes the heap allocation explicit and prevents potential aliasing issues.


⚠️ Important Observations

3. Missing Deep Copy in SetLastState

To fully prevent aliasing issues, SetLastState should also deep-copy the slices:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    e.lastState.Store(&stateCopy)
}

This ensures complete isolation between the stored state and caller's data.

4. Race Detector Testing

I notice the PR adds -race to test-cover target in scripts/test.mk:39, which is excellent! Make sure to run:

make test-cover
make test-integration -race

to verify no races are introduced.


✅ Strengths

  1. Appropriate optimization: Both lastState and daIncludedHeight are accessed frequently, making atomics a good fit
  2. Reduced lock contention: Eliminates RWMutex overhead on hot paths
  3. Consistent pattern: Same approach applied to both Executor and Submitter
  4. Good nil handling: Defensive check prevents panics on uninitialized state
  5. Test coverage: All tests properly updated

📊 Performance Considerations

Expected improvements:

  • 10-100x faster reads (lock-free vs mutex acquisition)
  • 2-5x faster writes (atomic CAS vs mutex)
  • Reduced goroutine scheduling overhead
  • Better performance under concurrent load

Trade-off:

  • Small heap allocation per SetLastState call (negligible compared to lock costs)

🔒 Security & Code Quality

  • Security: No vulnerabilities introduced ✅
  • Code style: Follows project conventions per CLAUDE.md ✅
  • Error handling: Appropriate error handling maintained ✅
  • Logging: No impact on logging ✅

📝 Recommendations

Required before merge:

  1. ✅ Fix slice aliasing in GetLastState() (both Executor and Syncer)
  2. ✅ Make heap allocation explicit in SetLastState()
  3. ✅ Run make test-cover with race detector

Optional improvements:

  • Consider deep-copying slices in SetLastState() for complete isolation
  • Add benchmark tests to quantify performance gains
  • Add comment documenting the thread-safety contract

Summary

This is a well-designed optimization that will deliver meaningful performance improvements. The atomic operations are used correctly, but the slice aliasing issue must be addressed to prevent potential data races. Once the critical fixes are applied, this PR is ready to merge.

Verdict: ⚠️ Changes requested (slice aliasing fix required)


🤖 Generated with Claude Code

@julienrbrt julienrbrt requested a review from alpe October 6, 2025 11:12
@julienrbrt
Copy link
Member Author

julienrbrt commented Oct 6, 2025

benchmark.go
// BenchmarkGetLastState measures the performance of reading state
func BenchmarkGetLastState(b *testing.B) {
	state := types.State{
		Version:         types.Version{Block: 1, App: 1},
		LastBlockHeight: 100,
		AppHash:         []byte("apphash1234567890"),
		LastResultsHash: []byte("results1234567890"),
	}

	e := &Executor{
		lastState: &atomic.Pointer[types.State]{},
	}
	e.lastState.Store(&state)

	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			_ = e.GetLastState()
		}
	})
}

// BenchmarkStateReadWrite_Mixed benchmarks mixed read/write workload (90% reads, 10% writes)
func BenchmarkStateReadWrite_Mixed(b *testing.B) {
	state := types.State{
		Version:         types.Version{Block: 1, App: 1},
		LastBlockHeight: 100,
		AppHash:         []byte("apphash1234567890"),
		LastResultsHash: []byte("results1234567890"),
	}

	e := &Executor{
		lastState: &atomic.Pointer[types.State]{},
	}
	e.lastState.Store(&state)

	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		i := 0
		for pb.Next() {
			if i%10 == 0 {
				// 10% writes
				newState := state
				newState.LastBlockHeight = uint64(i)
				e.SetLastState(newState)
			} else {
				// 90% reads
				_ = e.GetLastState()
			}
			i++
		}
	})
}
// BenchmarkGetLastState_WithMutex compares against mutex-based implementation
func BenchmarkGetLastState_WithMutex(b *testing.B) {
	state := types.State{
		Version:         types.Version{Block: 1, App: 1},
		LastBlockHeight: 100,
		AppHash:         []byte("apphash1234567890"),
		LastResultsHash: []byte("results1234567890"),
	}

	type mutexExecutor struct {
		lastState    types.State
		lastStateMtx sync.RWMutex
	}

	e := &mutexExecutor{lastState: state}

	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			e.lastStateMtx.RLock()
			s := e.lastState
			e.lastStateMtx.RUnlock()
			_ = s
		}
	})
}

// BenchmarkStateReadWrite_Mixed_WithMutex compares mixed workload against mutex
func BenchmarkStateReadWrite_Mixed_WithMutex(b *testing.B) {
	state := types.State{
		Version:         types.Version{Block: 1, App: 1},
		LastBlockHeight: 100,
		AppHash:         []byte("apphash1234567890"),
		LastResultsHash: []byte("results1234567890"),
	}

	type mutexExecutor struct {
		lastState    types.State
		lastStateMtx sync.RWMutex
	}

	e := &mutexExecutor{lastState: state}

	b.ResetTimer()
	b.RunParallel(func(pb *testing.PB) {
		i := 0
		for pb.Next() {
			if i%10 == 0 {
				// 10% writes
				e.lastStateMtx.Lock()
				e.lastState.LastBlockHeight = uint64(i)
				e.lastStateMtx.Unlock()
			} else {
				// 90% reads
				e.lastStateMtx.RLock()
				_ = e.lastState
				e.lastStateMtx.RUnlock()
			}
			i++
		}
	})
}
goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkGetLastState-12    	71430471	        18.99 ns/op	      48 B/op	       2 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	1.384s

goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkStateReadWrite_Mixed-12    	40779867	        25.97 ns/op	      56 B/op	       1 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	1.109s

goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkGetLastState_WithMutex-12    	 8119104	       150.4 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	1.381s

goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkStateReadWrite_Mixed_WithMutex-12    	49345582	        79.34 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	3.952s

Allocations are due to the deep copy, suggested by Claude. I personally think we can remove it, we do not mutate current state. We create a new one each block with currentState.NextState(header, newAppHash) (wdyt @alpe)

Removing it gives as well obviously an even better result:

goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkGetLastState-12    	1000000000	         0.05766 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	0.078s

goos: linux
goarch: arm64
pkg: github.com/evstack/ev-node/block/internal/executing
BenchmarkStateReadWrite_Mixed-12    	130789657	         9.246 ns/op	      12 B/op	       0 allocs/op
PASS
ok  	github.com/evstack/ev-node/block/internal/executing	2.150s

@alpe
Copy link
Contributor

alpe commented Oct 6, 2025

Thanks for adding some benchmark results! There is a significant difference without the deep copy. When I looked at the code, there is no mutation indeed but with public methods this is not guaranteed to be true in the future. The GetLastState is called by component by tests. I guess you can not simply make them private.
But you can provide 2 methods. A safe one which clones the data and the fast one that is used internally. WDYT?

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: Performance Optimization with sync/atomic

Thank you for this performance optimization PR! The migration from mutexes to sync/atomic for managing shared state is a good direction. Here's my detailed feedback:

✅ Strengths

  1. Performance improvement: Using atomic.Pointer and atomic.Uint64 eliminates lock contention for read-heavy workloads, which should improve throughput
  2. Consistent pattern: The change is applied uniformly across Executor, Syncer, and Submitter
  3. Good test coverage: Tests were updated appropriately in submitter_test.go
  4. Race detection enabled: Adding -race flag to test coverage is excellent for catching concurrency issues

🔴 Critical Issues

1. Data Race in GetLastState() - BLOCKING

Location: block/internal/executing/executor.go:154-164 and block/internal/syncing/syncer.go:151-161

The current implementation has a critical data race:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    
    stateCopy := *state  // Race condition here!
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    
    return stateCopy
}

Problem: While the pointer load is atomic, dereferencing (*state) is not atomic. The types.State struct contains multiple fields including slices. If another goroutine calls SetLastState() after Load() but during the dereference, you get a race on reading the slice headers and other fields.

Why bytes.Clone doesn't help: Cloning happens after the struct copy, so the race already occurred when reading the slice pointers from the original struct.

Solution: Store the entire state including cloned slices:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := types.State{
        Version:         state.Version,
        ChainID:         state.ChainID,
        InitialHeight:   state.InitialHeight,
        LastBlockHeight: state.LastBlockHeight,
        LastBlockTime:   state.LastBlockTime,
        DAHeight:        state.DAHeight,
        AppHash:         bytes.Clone(state.AppHash),
        LastResultsHash: bytes.Clone(state.LastResultsHash),
    }
    e.lastState.Store(&stateCopy)
}

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    return *state  // Safe now - stored state already has cloned slices
}

This ensures the stored pointer always points to an immutable, fully independent copy.

2. Missing time.Time handling

time.Time contains unexported fields and should be copied by value, not by cloning. The current approach is correct for time.Time, but make sure the struct copy in the solution above handles it properly (it does, since Go copies time.Time by value correctly).

⚠️ Moderate Issues

3. Test coverage for concurrent access

While existing tests were updated, I don't see new tests specifically validating concurrent access patterns. Consider adding:

func TestExecutor_GetLastStateConcurrent(t *testing.T) {
    exec := setupExecutor(t)
    
    var wg sync.WaitGroup
    for i := 0; i < 100; i++ {
        wg.Add(2)
        go func(height uint64) {
            defer wg.Done()
            state := exec.GetLastState()
            state.AppHash = append(state.AppHash, byte(height))
            exec.SetLastState(state)
        }(uint64(i))
        
        go func() {
            defer wg.Done()
            _ = exec.GetLastState()
        }()
    }
    wg.Wait()
}

Run with -race to ensure no data races.

4. Performance benchmarks missing

For a performance-focused PR, benchmarks would help quantify the improvement. Consider adding:

func BenchmarkGetLastState(b *testing.B) {
    exec := setupExecutor(b)
    b.RunParallel(func(pb *testing.PB) {
        for pb.Next() {
            _ = exec.GetLastState()
        }
    })
}

💡 Minor Suggestions

  1. Consider documenting the concurrency model: Add a comment explaining that GetLastState() returns a snapshot that's safe to use without locks:
// GetLastState returns a snapshot of the current state.
// The returned state is safe to use without additional synchronization
// as it contains deep copies of all mutable fields.
func (e *Executor) GetLastState() types.State
  1. PR description: The overview could be more detailed about why this change improves performance (reduced lock contention, better cache locality for atomic ops, etc.)

📋 Summary

  • Block merge: Yes - critical data race must be fixed
  • Required changes: Fix the race condition in GetLastState()/SetLastState()
  • Recommended: Add concurrent access tests and benchmarks
  • Overall direction: Excellent optimization effort, just needs the race condition resolved

The migration to atomics is the right approach for this use case, but the implementation needs adjustment to be truly race-free.

Copy link
Contributor

claude bot commented Oct 6, 2025

Code Review: PR #2735 - Use sync/atomic instead of mutexes

Summary

This PR replaces sync.RWMutex with sync/atomic.Pointer for state management and sync/atomic.Uint64 for DA height tracking. Overall, this is a good performance optimization, but there are critical data race issues that need to be addressed.


Critical Issues

1. Data Race in GetLastState() - CRITICAL 🔴

Location: block/internal/executing/executor.go:154-164 and block/internal/syncing/syncer.go:152-163

Problem: The new implementation has a subtle but critical data race:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    
    stateCopy := *state  // Shallow copy
    stateCopy.AppHash = bytes.Clone(state.AppHash)
    stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)
    
    return stateCopy
}

The issue: While AppHash and LastResultsHash are cloned, the shallow copy at line 160 creates a race condition because types.State contains other reference types like LastBlockTime (which contains internal pointers in time.Time) and potentially Version.

Why this is dangerous:

  • The shallow copy stateCopy := *state copies the struct, but reference fields still point to the original data
  • time.Time has internal pointer fields that could be mutated during concurrent access
  • Between loading the pointer and cloning the slices, another goroutine could call SetLastState() and store a completely different state
  • This violates Go's race detector rules and can cause data corruption

Solution:
Either:

  1. Perform a full deep copy of ALL fields including LastBlockTime
  2. Return the pointer directly and document that callers must not mutate it
  3. Use a proper read-copy-update (RCU) pattern with immutable states

Example fix:

func (e *Executor) GetLastState() types.State {
    state := e.lastState.Load()
    if state == nil {
        return types.State{}
    }
    
    // Deep copy all fields
    return types.State{
        Version:         state.Version,  // Assuming Version is a value type
        ChainID:         state.ChainID,
        InitialHeight:   state.InitialHeight,
        LastBlockHeight: state.LastBlockHeight,
        LastBlockTime:   state.LastBlockTime,  // time.Time copy is safe
        DAHeight:        state.DAHeight,
        AppHash:         bytes.Clone(state.AppHash),
        LastResultsHash: bytes.Clone(state.LastResultsHash),
    }
}

2. Pointer Escape in SetLastState() - HIGH 🟠

Location: block/internal/executing/executor.go:167 and block/internal/syncing/syncer.go:159

func (e *Executor) SetLastState(state types.State) {
    e.lastState.Store(&state)  // ⚠️ Taking address of parameter
}

Problem: Taking the address of a function parameter is dangerous:

  • The caller might reuse the state variable after calling this function
  • This creates aliasing where multiple pointers reference the same underlying data
  • Mutations to the caller's state variable will affect the stored state
  • This defeats the purpose of using atomics for thread-safety

Solution:

func (e *Executor) SetLastState(state types.State) {
    stateCopy := state  // Create copy on heap
    e.lastState.Store(&stateCopy)
}

Or better, change the API to accept a pointer and document the ownership transfer:

// SetLastState takes ownership of the provided state pointer.
// Caller must not modify the state after calling this function.
func (e *Executor) SetLastState(state *types.State) {
    e.lastState.Store(state)
}

Code Quality Issues

3. Missing Race Detector Testing 🟡

The change to scripts/test.mk adds -race flag to coverage tests, which is good:

-	@go run -tags=cover scripts/test_cover.go
+	@go run -tags=cover -race scripts/test_cover.go

Recommendation: Also add race detection to regular tests:

test:
	@go test -race ./...

4. Inconsistent Pattern Usage 🟡

The PR uses atomic.Pointer[types.State] for state but doesn't apply the same pattern to daHeight in syncer.go:

  • syncer.go:54 still uses daHeight uint64 with atomic.LoadUint64/StoreUint64 (old-style atomics)
  • submitter.go correctly uses daIncludedHeight *atomic.Uint64 (new-style)

Recommendation: For consistency, also convert syncer.go:54 to use *atomic.Uint64:

daHeight *atomic.Uint64

Performance Considerations

5. Good: Reduced Lock Contention

The change from RWMutex to atomic.Pointer is excellent for performance:

  • No lock contention on reads
  • Significantly faster in high-concurrency scenarios
  • Better cache performance (no mutex state)

6. Concern: Memory Allocation on Every Read 🟡

The current GetLastState() implementation allocates a new types.State and clones byte slices on every call:

stateCopy := *state
stateCopy.AppHash = bytes.Clone(state.AppHash)
stateCopy.LastResultsHash = bytes.Clone(state.LastResultsHash)

Impact:

  • If GetLastState() is called frequently (e.g., in hot paths), this could increase GC pressure
  • Consider adding metrics or benchmarks to validate this doesn't regress performance

Recommendation: Profile the hot paths and consider using a sync.Pool for temporary state copies if this becomes a bottleneck.


Security Considerations

7. Good: Deep Copy Prevents Mutation

Cloning AppHash and LastResultsHash prevents external mutation of critical cryptographic hashes - this is the right approach for security.


Test Coverage

8. Adequate Test Updates

The test updates in submitter_test.go correctly initialize the new atomic.Uint64:

-	s := &Submitter{store: st, daStateMtx: &sync.RWMutex{}, logger: zerolog.Nop()}
+	s := &Submitter{store: st, daIncludedHeight: &atomic.Uint64{}, logger: zerolog.Nop()}

Recommendation: Add specific tests for concurrent access patterns to verify the atomic operations work correctly under race conditions.


Action Items

Must Fix (Blocking):

  1. ✅ Fix data race in GetLastState() - implement proper deep copy
  2. ✅ Fix pointer escape in SetLastState() - don't take address of parameter

Should Fix (Recommended):
3. Make daHeight in syncer.go use *atomic.Uint64 for consistency
4. Add concurrent access tests with -race flag
5. Add benchmarks to validate performance improvement

Nice to Have:
6. Document thread-safety guarantees in godoc comments
7. Consider adding a bytes.Clone helper in a shared utility package


Conclusion

The performance optimization is sound in principle, but the implementation has critical correctness issues that could cause data races and undefined behavior. The pointer aliasing and shallow copy issues must be resolved before merging.

Once the critical issues are fixed, this will be a solid performance improvement for the block management system.

Overall Assessment: ⚠️ Needs Changes - Do not merge until data race issues are resolved.

Copy link
Contributor

@alpe alpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💪 Well done

@julienrbrt julienrbrt added this pull request to the merge queue Oct 7, 2025
Merged via the queue into main with commit 528081b Oct 7, 2025
28 of 29 checks passed
@julienrbrt julienrbrt deleted the julien/atomic branch October 7, 2025 08:03
@github-project-automation github-project-automation bot moved this to Done in Evolve Oct 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants