Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 26, 2026

Fix SSE integration test comparison logic

Following the STC reference test pattern and user guidance. All review feedback addressed.

Key Insight:
Insert/Remove operations (operations 1, 2, 3) test rollback mechanisms but DON'T change the final quote data. As the user pointed out: "If you remove a quote then add that same quote back, you obviously shouldn't attempt to add it back again."

Changes Made:

  • allQuotesWithOperations in StcHub test: Simple copy of allQuotes (no modifications)
  • allQuotesWithRevisions in AllHubs test: Copy of allQuotes with ONLY the index 1600 revision applied
  • Using TakeLast(cacheSize) pattern from reference test
  • All 79 indicator comparisons use this pattern

Operations Analysis:

  1. Operation 1 (Insert(allQuotes[80])): No-op - quote already in list
  2. Operation 2 (RemoveAt(100) then Insert(allQuotes[100])): No-op - remove then re-add same quote
  3. Operation 3 (AllHubs: revision at index 1600): ACTUAL change - apply to allQuotesWithRevisions
  4. Operation 4 (AllHubs: revisions at last quote): Final revision back to original - no net change

Review Feedback Addressed:

  • Added parameter validation for SetMinCacheSize
  • Fixed tail replacement state restoration bug
  • Fixed self-fulfilling assertion in pruning test
  • Added fail-fast for malformed SSE payloads
  • Added Remove actions to SSE server rollback scenarios

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits January 26, 2026 22:17
Both SSE integration tests were incorrectly comparing streaming results
to series computed on only the final cached quotes. This fails to account
for how streaming indicators process ALL quotes (including pruned ones)
and maintain running state.

Fixed by:
- Computing series on the FULL quote list with all revisions applied
- Truncating series results to match final cache size using TakeLast()
- Matching the pattern from STC WithCachePruning_MatchesSeriesExactly

This ensures tests validate that streaming indicators correctly maintain
state even after cache pruning removes early quotes.

Co-authored-by: DaveSkender <[email protected]>
Changed from collection expression spread to new List<Quote>(allQuotes)
for better performance with large quote lists (2000+ quotes).

Co-authored-by: DaveSkender <[email protected]>
Copilot AI changed the title [WIP] Fix logic validation issues in stream hub pruning tests Fix series comparison logic in stream hub pruning tests Jan 26, 2026
Copilot AI requested a review from DaveSkender January 26, 2026 22:19
@DaveSkender DaveSkender marked this pull request as ready for review January 26, 2026 23:58
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 26, 2026

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • 🔍 Trigger a full review
📝 Walkthrough

Walkthrough

This pull request replaces the public StreamHub.Insert() method with Add() as the primary API for adding late-arriving data, deprecates the Insert method, removes the Insert enum member from Act, and introduces a new MinCacheSize property to track minimum cache retention requirements. A protected InsertWithoutRebuild() method is added for internal cache operations. Documentation, migration guides, and hundreds of test files are updated to reflect the API change. The SSE server is extended with scenario support for testing rollback and pruning behaviors.

Possibly related issues

  • Stock.Indicators#1936 — Main changes introduce MinCacheSize property and update cache pruning tests to validate cache size after processing, which directly addresses the request for standardized cache-pruning tests across StreamHub implementations.

Possibly related PRs

  • Stock.Indicators#1922 — Both PRs modify the audit script (tools/scripts/audit-streamhub.sh) to update provider-history test detection patterns, changing from Insert to Add method checks.
  • Stock.Indicators#1897 — Both PRs modify StreamHub interfaces and observable types (IStreamHub, IStreamObservable, BaseProvider, StreamHub.Observable.cs) for cache access and subscription handling.
  • Stock.Indicators#1906 — Both PRs modify QuotePart.StreamHub.cs code paths, adding or changing OnAdd behavior and InsertWithoutRebuild logic that depends on this PR's API refactoring.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@DaveSkender

This comment was marked as outdated.

…ering

Tests still failing - need to resolve timestamp alignment between streaming
results and series results after pruning. The fundamental issue is that
TakeLast doesn't account for insert/remove operations changing which quotes
are in the cache.

Current approach using timestamp filtering is getting warmup nulls included.
Need to filter by streaming hub result timestamps, not quote cache timestamps.

Co-authored-by: DaveSkender <[email protected]>

This comment was marked as outdated.

Reverted to original test from commit 3f24bf2. Analysis shows:
- StcHub test passes with original logic
- AllHubs test fails due to EPMA floating point precision issues (not test logic)
- Differences are at 15th decimal place, caused by quote revision at index 1600
- Core test logic is sound; need precision tolerance adjustment for EPMA

Co-authored-by: DaveSkender <[email protected]>
Copilot AI changed the title Fix series comparison logic in stream hub pruning tests Investigate and document StreamHub test failures - EPMA and Slope precision issues Jan 27, 2026
@DaveSkender

This comment was marked as outdated.

Implemented the correct comparison pattern following the reference test:
- Compute series on FULL quote list
- Use TakeLast or timestamp filtering to match cache size
- Applied revisions (index 1600 modification) to quote list

Current status:
- StcHub test: FAILING - Insert operations change cache composition
- AllHubs test: Using TakeLast pattern with revisions applied

The STC reference test works because it does a simple Add() with no Insert/Remove
operations. The SSE tests use Insert() to add earlier quotes back to the cache,
which creates a mismatch between TakeLast expectations and actual cache composition.

Need guidance on whether to:
1. Modify SSE test to remove Insert operations
2. Use different comparison logic for tests with Insert/Remove

Co-authored-by: DaveSkender <[email protected]>

This comment was marked as outdated.

Copilot AI changed the title Investigate and document StreamHub test failures - EPMA and Slope precision issues Fix SSE integration test comparison logic to use correct series pattern Jan 27, 2026
DaveSkender

This comment was marked as outdated.

Following user guidance to insert quotes to allQuotesWithRevisions before computing
series. However, tests still failing because Insert operations of earlier quotes
(80, 100, 500) cause those quotes to be in the cache with warmup null values from
their original processing, while series calculation sees them past warmup with values.

Current approach:
- Built allQuotesWithRevisions including all modifications
- Applied revision at index 1600 for AllHubs test
- Using TakeLast on quote cache size

Issue: Streaming hub "remembers" that quote 80 was processed during warmup (null),
but series processes all quotes together so quote 80 is past warmup (has value).
This creates a fundamental mismatch that TakeLast doesn't resolve.

Need clarification on how to handle Insert operations that add earlier quotes
back to cache with their original (null) values.

Co-authored-by: DaveSkender <[email protected]>
coderabbitai[bot]

This comment was marked as resolved.

@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 09:17 — with GitHub Actions Inactive
coderabbitai[bot]

This comment was marked as resolved.

…#1957)

Signed-off-by: Dave Skender <[email protected]>
Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: DaveSkender <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 09:37 — with GitHub Actions Inactive
@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 09:45 — with GitHub Actions Inactive
@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 09:57 — with GitHub Actions Inactive
coderabbitai[bot]

This comment was marked as resolved.

@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 10:12 — with GitHub Actions Inactive
coderabbitai[bot]

This comment was marked as resolved.

@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 10:31 — with GitHub Actions Inactive
@DaveSkender DaveSkender temporarily deployed to stockindicators.dev February 2, 2026 10:36 — with GitHub Actions Inactive
@DaveSkender DaveSkender merged commit 8041814 into test-pruning Feb 2, 2026
2 of 6 checks passed
@DaveSkender DaveSkender deleted the copilot/sub-pr-1937 branch February 2, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

2 participants