feat(csharp): add StragglerDownloadDetector for identifying slow downloads #39
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…loads - Implement core detection algorithm using median throughput analysis - Configurable multiplier (default 1.5x slower than median) - Minimum completion quantile (default 60%) - Straggler padding grace period (default 5 seconds) - Sequential fallback threshold tracking - Duplicate detection prevention via tracking dictionary - Add comprehensive unit tests for parameter validation, median calculation, and edge cases Builds on: stack/straggler-metrics
This was referenced Nov 17, 2025
jadewang-db
requested changes
Nov 17, 2025
| /// <param name="stragglerDetectionPadding">Extra buffer time before declaring a download as a straggler.</param> | ||
| /// <param name="maxStragglersBeforeFallback">Maximum stragglers before triggering sequential fallback.</param> | ||
| public StragglerDownloadDetector( | ||
| double stragglerThroughputMultiplier, |
Collaborator
There was a problem hiding this comment.
should we use a struggler download config object?
| /// <param name="alreadyCounted">Dictionary to track already counted stragglers (prevents duplicate counting).</param> | ||
| /// <returns>Collection of file offsets identified as stragglers.</returns> | ||
| public IEnumerable<long> IdentifyStragglerDownloads( | ||
| IReadOnlyList<FileDownloadMetrics> allDownloadMetrics, |
Collaborator
There was a problem hiding this comment.
should we make these 2 list part of the detector?
| @@ -0,0 +1,219 @@ | |||
| /* | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
Summary
Implements core straggler detection algorithm that analyzes download metrics and identifies abnormally slow downloads using median throughput-based analysis.
Detection Algorithm:
Key Changes:
StragglerDownloadDetectorclass implementation