Skip to content

feat: add new shell operator with compaction options#646

Merged
ldmonster merged 2 commits intomainfrom
feat/shell-queue-rework-compaction
Aug 8, 2025
Merged

feat: add new shell operator with compaction options#646
ldmonster merged 2 commits intomainfrom
feat/shell-queue-rework-compaction

Conversation

@timmilesdw
Copy link
Contributor

@timmilesdw timmilesdw commented Aug 5, 2025

Overview

Updates shell-operator dependency to the latest version that introduces a new high-performance linked list-based task queue implementation with intelligent task compaction capabilities.

What this PR does / why we need it

Performance Improvements:

  • 160-200x faster AddFirst operations - critical for priority task handling
  • 6-53x faster GetByID operations - essential for debugging and monitoring
  • 1.8x faster task compaction with 31x better memory efficiency
  • Consistent O(1) performance for all queue operations regardless of queue size

New Features:

  • Intelligent task compaction - merges HookRun tasks for the same hook to prevent queue bloat
  • Object pooling - reduces GC pressure in high-throughput scenarios
  • Memory optimization - up to 31x less memory usage during compaction operations

Why this matters:

  • Eliminates performance spikes during high-priority task insertion
  • Improves debugging efficiency with fast task lookup by ID
  • Reduces memory footprint and GC pressure in production environments
  • Provides stable, predictable performance as queue size grows

Backward Compatibility:

  • Existing slice-based queue implementation remains available
  • No breaking changes to public APIs

Special notes for your reviewer

  • All existing tests pass with the new implementation
  • Performance improvements are most noticeable in high-load scenarios

@timmilesdw timmilesdw added the enhancement New feature or request label Aug 5, 2025
@timmilesdw timmilesdw self-assigned this Aug 5, 2025
@timmilesdw timmilesdw force-pushed the feat/shell-queue-rework-compaction branch from d0b4618 to 6ea499c Compare August 5, 2025 13:53
@timmilesdw timmilesdw marked this pull request as ready for review August 5, 2025 13:54
@timmilesdw timmilesdw force-pushed the feat/shell-queue-rework-compaction branch 19 times, most recently from 50e86c9 to 0209937 Compare August 6, 2025 16:30
Signed-off-by: Timur Tuktamyshev <timur.tuktamyshev@flant.com>
@timmilesdw timmilesdw force-pushed the feat/shell-queue-rework-compaction branch from 0209937 to 73ff2fe Compare August 6, 2025 17:57
Signed-off-by: Pavel Okhlopkov <pavel.okhlopkov@flant.com>
@ldmonster ldmonster merged commit 5b4657f into main Aug 8, 2025
8 of 9 checks passed
@ldmonster ldmonster deleted the feat/shell-queue-rework-compaction branch August 8, 2025 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants