Skip to content

I/O Timeouts can cause error and audit ingestion to silently stop#4792

Merged
ramonsmits merged 1 commit intorelease-5.11from
hanging-ingestion-5.11
Feb 10, 2025
Merged

I/O Timeouts can cause error and audit ingestion to silently stop#4792
ramonsmits merged 1 commit intorelease-5.11from
hanging-ingestion-5.11

Conversation

@ramonsmits
Copy link
Member

…ich can cause the ingestion to never used incoming context tasks and hang.

- Rectored to use BackgroundService
- Fix `catch (OperationCancelledException)` blocks to add `when` guards - only ignore cancellations set by caller
- Overriding Start and Stop to support graceful shutdown and improve intent and readability
- Ensure TrySetException is always set on exception
- Logging improvements around cancellation to inform that tasks got cancelled. Important when shutting down to diagnose where teardown "hangs".
@ramonsmits ramonsmits added this to the 5.11.4 milestone Feb 10, 2025
@ramonsmits ramonsmits changed the title https://github.com/Particular/ServiceControl/pull/4780 OperationCancelledException can causes hanging ingestion, unnoticed failing tasks, and no logging of timeouts Feb 10, 2025
@ramonsmits ramonsmits merged commit dd05a74 into release-5.11 Feb 10, 2025
31 checks passed
@ramonsmits ramonsmits deleted the hanging-ingestion-5.11 branch February 10, 2025 13:07
@ramonsmits ramonsmits added the Bug label Feb 11, 2025
@andreasohlund andreasohlund changed the title OperationCancelledException can causes hanging ingestion, unnoticed failing tasks, and no logging of timeouts I/O Timeouts can cause error and audit ingestion to silently stop. Feb 11, 2025
@ramonsmits ramonsmits changed the title I/O Timeouts can cause error and audit ingestion to silently stop. I/O Timeouts can cause error and audit ingestion to silently stop Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants