Implement a strategy to handle OOM in direct memory by andsel · Pull Request #475 · logstash-plugins/logstash-input-beats

Avoid to run Beats parser and Beats protocol handler in separate executors group (beatsHandlerExecutorGroup)

andsel

committed

Ported core part of original #410 PR

andsel

committed

Added pull way of fetching data

andsel

committed

Extracted the pull mode part into a separate handler

andsel

committed

Update the flow control handler to avoid new reads if the channel becomes not writable, to excert backpressure to the sender system

andsel

committed

Separated the logic to drop incoming connections into specific handler

andsel

committed

Fist draft of the integration test

andsel

committed

Removed tests using EmbeddedChannel because doesn't manage the writeable status due to that offload every message to the outbound list

andsel

committed

Reshaped the asynch code to be more linear

andsel

committed

Covered the number of connection limiter with unit test

andsel

committed

Pessimistic remediation, when a direct OOM happens close the channel

andsel

committed

Removed from the log string any reference to Filebeat

andsel

committed

Raised up the log level level when dropping connections becuase of thundering and going to OOM condition

andsel

committed

Better actionable suggestion to user in case of OOM

andsel

and

jsvd

committed

Updated OOMConnectionCloser to monitor the consumption of memory also during the read opeartion and not only on exception

andsel

committed

Re-introduce the beats handlers worker group to separata the Beats protocol processing from the boss group that accepts and listed to new sockets

andsel

committed

Added feature flag named protect_direct_memory to control the usage of OOM checking or not. Enabled by default.

andsel

committed

Throw a configuration error if Netty reserved direct memory is not anough (256MB)

andsel

committed

Add missed shutdown of beat's worker loop

andsel

committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Implement a strategy to handle OOM in direct memory#475

Implement a strategy to handle OOM in direct memory#475
andsel wants to merge 19 commits intologstash-plugins:mainfrom
andsel:mitigate_thundering_herd_2

Commits on Sep 27, 2023

Avoid to run Beats parser and Beats protocol handler in separate executors group (beatsHandlerExecutorGroup)

Ported core part of original #410 PR

Added pull way of fetching data

Extracted the pull mode part into a separate handler

Update the flow control handler to avoid new reads if the channel becomes not writable, to excert backpressure to the sender system

Separated the logic to drop incoming connections into specific handler

Fist draft of the integration test

Removed tests using EmbeddedChannel because doesn't manage the writeable status due to that offload every message to the outbound list

Reshaped the asynch code to be more linear

Covered the number of connection limiter with unit test

Pessimistic remediation, when a direct OOM happens close the channel

Removed from the log string any reference to Filebeat

Raised up the log level level when dropping connections becuase of thundering and going to OOM condition

Better actionable suggestion to user in case of OOM

Updated OOMConnectionCloser to monitor the consumption of memory also during the read opeartion and not only on exception

Re-introduce the beats handlers worker group to separata the Beats protocol processing from the boss group that accepts and listed to new sockets

Added feature flag named protect_direct_memory to control the usage of OOM checking or not. Enabled by default.

Throw a configuration error if Netty reserved direct memory is not anough (256MB)

Add missed shutdown of beat's worker loop

Uh oh!

Implement a strategy to handle OOM in direct memory#475andsel wants to merge 19 commits intologstash-plugins:mainlogstash-plugins/logstash-input-beats:mainfrom andsel:mitigate_thundering_herd_2andsel/logstash-input-beats:mitigate_thundering_herd_2Copy head branch name to clipboard

Commits

Commits on Sep 27, 2023

Implement a strategy to handle OOM in direct memory#475
andsel wants to merge 19 commits intologstash-plugins:mainfrom
andsel:mitigate_thundering_herd_2