Debuggability for OOMs due to split explosions

[lucabelluccini]

Ideas to make the Split filter debuggable

• Add a trace logger which logs the exact count of items before the actual split

• Add a warning logger which logs only if:

  • by default, if the items being generated >= bulk size
  • configurable by the split filter itself (via a parameter e.g. warn_threshold)

• Collect metrics of min/max/average or an histogram of number of split events and report them in the diagnostics

• Have an optional circuit breaker limit which would not split the events if they exceed a threshold and tag the events as split_failed or split_limited