Warn when split drops event.

[cnico]

Hello,

I have recently upgraded from logstash 2.3.1 to 5.1.1.
It took me a while to identify bugs in my logstash filter ruby code for a complex data to handle.

I faced a problem where I was sending data and nothing was going out as if the event was dropped while it is filtered.
After removing each line of logstash configuration, I finally found the problem was in the filter split.
And with some logs added in the split.rb file, I finally found that the problem is in this line of the plugin :

# Skip filtering if splitting this event resulted in only one thing found.
return if splits.length == 1 && original_value.is_a?(String)
#or splits[1].empty?

Actually my event stops at the return and nothing more happens : the event filter is stopped and there is nothing in the output even in debug mode.

So I created this issue in order to improve the split plugin to add a warning or debug log before the return and the drop of the event. It can save lots of searching time.

Regards,

[jordansissel]

It seems like a bug, not a feature, if this filter ever drops an event.

On the other hand, return if ... does not drop the event. The original event is cancelled at the end of the method after the new events are created by splitting. It's possible there's a bug here, but I'm not seeing where it is based on your description.

Can you show your Logstash configuration and describe what events are being lost?

[cnico]

Here is an extract of my configuration :

ruby {
	    code => "
                               unless event.get('[SUBEVT]').is_a?(Array) then 
					event.set('[SUBEVT]', Array.new);
				end;
...
                           " 
}
if [SUBEVT] { 
		split {
			field => "SUBEVT"
			add_tag => "event_splitted"
		}
...

The SUBEVT exists (because of a mistake I corrected since it was not normal) and is empty. My original event containing the SUBEVT is simply cancelled without information and I spent a lot of time finding where it comes from.

If you want to have a look at my full logstash configuration, I uploaded here : elastic/logstash#6496.