MaxBSONSize error from Mongo DB - Marking as Unrecoverable error to stop retry

[jeevamanivel]

Mongo DB accepts only max 16MB data for BSON. When BSON size is more than 16MB, Mongo DB returns error 'Mongo::Error::MaxBSONSize - The document exceeds maximum allowed BSON object size after serialization'.

This error cannot be recovered, but Fluentd tries to retry these message since it doesn't know that this is Unrecoverable error. Please mark this as Unrecoverable error, so that fluentd will move the buffer to backup path & won't retry.

Fix:
File : out_mongo.rb

Function : def operate(database, collection, records)

rescue Mongo::Error::BulkWriteError => e
log.warn "#{records.size - e.result["n_inserted"]} documents are not inserted. Maybe these documents are invalid as a BSON."
forget_collection(collection)
rescue Mongo::Error::MaxBSONSize => e
log.warn e
raise Fluent::UnrecoverableError, "MaxBSONSize error"
rescue ArgumentError => e
log.warn e
end