Loosing data

[javaenthugm]

Hi ,

We are loosing lots of data points while using this plugin.

I don't see any error in the region server logs or hdfs logs.

I have enabled some more logging in the plugin code and I think, the plugin is not able to read all messages when the through-put is high.

The TSDB plugin configuration is given below.

KafkaRpcPlugin.groups=tsdbpublisher
KafkaRpcPlugin.tsdbpublisher.topics=xxxxx.data.10m.tsdb
KafkaRpcPlugin.tsdbpublisher.consumerType = raw
KafkaRpcPlugin.tsdbpublisher.deserializer = net.opentsdb.data.deserializers.JSONDeserializer
KafkaRpcPlugin.tsdbpublisher.rate = 10000
KafkaRpcPlugin.tsdbpublisher.threads = 16

Any suggestions for tuning?

Thanks,
-Gk

[javaenthugm]

@manolama - any suggestions?

[javaenthugm]

@manolama - Nobody is looking in these issues?
Should we try some alternative solution than this plugin?

[manolama]

Hi sorry. So you'll want to enable debug logging and track the stats to see what kind of exception is being thrown and incremented.