high cpu for low traffic topics #8
Description
Love the websocket gateway. It's allowing us to make use of kafka across the open internet protected in a TLS tunnel.
I'm seeing pretty high CPU utilization for a topic with very, very low message rates ( < 1msg/sec.) Our setup is that we have ~70 clients all in their own consumer group since we want to broadcast messages. I have a feeling the cpu is due to the MaxWait of 100ms and having so many consumer groups.
TRACE 22:54:55,331 [ConsumerFetcherThread-imgix-worker-0062_svab3-1408575091046-704ad865-0-104], issuing to broker 104 of fetch request Name: FetchRequest; Version: 0; CorrelationId: 2028; ClientId: imgix-worker-0062-ConsumerFetcherThread-imgix-worker-0062_svab3-1408575091046-704ad865-0-104; ReplicaId: -1; MaxWait: 100 ms; MinBytes: 1 bytes; RequestInfo: [purge.origin,0] -> PartitionFetchInfo(496,1048576)
Could you expose these consumer config params to the command line or through the URL?