Skip to content

Set up a local kafka cluster

Jump to bottom
Viswanath Jayachandran edited this page Feb 21, 2016 · 7 revisions

Kafka installation

Download kafka from its official website. The latest version of kafka was 0.9.0.0 at the time of writing so I've extracted kafka_2.11-0.9.0.0.tgz to install kafka locally on my machine under ~/tools/. Kafka comes with a bundled zookeeper so that's taken care for us.

  • Start bundled zookeeper server by pointing to its configuration file. cd tools/kafka_2.11-0.9.0.0/ ./bin/zookeeper-server-start.sh config/zookeeper.properties

  • Start default kafka broker. Broker would connect to zookeeper and that'd be evident in log file entries cd tools/kafka_2.11-0.9.0.0/ ./bin/kafka-server-start.sh config/server.properties

  • Create a topic named demo with 1 partition and replication factor 1 cd tools/kafka_2.11-0.9.0.0/ ./bin/kafka-topics.sh --create --topic demo --zookeeper localhost:2181 --partitions 1 --replication-factor 1

  • Consume from the topic using console consumer ./bin/kafka-console-consumer.sh --topic demo --zookeeper localhost:2181

  • Populate the topic using console producer (Write something and hit ENTER) cd tools/kafka_2.11-0.9.0.0/ ./bin/kafka-console-producer.sh --topic demo --broker-list localhost:9092

Hello world This is kafka and it works Now, isn't that absolutely fantastic?

Clone this wiki locally