Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- OS: Centos
- Requisite: JAVA 7+8
- sudo vi /etc/profile
- export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
- export JRE_HOME=/usr/lib/jvm/jre
- Download Kafka from any of the given website: the one use here is kafka_2.11-0.10.1.1.tgz
- Untar: tar -xvf kafka_2.11-0.10.1.1.tgz
- sudo mv kafka_2.11-0.10.1.1.tgz /opt
- To operate anything on Kafka go to its folder
- cd /opt/kafka_2.11-0.10.1.1
- Requisites for kafka to setup and work:
- Zookeeper and Kafka broker setup
- Start zookeeper
- $ bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
- Configure Kafka server in kafka-server-start.sh
- Modify KAFKA_HEAP_OPTS = "-Xmx1G -Xms1G" to higher if more memory is required
- Kafka- server is ready to be started by specifying the brokers
- Multiple Brokers can be created by having multiple server.properties.
- Start the kafka server -- which means to say have multiple brokers
- $ bin/kafka-server-start.sh --daemon config/server.properties [--daemon to run as daemon process]
- $ bin/kafka-server-start.sh --daemon config/server-1.properties
- $ bin/kafka-server-start.sh --daemon config/server-2.properties
- Create a topic and notify to the zookeeper
- $ bin/kafka-topics.sh --create--zookeeper localhost:2181 --topic cdpTopic --replication-factor 3 --partitions 1 --config min.insync.replicas=3
- Just check whether the topic is created by listing the topics
- $ bin/kafka-topics.sh --list --zookeeper localhost:2181
- Host a kafka consumer in a separate terminal.
- $ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --from-beginning
Add Comment
Please, Sign In to add comment