Guest User

Untitled

a guest
Jul 18th, 2018
99
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.51 KB | None | 0 0
  1. OS: Centos
  2. Requisite: JAVA 7+8
  3.  
  4. sudo vi /etc/profile
  5. export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
  6. export JRE_HOME=/usr/lib/jvm/jre
  7.  
  8. Download Kafka from any of the given website: the one use here is kafka_2.11-0.10.1.1.tgz
  9. Untar: tar -xvf kafka_2.11-0.10.1.1.tgz
  10. sudo mv kafka_2.11-0.10.1.1.tgz /opt
  11.  
  12. To operate anything on Kafka go to its folder
  13. cd /opt/kafka_2.11-0.10.1.1
  14.  
  15. Requisites for kafka to setup and work:
  16. Zookeeper and Kafka broker setup
  17.  
  18. Start zookeeper
  19. $ bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
  20.  
  21. Configure Kafka server in kafka-server-start.sh
  22. Modify KAFKA_HEAP_OPTS = "-Xmx1G -Xms1G" to higher if more memory is required
  23.  
  24.  
  25. Kafka- server is ready to be started by specifying the brokers
  26. Multiple Brokers can be created by having multiple server.properties.
  27.  
  28. Start the kafka server -- which means to say have multiple brokers
  29. $ bin/kafka-server-start.sh --daemon config/server.properties [--daemon to run as daemon process]
  30. $ bin/kafka-server-start.sh --daemon config/server-1.properties
  31. $ bin/kafka-server-start.sh --daemon config/server-2.properties
  32.  
  33. Create a topic and notify to the zookeeper
  34.  
  35. $ bin/kafka-topics.sh --create--zookeeper localhost:2181 --topic cdpTopic --replication-factor 3 --partitions 1 --config min.insync.replicas=3
  36.  
  37.  
  38. Just check whether the topic is created by listing the topics
  39. $ bin/kafka-topics.sh --list --zookeeper localhost:2181
  40.  
  41. Host a kafka consumer in a separate terminal.
  42. $ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --from-beginning
Add Comment
Please, Sign In to add comment