Advertisement
Guest User

Untitled

a guest
Nov 25th, 2015
68
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.76 KB | None | 0 0
  1. # Commands to check various states of Kafka
  2.  
  3. > NOTE: Code blocks indicate input on prompt with a '$'. Everything else is output
  4.  
  5.  
  6. ## Create a new topic
  7. ```
  8. $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --create --partitions 1 --replication-factor 1
  9. Created topic "my_topic".
  10. ```
  11.  
  12. ## List all topics
  13. ```
  14. $ /opt/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181__consumer_offsets
  15. bro_raw
  16. my_topic
  17. ```
  18.  
  19. ## List topic details
  20. ```
  21. $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --describe
  22. Topic:my_topic PartitionCount:1 ReplicationFactor:1 Configs:
  23. Topic: my_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
  24. ```
  25.  
  26. ## Change number of partitions
  27. ```
  28. $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --alter --partitions 4
  29. WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
  30. Adding partitions succeeded!
  31.  
  32. $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --describe
  33. Topic:my_topic PartitionCount:4 ReplicationFactor:1 Configs:
  34. Topic: my_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
  35. Topic: my_topic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
  36. Topic: my_topic Partition: 2 Leader: 0 Replicas: 0 Isr: 0
  37. Topic: my_topic Partition: 3 Leader: 0 Replicas: 0 Isr: 0
  38. ```
  39.  
  40. ## Check topic offsets
  41.  
  42. Ideally the lag is 0 or close to it. The `Offset` field indicates the last record that the consumer read (in this case, logstash). The `logSize` indicates how many total records are in the topic partition. The `lag` indicates the difference between the two and should approach 0.
  43.  
  44. ```
  45. $ /opt/kafka/bin/kafka-consumer-offset-checker.sh --zookeeper 127.0.0.1:2181 --topic bro_raw --group logstash
  46. Group Topic Pid Offset logSize Lag Owner
  47. logstash bro_raw 0 2642920 2642920 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  48. logstash bro_raw 1 221536 221536 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  49. logstash bro_raw 2 86 86 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  50. logstash bro_raw 3 438 438 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  51. logstash bro_raw 4 0 0 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  52. logstash bro_raw 5 0 0 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
  53. ```
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement