Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- # Commands to check various states of Kafka
- > NOTE: Code blocks indicate input on prompt with a '$'. Everything else is output
- ## Create a new topic
- ```
- $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --create --partitions 1 --replication-factor 1
- Created topic "my_topic".
- ```
- ## List all topics
- ```
- $ /opt/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181__consumer_offsets
- bro_raw
- my_topic
- ```
- ## List topic details
- ```
- $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --describe
- Topic:my_topic PartitionCount:1 ReplicationFactor:1 Configs:
- Topic: my_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
- ```
- ## Change number of partitions
- ```
- $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --alter --partitions 4
- WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
- Adding partitions succeeded!
- $ /opt/kafka/bin/kafka-topics.sh --topic my_topic --zookeeper localhost:2181 --describe
- Topic:my_topic PartitionCount:4 ReplicationFactor:1 Configs:
- Topic: my_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
- Topic: my_topic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
- Topic: my_topic Partition: 2 Leader: 0 Replicas: 0 Isr: 0
- Topic: my_topic Partition: 3 Leader: 0 Replicas: 0 Isr: 0
- ```
- ## Check topic offsets
- Ideally the lag is 0 or close to it. The `Offset` field indicates the last record that the consumer read (in this case, logstash). The `logSize` indicates how many total records are in the topic partition. The `lag` indicates the difference between the two and should approach 0.
- ```
- $ /opt/kafka/bin/kafka-consumer-offset-checker.sh --zookeeper 127.0.0.1:2181 --topic bro_raw --group logstash
- Group Topic Pid Offset logSize Lag Owner
- logstash bro_raw 0 2642920 2642920 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- logstash bro_raw 1 221536 221536 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- logstash bro_raw 2 86 86 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- logstash bro_raw 3 438 438 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- logstash bro_raw 4 0 0 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- logstash bro_raw 5 0 0 0 logstash_lab003.ds.mocyber.org-1448462427435-5d63162f-0
- ```
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement