Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- ## Kafka Cheat Sheet
- ### 1. Start kafka server
- ```bash
- # Using docker
- Docker —rm -d —name Kafka -e KAFKA_CREATE_TOPICS=test:1:1 wurstmeister/kafka -p 9092 9092
- # Using OS
- nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
- nohup bin/kafka-server-start.sh config/server.properties &
- ```
- ### 2. Woring with topics
- ```bash
- # List topic
- kafka-topics --list --zookeeper localhost:2181
- # Create topic
- kafka-topics --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 3 --topic test_topic
- # Describe topic
- kafka-topics --describe --bootstrap-server localhost:9092 --topic test_topic
- # Alter Topic to change partitions and replications
- kafka-topics --alter --bootstrap-server localhost:9092 --topic test_topic --partitions 5
- ```
- ### 3. Producer and Consumer
- ```bash
- # Start Provider
- kafka-console-producer --broker-list localhost:9092 --topic test_topic
- # Start producer with key:value separator
- kafka-console-producer --broker-list localhost:9092 --topic test_topic --property "parse.key=true" --property "key.separator=:"
- # Start Consumer
- kafka-console-consumer --bootstrap-server localhost:9092 --topic test_topic --from-beginning
- ```
- ### 4. Consumer Group
- ```bash
- # Create new and describe consumer group
- kafka-consumer-groups --new-consumer --bootstrap-server localhost:9092 --describe --group group_name
- # Describe consumer group
- kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group group_name
- # Reset offsets of consumer group [--to-earliest, --to-latest]
- kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --to-earliest --execute
- # Seek to offset
- kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --to-offset 12345 --execute
- # Shift by offsets
- kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --shift-by -1000 --execute
- ```
- ### 5. Unset JMX_PORT to delete topic in kafka
- ```bash
- unset JMX_PORT; kafka-streams-application-reset.sh --input-topics order_topic --application-id KafkaManagerOffsetCache --bootstrap-servers 172.16.8.98:9095
- unset JMX_PORT; kafka-topics.sh --zookeeper 172.16.8.98:2184 --delete --topic order_topic
- ```
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement