SHARE
TWEET

Untitled

a guest Sep 15th, 2019 73 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. ## Kafka Cheat Sheet
  2. ### 1. Start kafka server
  3.  
  4. ```bash
  5. # Using docker
  6. Docker —rm -d —name Kafka -e KAFKA_CREATE_TOPICS=test:1:1 wurstmeister/kafka -p 9092 9092
  7. # Using OS
  8. nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
  9. nohup bin/kafka-server-start.sh config/server.properties &
  10. ```
  11.  
  12. ### 2. Woring with topics
  13.  
  14. ```bash
  15. # List topic
  16. kafka-topics --list --zookeeper localhost:2181
  17. # Create topic
  18. kafka-topics --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 3 --topic test_topic
  19. # Describe topic
  20. kafka-topics --describe --bootstrap-server localhost:9092 --topic test_topic
  21. # Alter Topic to change partitions and replications
  22. kafka-topics --alter --bootstrap-server localhost:9092 --topic test_topic --partitions 5
  23. ```
  24.  
  25. ### 3. Producer and Consumer
  26.  
  27. ```bash
  28. # Start Provider
  29. kafka-console-producer --broker-list localhost:9092 --topic test_topic
  30. # Start producer with key:value separator
  31. kafka-console-producer --broker-list localhost:9092 --topic test_topic --property "parse.key=true" --property "key.separator=:"
  32. # Start Consumer
  33. kafka-console-consumer --bootstrap-server localhost:9092 --topic test_topic --from-beginning
  34. ```
  35.  
  36. ### 4. Consumer Group
  37.  
  38. ```bash
  39. # Create new and describe consumer group
  40. kafka-consumer-groups --new-consumer --bootstrap-server localhost:9092 --describe --group group_name
  41. # Describe consumer group
  42. kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group group_name
  43. # Reset offsets of consumer group [--to-earliest, --to-latest]
  44. kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --to-earliest --execute
  45. # Seek to offset
  46. kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --to-offset 12345 --execute
  47. # Shift by offsets
  48. kafka-consumer-groups --bootstrap-server localhost:9092 --group group_name --topic test_topic --reset-offsets --shift-by -1000 --execute
  49. ```
  50.  
  51. ### 5. Unset JMX_PORT to delete topic in kafka
  52.  
  53. ```bash
  54. unset JMX_PORT; kafka-streams-application-reset.sh  --input-topics order_topic   --application-id KafkaManagerOffsetCache --bootstrap-servers 172.16.8.98:9095
  55. unset JMX_PORT; kafka-topics.sh --zookeeper 172.16.8.98:2184 --delete --topic order_topic
  56. ```
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top