Kafka-Mount-Balancing


        
            
                Markdown 1.06 KB
                                    
                        | None                    
                
                                        |
    0    0                            

            
                
                                    raw
                    download
                    clone
                    embed
                    print
                
                                    report
                
                
                            
        
        
            Shutdown Broker
go to ambari host and shutdown kafka broker only after validating that all topics have multiple ISRs
backup files /grid/N/kafka-logs/{replication-offset-checkpoint,recovery-point-offset-checkpoint}
cp $PWD/recovery-point-offset-checkpoint /tmp/dirmove/

cp $PWD/replication-offset-checkpoint /tmp/dirmove/
identify partition be move /grid/N/kafka-logs/SOMETIPICFOLDER  to another log-dir
sudo mv reddit_dev_comments_fh-0 /tmp/kafka-logs2
copy offset file entries that match partition to separate location, and remove the entries from the offset file.. Then update the topic-partition entries by reducing it number after moving the other partition directory
replication
reddit_dev_comments_fh 0 100621118
recovery
reddit_dev_comments_fh 0 100621165
in the dest directory update the offset checkpoint file. Add the copied entires to this file for the partition that was moved. Update the number of partitions entires (second line in the file)
restart broker
input -> sourcelogdir targetlogdir topic_name partition_number