How to move Kafka Partition log directory within a Broker Node
This video explains how to move Kafka partitions between log.dirs when one of the folders is filling up faster than others.
At times Kafka Brokers can find one of its log directory utilization at 100% and the broker process would fail to start.
This article provides the instructions to manually move partition data between different log directories within a Kafka Broker.
The Kafka brokers maintain two offset checkpoint files inside each log directory:
replication-offset-checkpoint
recovery-point-offset-checkpoint
And both these files have the following format:
(a) 1st line: Version Number
(b) 2nd line: Number of topic-partition entries in the file
(c) All the remaining lines: Replication Offset/Recovery Point Offset, for every partition data maintained within the current log directory.
Watch video How to move Kafka Partition log directory within a Broker Node online, duration hours minute second in high quality that is uploaded to the channel Cloudera, Inc. 02 July 2019. Share the link to the video on social media so that your subscribers and friends will also watch this video. This video clip has been viewed 2,706 times and liked it 17 visitors.
