Kafka生产集群中有一台机器cdh-003由于物理故障原因挂掉了,并且系统起不来了,使得线上的spark Streaming实时任务不能正常消费,重启实时任务都不行。查看kafka topic状态,发现broker Leader出现-1的情况,如下图
Kafka Broker Leader 为-1,表示有partition在选举Leader的时候失败了,因此引起了消费该Topic的实时任务都出现了异常,经过排除发现挂掉的cdh-003机器正好是broker id为257。(但为何192没有被选举为leader呢??)
kafka partition状态信息存在Zookeeper上,我的环境目录是 /kafka/brokers/topics/,具体操作如下:
[zk: localhost:2181(CONNECTED) 2] get /kafka/brokers/topics/mds001/partitions/1/state
{"controller_epoch":87,"leader":-1,"version":1,"leader_epoch":96,"isr":[257]}
[zk: localhost:2181(CONNECTED) 3] set /kafka/brokers/topics/mds001/partitions/1/state {"controller_epoch":87,"leader":192,"version":1,"leader_epoch":96,"isr":[192]}
[zk: localhost:2181(CONNECTED) 4] get /kafka/brokers/topics/mds001/partitions/1/state
{"controller_epoch":87,"leader":192,"version":1,"leader_epoch":96,"isr":[192]}
[zk: localhost:2181(CONNECTED) 5]
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。