我们正在2节点集群中运行kafka(vesion kafka_2.11-0.10.1.0)。我们有两个生产者(javaapi)在不同的主题上工作。每个主题都有一个分区。我们讨论这个问题的主题有一个消费者在运行。这个设置已经运行了3个月了,我们看到了这个问题。其他论坛中针对这个问题的所有建议案例/解决方案似乎都不适用于我的场景。
生产商例外;
-2017-11-25t17:40:33035[kafka producer network thread | producer-1]error client.producer.binglogproducercallback-发送消息时遇到异常;>org.apache.kafka.common.errors.notleaderforpartitionexception:此服务器不是该主题分区的领导者。
我们没有为消息启用重试,因为这是事务性数据,我们希望保持顺序。
生产者配置:
bootstrap.servers : server1ip:9092
acks :all
retries : 0
linger.ms :0
buffer.memory :10240000
max.request.size :1024000
key.serializer : org.apache.kafka.common.serialization.StringSerializer
value.serializer : org.apache.kafka.common.serialization.StringSerializer
我们正在连接到生产商和消费者的server1。server2上的控制器日志表明在sametime期间发生了一些关闭,但我不明白为什么会发生这种情况。
[2017-11-25 17:31:44776]调试[controller 2]:首选副本Map()中没有的主题(kafka.controller.kafkantroller)[2017-11-25 17:31:44776]跟踪[controller 2]:代理2的前导不平衡比率为0.000000(kafka.controller.kafkantroller)[2017-11-25 17:31:44,776]调试[controller 2]:主题不在preferred replica map()(kafka.controller.kafkantroller)[2017-11-25 17:31:44776]跟踪[controller 2]:代理1的前导不平衡比率为0.000000(kafka.controller.kafkantroller)[2017-11-25 17:34:18314]信息[sessionexpirationlistener on 2],zk过期;关闭所有控制器组件并尝试重新选择(kafka.controller.kafkantroller$sessionexpirationlistener)[2017-11-25 17:34:18317]调试[controller 2]:控制器辞职,代理id 2(kafka.controller.kafkantroller)[2017-11-25 17:34:18,317]调试[controller 2]:取消注册isrchangenotificationlistener(kafka.controller.kafkacontroller)[2017-11-25 17:34:18317]信息[delete-topics-thread-2],关闭(kafka.controller.topicdeletionmanager$deletetopicsthread)[2017-11-25 17:34:18317]信息[delete-topics-thread-2],停止(kafka.controller.topicdeletionmanager$deletetopicsthread)[2017-11-25 17:34:18318]信息[delete-topics-thread-2],关闭完成(kafka.controller.topicdeletionmanager$deletetopicsthread)[2017-11-25 17:34:18,318]信息[controller 2上的分区状态机]:停止分区状态机(kafka.controller.partitionstatemachine)[2017-11-25 17:34:18318]信息[controller 2上的副本状态机]:停止副本状态机(kafka.controller.replicastatemachine)[2017-11-25 17:34:18318]信息[controller-2-to-broker-2-send-thread],正在关闭(kafka.controller.requestsendthread)[2017-11-25 17:34:18318]信息[controller-2-to-broker-2-send-thread],已停止(kafka.controller.requestsendthread)[2017-11-25 17:34:18319]信息[controller-2-to-broker-2-send-thread],关闭已完成(kafka.controller.requestsendthread)[2017-11-25 17:34:18,319]信息[controller-2-to-broker-1-send-thread],正在关闭(kafka.controller.requestsendthread)[2017-11-25 17:34:18319]信息[controller-2-to-broker-1-send-thread],已停止(kafka.controller.requestsendthread)[2017-11-25 17:34:18319]信息[controller-2-to-broker-1-send-thread],关闭完成(kafka.controller.requestsendthread)[2017-11-25 17:34:18319]信息[controller 2]:代理2辞去控制器的职务(kafka.controller.kafkacontroller)[2017-11-25 17:34:18353]调试[isrchangenotificationlistener]已启动(kafka.controller.isrchangenotificationlistener)[2017-11-25 17:34:18353]调试[isrchangenotificationlistener]已启动(kafka.controller.isrchangenotificationlistener)[2017-11-25 17:34:18354]信息[brokerchangelistener on controller 2]:为路径/brokers/id和子项1,2触发代理更改侦听器(kafka.controller.replicastatemachine$brokerchangelistener)[2017-11-25 17:34:18,355]调试[deletetopicslistener on 2]:为要删除的主题触发的删除主题侦听器(kafka.controller.partitionstatemachine$deletetopicslistener)[2017-11-25 17:34:18362]信息[addpartitionslistener on 2]:触发的分区修改{“版本”:1,“partitions”:{“0”:[1]}对于路径/brokers/topics/esq(kafka.controller.partitionstatemachine$partitionmodificationslistener)[2017-11-25 17:34:18368]信息[addpartitionslistener on 2]:触发的分区修改{“版本”:1,“partitions”:{“0”:[1]}对于路径/brokers/topics/test1(kafka.controller.partitionstatemachine$partitionmodificationsliner)[2017-11-25 17:34:18369]信息[addpartitionslistener on 2]:触发的分区修改{“version”:1,“partitions”:{“0”:[2]}用于路径/brokers/topics/imageq(kafka.controller.partitionstatemachine$partitionmodificationsliner)[2017-11-25 17:34:18374]信息[addpartitionslistener on 2]:分区修改触发{“version”:1,“partitions”:{“8”:[1,2],“4”:[1,2],“9”:[2,1],“5”:[2,1],“6”:[1,2],“1”:[2,1],“0”:[1,2],“2”:[1,2],“7”:[2,1],“3”:[2],1] }}对于路径/brokers/topics/nms\u notifyq(kafka.controller.partitionstatemachine$partitionmodificationslistener)[2017-11-25 17:34:18375]信息[addpartitionslistener on 2]:对于路径/brokers/topics/tempbinlogreqq,分区修改触发了{“version”:1,“partitions”:{“0”:[1]}}@
暂无答案!
目前还没有任何答案,快来回答吧!