Kafka经常提出无效的生产异常

nvbavucw  于 2021-07-15  发布在  Kafka
关注(0)|答案(0)|浏览(935)

我有一个kafka streams应用程序,有4个示例,每个示例运行在一个单独的ec2示例上,有16个线程。总螺纹=16*4。输入主题只有32个分区。我知道有些线程将保持空闲。
我一直看到这个例外

Caused by: org.apache.kafka.common.errors.InvalidProducerEpochException: Producer attempted to produce with an old epoch.
01:57:23.971 [kafka-producer-network-thread | bids_kafka_streams_beta_007-fd78c6fa-62bc-437d-add0-c31f5b7c1901-StreamThread-12-1_6-producer] ERROR org.apach
e.kafka.streams.processor.internals.RecordCollectorImpl - stream-thread [bids_kafka_streams_beta_007-fd78c6fa-62bc-437d-add0-c31f5b7c1901-StreamThread-12] t
ask [1_6] Error encountered sending record to topic kafka_streams_bids_output for task 1_6 due to:
org.apache.kafka.common.errors.InvalidProducerEpochException: Producer attempted to produce with an old epoch.
Written offsets would not be recorded and no more records would be sent since the producer is fenced, indicating the task may be migrated out

我在streams配置中唯一更改的设置是producer配置,以减少代理上的cpu使用

linger.ms=10000
commit.interval.ms=10000

记录的窗口时间为2分钟
是因为重新平衡吗?为什么这么频繁?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题