kafka(0.10.0.1)mirror maker不工作,似乎进程正在运行,但什么也没做

5sxhfpxr  于 2021-06-08  发布在  Kafka
关注(0)|答案(0)|浏览(352)

我面临一个奇怪的问题,我的镜像生成器能够镜像目标集群中的主题名称,但是当我尝试使用来自它的消息时,却看不到任何消息。下面是我的配置文件-

Kafka version : kafka_10.0.1
All Certs /Keys are my organization issued certificate so no Error for SSL 

Source Cluster :
==============

Zookeeeper Properties
dataDir=/home/kafka/zookeeper
clientPort=2181
maxClientCnxns=3

Server Properties

broker.id=1
listeners= SSL://123.456.789.110:9092
advertised.listeners=SSL://123.456.789.110:9092
security.inter.broker.protocol=SSL
num.network.threads=3
num.io.threads=8
auto.create.topics.enable=true
ssl.client.auth=none
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=test1234
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=test1234
ssl.enabled.protocols=TLSv1.2
log.dirs=/home/kafka/kafka-logs
zookeeper.connect=123.456.789.110:2181

 Kafka Up and Running . Log persistently logging in Source Cluster 

Target Cluster :
================
Zookeeeper Properties

dataDir=/home/kafka/zookeeper
clientPort=2181
maxClientCnxns=3

Server Properties
broker.id=1
listeners= SSL://234.456.789.110:9092
advertised.listeners=SSL://sl73easapd04.visa.com:9092
security.inter.broker.protocol=SSL
num.network.threads=3
num.io.threads=8
ssl.client.auth=none
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=test1234
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=test1234
ssl.enabled.protocols=TLSv1.2
ssl.keystore.type = JKS
ssl.truststore.type = JKS
log.dirs=/data/visingh/kafka/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=234.456.789.110:2181
zookeeper.connection.timeout.ms=6000

mm_consumer.properties :
bootstrap.servers=123.456.789.110:9092
exclude.internal.topics = true
auto.offset.reset=earliest
security.protocol = SSL
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=Test1234
ssl.enabled.protocols= TLSv1.2
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=Test1234
ssl.truststore.type=JKS
ssl.keystore.type=JKS
zookeeper.connection.timeout.ms=6000
group.id=test-consumer-group
client.id=mirror_maker_consumer
consumer.timeout.ms=50000

mm_producer.properties :
bootstrap.servers=234.456.789.110:9092
acks=1
batch.size=100
security.protocol = SSL
ssl.truststore.location=/etc/CApath/trustedstore
ssl.truststore.password=Test1234 
ssl.enabled.protocols=TLSv1.2
ssl.keystore.location=/etc/keys/keystore
ssl.keystore.password=Test1234
ssl.truststore.type=JKS
ssl.keystore.type=JKS
compression.type=gzip

然后我用以下命令启动mirror maker,这个命令运行fines和topic app logging,创建于目标集群,而不是目标集群。
bin/kafka-run-class.sh kafka.tools.mirrormaker--new.consumer--consumer.config配置/mm\u consumer.properties--producer.config配置/mm\u producer.properties--白名单*
kafka mirromaker已启动,但目标群集上没有日志。
我使用主题名和下面的命令执行了另一个mirror maker命令,该命令在目标hostcluster上失败
/bin/kafka-mirror-maker.sh--new.consumer--consumer.config/mm\u consumer.properties--producer.config/mm\u producer.properties--白名单应用程序日志记录

[2018-06-22 03:59:27,536] FATAL [mirrormaker-thread-0] Mirror maker thread 
failure due to  (kafka.tools.MirrorMaker$MirrorMakerThread)
java.lang.IllegalArgumentException: Invalid timestamp -1
        at org.apache.kafka.clients.producer.ProducerRecord.<init> (ProducerRecord.java:60)
    at kafka.tools.MirrorMaker$defaultMirrorMakerMessageHandler$.handle(MirrorMaker.scala:678)
    at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:414)
[2018-06-22 03:59:27,563] FATAL [mirrormaker-thread-0] Mirror maker thread exited abnormally, stopping the whole mirror maker. (kafka.tools.MirrorMaker$MirrorMakerThread)

同时运行“.”包括引号,而不是只运行
bin/kafka-run-class.sh kafka.tools.mirrormaker--new.consumer--consumer.config配置/mm\u consumer.properties--producer.config配置/mm\u producer.properties--白名单“.*”

[2018-06-22 04:22:01,647] FATAL [mirrormaker-thread-0] Mirror maker thread failure due to  (kafka.tools.MirrorMaker$MirrorMakerThread)
java.lang.IllegalArgumentException: Invalid timestamp -1
        at org.apache.kafka.clients.producer.ProducerRecord.<init>(ProducerRecord.java:60)
        at kafka.tools.MirrorMaker$defaultMirrorMakerMessageHandler$.handle(MirrorMaker.scala:678)
        at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:414)
[2018-06-22 04:22:01,674] FATAL [mirrormaker-thread-0] Mirror maker thread exited abnormally, stopping the whole mirror maker. (kafka.tools.MirrorMaker$MirrorMakerThread)

我错过什么了吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题