将消息发布到kafka主题时出错

dvtswwa3  于 2021-06-08  发布在  Kafka
关注(0)|答案(6)|浏览(544)

我对Kafka很陌生,我试着为它建立环境。我试图运行一个单节点Kafka,但我得到的错误。
在mac上执行以下步骤

1. brew install zookeeper
2. brew install kafka
3. zkServer start
4.  kafka-server-start.sh /usr/local/etc/kafka/server.properties
5.bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
6.bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test 
This is a message

但我有以下错误。如果我错过了什么,请告诉我

[2015-10-19 15:48:46,632] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,637] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,638] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-10-19 15:48:46,746] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,749] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,749] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-10-19 15:48:46,860] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,863] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,863] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-10-19 15:48:46,973] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,977] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:46,977] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-10-19 15:48:47,083] WARN Error while fetching metadata [{TopicMetadata for topic test -> 
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)
[2015-10-19 15:48:47,084] ERROR Failed to send requests for topics test with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-10-19 15:48:47,086] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
    at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
    at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
    at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
    at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
    at scala.collection.immutable.Stream.foreach(Stream.scala:547)
    at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
    at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)

谢谢

polkgigr

polkgigr1#

在使用kafka代理时,zookeeper是必须的。你必须开始 zookeeper-server 启动前 kafka-server .
使用端口时要非常小心 zookeeper port ,何时使用 kafka broker port 如果您得到任何与元数据相关的异常,请检查您的主题是否存在于代理中。 ./kafka-topics.sh --zookeeper <host>:<port> --describe --topic <topic-name> 对于主题和使用者,使用zookeeper端口(服务),对于使用者,使用kafka代理。

4szc88ey

4szc88ey2#

你好,这是一个非常常见的港口和主机问题。
你能确认一下zookeeper和kafka经纪人的主机和端口吗。我假设您已经在本地配置了kafka,所以请确保您提供了正确的主机名。
运行下面的命令,确保zookeeper和kafka代理在命令中提到的端口上运行:
netstat-grep端口
如果你还面临任何问题,请告诉我。

ugmeyewa

ugmeyewa3#

我在server.properties中将localhost设置为127.0.0.1来解决这个问题

mrwjdhj3

mrwjdhj34#

通过设置值,我这边的问题已经解决了 localhostconfig/server.properties 文件:

host.name
advertised.host.name
hm2xizp9

hm2xizp95#

我通过更改server.properties文件中的host.name属性来解决这个问题。默认情况下,它被注解掉并指向本地计算机。所以,如果你想在本地启动Kafka,只要取消注解就可以了。意思是-host.name=localhost

4dc9hkyq

4dc9hkyq6#

尝试使用正确的配置启动zookeeper,就像使用kafka一样:

bin/zookeeper-server-start.sh config/zookeeper.properties

您可以删除代理日志,然后重试:

rm -R /tmp/kafka-logs (default location)

如果仍然有问题,请检查日志。

相关问题