我已经成功地在Ubuntu 18.04 LTS上使用Zookeeper设置了Kafka,但是当在Ubuntu 20.04.6上尝试相同的配置时,Kafka无法连接到Zookeeper。我遇到的错误消息是Timed out waiting for connection while in state: CONNECTING.
。这令人困惑,因为相同的设置在Ubuntu 18.04 LTS上无缝工作。
这是我的Kafka server.log异常
这是kfka.yml。
---
- name: Setup Kafka Group
group: name=kafka system=yes
- name: Setup Kafka User
user: name=kafka system=yes group=kafka
- name: Download Kafka
get_url: url = https://archive.apache.org/dist/kafka/2.4.1/kafka_2.13-2.4.1.tgz dest=/opt/kafka_2.13-2.4.1.tgz owner=kafka group=kafka
- name: Untar kafka
unarchive:
src=/opt/kafka_2.13-2.4.1.tgz
dest=/opt
creates= /opt/kafka_2.13-2.4.1
copy=no
owner=kafka
group=kafka
- name: Create Symlink
file:
src= /opt/kafka_2.13-2.4.1
path=/opt/kafka
state=link
- name: Create systemd configuration
template: dest =/etc/systemd/system/kafka.service owner=root group=root mode=644 src=files/kafka.service.j2
- command: systemctl daemon-reload
- name: Create Kafka Data directory
file: path=/app/kafka/data owner=kafka group=kafka mode=u=rwX,g=rX,o=rX recurse=yes
- name: Remove lost+found in the Data directory
file: path="/app/kafka/data/lost+found" state=absent
- name: Create Log directory
file: path=/var/log/kafka state=directory owner=kafka group=kafka mode=755
- name: Link configuration directory to /opt/kafka/config
file: path=/etc/kafka state=link src=/opt/kafka/config
- name: Create log4j.properties
template: dest={/etc/kafka/log4j.properties owner=kafka group=kafka mode=644 src=files/log4j.properties.j2
- name: Copy logrotate configuration
template: dest=/etc/logrotate.d/kafka owner=root group=root mode=644 src=files/logrotate.conf.j2
- name: Create server.properties
template: dest=/etc/kafka/server.properties owner=kafka group=kafka mode=644 src=files/server.properties.j2
- name: Enable Kafka daemon service
shell: systemctl enable kafka
- name: Start kafka daemon service
service: name=kafka state=restarted
- set_fact: kafka_ports="[]"
- name: Verify PLAINTEXT listener
set_fact: kafka_ports="['9092']"
when: verify
- name: Verify Kafka port
debug: msg="ports to verify {{kafka_ports}} Kafak listen address {{kafka_listen_address | default('localhost')}} - {{ansible_hostname}}}}"
- name: Wait for Kafka to start
wait_for: host={{kafka_listen_address | default('localhost')}} port=9092 state=started timeout=30
with_items:
- "{{kafka_ports}}"
when: verify
- name: List topics
command: /opt/kafka/bin/kafka-topics.sh --list --zookeeper localhost:2181
# sudo_user: kafka
register: kafka_topics
- name: Create Kafka topics
shell: /opt/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor {{replication_factor}} --partitions {{partition}} --topic {{item.topic}}
with_items:
"{{topics}}"
when: item.topic not in kafka_topics.stdout_lines
字符串
Kafka错误日志
[2023-12-10 04:48:06,889] INFO starting (kafka.server.KafkaServer)
[2023-12-10 04:48:06,890] INFO Connecting to zookeeper on localhost:2121 (kafka.server.KafkaServer)
[2023-12-10 04:48:06,906] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2121. (kafka.zookeeper.ZooKeeperClient)
[2023-12-10 04:48:06,937] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2023-12-10 04:48:12,941] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2023-12-10 04:48:13,057] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2023-12-10 04:48:13,064] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:259)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:255)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:113)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1858)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:375)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:399)
at kafka.server.KafkaServer.startup(KafkaServer.scala:207)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
at kafka.Kafka$.main(Kafka.scala:84)
at kafka.Kafka.main(Kafka.scala)
型
这是zookeeper.yml
---
- name: Setup Zookeeper Group
group: name=zookeeper system=yes
- name: Setup Zookeeper User
user: name=zookeeper system=yes group=zookeeper
- name: Update apt cache and install Zookeeper
apt:
name: zookeeperd
update_cache: yes
- name: Setup zoo.cfg file
template: src=files/zoo.cfg.j2 dest=/etc/zookeeper/conf/zoo.cfg
- name: Setup myid file
template: src=files/myid.j2 dest=/etc/zookeeper/conf/myid
- name: Setup Java Environment
template: src=files/java.env.j2 dest=/etc/zookeeper/conf/java.env
- name: Create Log directory
file: path=/var/log/zookeeper state=directory owner=zookeeper group=zookeeper mode=755
- name: Create Data directory
file: path=/var/lib/zookeeper state=directory owner=zookeeper group=zookeeper mode=755
- name: Create log4j properties
copy: src=files/log4j.properties dest=/etc/zookeeper/conf/log4j.properties owner=zookeeper group=zookeeper mode=644
- name: Create logrotate configuration
template: src=files/logrotate.conf.j2 dest=/etc/logrotate.d/zookeeper owner=zookeeper group=root mode=644
- name: Restart Zookeeper Service
service:
name: zookeeper.service
state: restarted
型
我是Zookeeper日志
2023-12-08 19:31:59,213 - INFO [main:QuorumPeerConfig@136] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg
2023-12-08 19:31:59,217 - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2023-12-08 19:31:59,219 - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2023-12-08 19:31:59,220 - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2023-12-08 19:31:59,221 - WARN [main:QuorumPeerMain@116] - Either no config or no quorum defined in config, running in standalone mode
2023-12-08 19:31:59,232 - INFO [main:QuorumPeerConfig@136] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg
2023-12-08 19:31:59,233 - INFO [main:ZooKeeperServerMain@98] - Starting server
2023-12-08 19:31:59,245 - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.13-5build1--1, built on Tue, 18 Feb 2020 10:26:56 +0100
2023-12-08 19:31:59,251 - INFO [main:Environment@100] - Server environment:host.name=kafka-node
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.version=1.8.0_392
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.vendor=Private Build
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.class.path=/etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
2023-12-08 19:31:59,252 - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
2023-12-08 19:31:59,254 - INFO [main:Environment@100] - Server environment:os.name=Linux
2023-12-08 19:31:59,255 - INFO [main:Environment@100] - Server environment:os.arch=amd64
2023-12-08 19:31:59,255 - INFO [main:Environment@100] - Server environment:os.version=5.4.0-167-generic
2023-12-08 19:31:59,255 - INFO [main:Environment@100] - Server environment:user.name=zookeeper
2023-12-08 19:31:59,255 - INFO [main:Environment@100] - Server environment:user.home=/home/zookeeper
2023-12-08 19:31:59,255 - INFO [main:Environment@100] - Server environment:user.dir=/
2023-12-08 19:31:59,262 - INFO [main:ZooKeeperServer@836] - tickTime set to 2000
2023-12-08 19:31:59,262 - INFO [main:ZooKeeperServer@845] - minSessionTimeout set to -1
2023-12-08 19:31:59,263 - INFO [main:ZooKeeperServer@854] - maxSessionTimeout set to -1
2023-12-08 19:31:59,270 - INFO [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2023-12-08 19:31:59,274 - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
型
1条答案
按热度按时间41zrol4v1#
zookeeper中的端口错误。connect属性
来自Zookeeper日志
字符串
从Kafka日志中
型
如果你想使用Ansible来安装Kafka,我建议你使用Confluent的剧本和最新版本的Ubuntu