docker swarm中连接kafka多代理时出现debezium错误

yacmzcpb  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(639)

当我用这个堆栈建立我的群时:kafka(多代理)、zookeeper、debezium。kafka和zookeeper正在工作,可以创建主题、使用者和生产者,但是debezium显示错误:org.apache.kafka.connect.errors.connectexception:无法连接到和描述kafka集群。检查工作者的代理连接和安全属性。我没有修改任何东西,只是默认配置为下面的docker堆栈:

  1. version: '3.6'
  2. services:
  3. zoo:
  4. image: wurstmeister/zookeeper
  5. ports:
  6. - '2181:2181'
  7. volumes:
  8. - zoo-data:/tmp/zookeeper
  9. deploy:
  10. replicas: 1
  11. placement:
  12. constraints:
  13. - node.labels.type==zoo
  14. kafka:
  15. image: wurstmeister/kafka:latest
  16. ports:
  17. - target: 9094
  18. published: 9094
  19. protocol: tcp
  20. mode: host
  21. environment:
  22. HOSTNAME_COMMAND: "docker info | grep ^Name: | cut -d' ' -f 2"
  23. KAFKA_ZOOKEEPER_CONNECT: zoo:2181
  24. KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
  25. KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://_{HOSTNAME_COMMAND}:9094
  26. KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://:9094
  27. KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
  28. #KAFKA_CREATE_TOPICS: "Topic1:1:2,Topic2:1:1:compact"
  29. volumes:
  30. - /var/run/docker.sock:/var/run/docker.sock
  31. - kafka-data:/tmp/kafka-logs
  32. deploy:
  33. mode: global
  34. placement:
  35. constraints:
  36. - node.labels.name==kafka
  37. depends_on:
  38. - zoo
  39. debezium:
  40. image: debezium/connect:0.8
  41. hostname: connect
  42. ports:
  43. - '8083:8083'
  44. environment:
  45. BOOTSTRAP_SERVERS: kafka:9094
  46. GROUP_ID: 1
  47. CONFIG_STORAGE_TOPIC: my_connect_configs
  48. OFFSET_STORAGE_TOPIC: my_connect_offsets
  49. deploy:
  50. placement:
  51. constraints:
  52. - node.labels.type==dbz
  53. depends_on:
  54. - kafka
  55. volumes:
  56. kafka-data:
  57. zoo-data:

当我检查docker服务日志debezium时,它显示错误

  1. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | 2018-08-03 04:33:27,034 ERROR || Stopping due to error [org.apache.kafka.connect.cli.ConnectDistributed]
  2. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker's broker connection and security properties.
  3. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
  4. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:45)
  5. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:77)
  6. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
  7. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
  8. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
  9. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
  10. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:258)
  11. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:58)
  12. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | ... 2 more
  13. shippo_kafka_debezium.1.5l1yhz27r6p2@kafka1 | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.

有人能告诉我如何修复这个错误吗,我对这个堆栈还不熟悉,所以,在几天的研究中,我搞不懂它。非常感谢!

igsr9ssn

igsr9ssn1#

你可以换这条线 HOSTNAME_COMMAND: "docker info | grep ^Name: | cut -d' ' -f 2"HOSTNAME_COMMAND: "docker info | grep 'Node Address:' | cut -d' ' -f 4" 或者你可以使用这个docker文件

  1. version: '3.2'
  2. services:
  3. zookeeper:
  4. image: wurstmeister/zookeeper
  5. ports:
  6. - "2181:2181"
  7. deploy:
  8. mode: global
  9. volumes:
  10. - /shared_data/zoo1/data:/data
  11. - /shared_data/zoo1/datalog:/datalog
  12. environment:
  13. ZOO_MY_ID: 1
  14. ZOO_PORT: 2181
  15. ZOO_SERVERS: server.1=zookeeper:2888:3888
  16. kafka:
  17. image: wurstmeister/kafka:latest
  18. ports:
  19. - target: 9094
  20. published: 9094
  21. protocol: tcp
  22. mode: host
  23. deploy:
  24. mode: global
  25. environment:
  26. HOSTNAME_COMMAND: "docker info | grep 'Node Address:' | cut -d' ' -f 4"
  27. KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  28. KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
  29. KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://_{HOSTNAME_COMMAND}:9094
  30. KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://:9094
  31. KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
  32. volumes:
  33. - /var/run/docker.sock:/var/run/docker.sock
  34. - /shared_data/kafka:/var/lib/kafka/data
  35. depends_on:
  36. - zookeeper
展开查看全部

相关问题