Kafka对Kubernetes在米尼库布不起作用

xam8gpfp  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(628)

我试图设置Kafka在minikube,一个非常基本的设置。我无法验证Kafka和zookeeper的设置是否正确,因为 kafkacat 失败。
这是我的配置:
Zookeeper

  1. kind: Deployment
  2. apiVersion: apps/v1
  3. metadata:
  4. name: zookeeper-deployment-1
  5. spec:
  6. selector:
  7. matchLabels:
  8. app: zookeeper-1
  9. template:
  10. metadata:
  11. labels:
  12. app: zookeeper-1
  13. spec:
  14. containers:
  15. - name: zoo1
  16. image: digitalwonderland/zookeeper
  17. ports:
  18. - containerPort: 2181
  19. env:
  20. - name: ZOOKEEPER_ID
  21. value: "1"
  22. - name: ZOOKEEPER_SERVER_1
  23. value: zoo1
  24. ---
  25. apiVersion: v1
  26. kind: Service
  27. metadata:
  28. name: zoo1
  29. labels:
  30. app: zookeeper-1
  31. spec:
  32. ports:
  33. - name: client
  34. port: 2181
  35. protocol: TCP
  36. - name: follower
  37. port: 2888
  38. protocol: TCP
  39. - name: leader
  40. port: 3888
  41. protocol: TCP
  42. selector:
  43. app: zookeeper-1

Kafka

  1. kind: Deployment
  2. apiVersion: apps/v1
  3. metadata:
  4. name: kafka-broker0
  5. spec:
  6. selector:
  7. matchLabels:
  8. app: kafka
  9. template:
  10. metadata:
  11. labels:
  12. app: kafka
  13. id: "0"
  14. spec:
  15. containers:
  16. - name: kafka
  17. image: wurstmeister/kafka
  18. ports:
  19. - containerPort: 9092
  20. env:
  21. - name: KAFKA_ADVERTISED_PORT
  22. value: "9092"
  23. - name: KAFKA_ADVERTISED_HOST_NAME
  24. value: kafka-service
  25. - name: KAFKA_ZOOKEEPER_CONNECT
  26. value: zoo1:2181
  27. - name: KAFKA_BROKER_ID
  28. value: "0"
  29. ---
  30. apiVersion: v1
  31. kind: Service
  32. metadata:
  33. name: kafka-service
  34. labels:
  35. name: kafka
  36. spec:
  37. ports:
  38. - port: 9092
  39. name: kafka-port
  40. protocol: TCP
  41. selector:
  42. app: kafka
  43. id: "0"

豆荚

  1. kafka-broker0-6885746967-6vktz 1/1 Running 0 5m20s
  2. zookeeper-deployment-1-7f5bb9785f-7pplk 1/1 Running 0 5m25s

svc公司

  1. kafka-service ClusterIP 10.99.226.129 <none> 9092/TCP 6m30s
  2. zoo1 ClusterIP 10.96.140.187 <none> 2181/TCP,2888/TCP,3888/TCP 6m35s

Kafka卡特原木

  1. kafkacat -L -b kafka-service:9092 -d broker
  2. %7|1596239513.610|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
  3. %7|1596239513.610|BROKER|rdkafka#producer-1| [thrd:app]: kafka-service:9092/bootstrap: Added new broker with NodeId -1
  4. %7|1596239513.610|BRKMAIN|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: Enter main broker thread
  5. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:app]: kafka-service:9092/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
  6. %7|1596239513.610|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.4.0 (0x10400ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, CC CXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
  7. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: Received CONNECT op
  8. %7|1596239513.610|STATE|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
  9. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: broker in state TRY_CONNECT connecting
  10. %7|1596239513.610|STATE|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
  11. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  12. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  13. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  14. %7|1596239513.610|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  15. %7|1596239513.614|BROKERFAIL|rdkafka#producer-1| [thrd:kafka-service:9092/bootstrap]: kafka-service:9092/bootstrap: failed: err: Local: Host resolution failure: (errno: Bad address)

节点端口更新
✗ Kafka卡特-l-bKafka-service:30236 -d 经纪人

  1. %7|1596476848.078|STATE|rdkafka#producer-1| [thrd:kafka-service:30236/bootstrap]: kafka-service:30236/bootstrap: Broker changed state CONNECT -> DOWN
  2. %7|1596476848.078|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 46ms: application metadata request
  3. %7|1596476848.078|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 46ms: application metadata request
  4. %7|1596476849.065|CONNECT|rdkafka#producer-1| [thrd:app]: Cluster connection already in progress: application metadata request
  5. %7|1596476849.065|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  6. % ERROR: Failed to acquire metadata: Local: Broker transport failure

minikube ip✗ Kafka卡特-l-b 192.168.64.2:30236-d经纪人

  1. |1596476908.164|BRKMAIN|rdkafka#producer-1| [thrd:192.168.64.2:30236/bootstrap]: 192.168.64.2:30236/bootstrap: Enter main broker thread
  2. %7|1596476908.164|CONNECT|rdkafka#producer-1| [thrd:app]: 192.168.64.2:30236/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
  3. %7|1596476908.164|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.4.0 (0x10400ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, CC CXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
  4. %7|1596476908.164|CONNECT|rdkafka#producer-1| [thrd:192.168.64.2:30236/bootstrap]: 192.168.64.2:30236/bootstrap: Received CONNECT op
  5. %7|1596476908.164|STATE|rdkafka#producer-1| [thrd:192.168.64.2:30236/bootstrap]: 192.168.64.2:30236/bootstrap: Broker changed state INIT -> TRY_CONNECT
  6. %7|1596476908.164|CONNECT|rdkafka#producer-1| [thrd:192.168.64.2:30236/bootstrap]: 192.168.64.2:30236/bootstrap: broker in state TRY_CONNECT connecting
  7. %7|1596476908.164|STATE|rdkafka#producer-1| [thrd:192.168.64.2:30236/bootstrap]: 192.168.64.2:30236/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
  8. %7|1596476908.164|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request

✗ Kafka卡特-l-blocalhost:30236 -d 经纪人

  1. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Received CONNECT op
  2. %7|1596477098.454|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.4.0 (0x10400ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, CC CXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
  3. %7|1596477098.454|STATE|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Broker changed state INIT -> TRY_CONNECT
  4. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: broker in state TRY_CONNECT connecting
  5. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  6. %7|1596477098.454|STATE|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
  7. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  8. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  9. %7|1596477098.454|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
  10. %7|1596477098.460|CONNECT|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Connecting to ipv4#127.0.0.1:30236 (plaintext) with socket 9
  11. %7|1596477098.461|BROKERFAIL|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: failed: err: Local: Broker transport failure: (errno: Connection refused)
  12. %7|1596477098.461|STATE|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Broker changed state CONNECT -> DOWN
  13. %7|1596477098.461|STATE|rdkafka#producer-1| [thrd:localhost:30236/bootstrap]: localhost:30236/bootstrap: Broker changed state DOWN -> INIT
  14. %7|1596477098.461|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 43ms: application metadata request
  15. %7|1596477098.461|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 42ms: application metadata request
ftf50wuq

ftf50wuq1#

  1. apiVersion: v1
  2. kind: Service
  3. metadata:
  4. name: kafka-service
  5. labels:
  6. name: kafka
  7. spec:
  8. type: NodePort
  9. ports:
  10. - port: 9092
  11. nodePort: 30236
  12. name: kafka-port
  13. protocol: TCP
  14. selector:
  15. app: kafka
  16. id: "0"

我从未使用过kafkacat,但如果它是安装在主机上的cli,而不是安装在另一个容器中,则现在可以这样使用它:

  1. kafkacat -L -b localhost:30236 -d broker

localhost,或kubernetes节点的ip。
使用服务的默认值是clusterip类型,并且只能在kubernetes集群内部访问

展开查看全部

相关问题