使用loadbalancer服务时,无法在kubernetes上生成/使用kafka

jckbn6z7  于 2021-06-04  发布在  Kafka
关注(0)|答案(1)|浏览(396)

背景
我正在使用合流的开源helm图表在kuberentes上运行kafka。我已经有一个eks集群运行托管节点组。
当我使用nodeport暴露代理时,效果很好。然而,我想启用负载平衡器,我能够启用它和服务是每个代理创建吊舱(启用了内部网络负载均衡器)我们所有的生产商都在aws。

$ kubectl get svc -n kafka
NAME                          TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)             AGE
kafka-0-external              LoadBalancer   10.100.185.40    ac5b7fccb69bc4738b2e498995e65de2-9d6b81206f5d1d7d.elb.us-east-2.amazonaws.com   31090:30366/TCP     10m
kafka-1-external              LoadBalancer   10.100.192.249   ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com   31090:31063/TCP     10m
kafka-2-external              LoadBalancer   10.100.80.80     a36dc44c757f4429b81163ab651a7012-e94e40584210b988.elb.us-east-2.amazonaws.com   31090:32700/TCP     10m
kafka-cp-kafka                ClusterIP      10.100.163.158   <none>                                                                          9092/TCP            10m
kafka-cp-kafka-connect        ClusterIP      10.100.139.66    <none>                                                                          8083/TCP            10m
kafka-cp-kafka-headless       ClusterIP      None             <none>                                                                          9092/TCP            10m
kafka-cp-kafka-rest           ClusterIP      10.100.146.106   <none>                                                                          8082/TCP            10m
kafka-cp-schema-registry      ClusterIP      10.100.103.114   <none>                                                                          8081/TCP            10m
kafka-cp-zookeeper            NodePort       10.100.22.195    <none>                                                                          2181:32724/TCP      10m
kafka-cp-zookeeper-headless   ClusterIP      None             <none>                                                                          2888/TCP,3888/TCP   10m

现在我想通过生产和消费来测试。我在同一个vpc中启动了一个新的ec2示例。我可以得到元数据,但我不能生产和消费。

ubuntu@ip-192-168-87-196:~/kafka_2.11-2.3.1/bin$ kafkacat -b ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090 -L
Metadata for all topics (from broker -1: ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090/bootstrap):
 3 brokers:
  broker 0 at kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
  broker 2 at kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
  broker 1 at kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
 8 topics:
  topic "test" with 25 partitions:
    partition 0, leader 1, replicas: 1,2,0, isrs: 1,0,2
    partition 5, leader 0, replicas: 0,2,1, isrs: 1,0,2
    partition 10, leader 2, replicas: 2,1,0, isrs: 1,0,2

当我试图产生这个错误

ubuntu@ip-192-168-87-196:~/kafka_2.11-2.3.1/bin$ kafkacat -b ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090 -C -t test
% ERROR: Local: Host resolution failure: kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/0: Failed to resolve 'kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution
% ERROR: Local: Host resolution failure: kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/2: Failed to resolve 'kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution
% ERROR: Local: Host resolution failure: kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/1: Failed to resolve 'kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution

这些是我的听众

$ kubectl logs kafka-cp-kafka-2 -n kafka -c cp-kafka-broker | grep -i listeners
+ export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
        advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
        listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090
        advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
        listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090
        advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
        listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090

我试了几天,想要一些指导。让我知道如果有人有什么要分享的,我错过了什么?

p8ekf7hl

p8ekf7hl1#

如果您在k8s群集外部运行kafka客户端,则必须为kafka\u播发的\u侦听器使用群集外部可见的外部ip或主机名:
Kafka的听众=plaintext://kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka:9092,external://ac5b7fccb69bc4738b2e498995e65de2-9d6b81206f5d1d7d.elb.us-east-2.amazonaws.com:30366

相关问题