背景
我正在使用合流的开源helm图表在kuberentes上运行kafka。我已经有一个eks集群运行托管节点组。
当我使用nodeport暴露代理时,效果很好。然而,我想启用负载平衡器,我能够启用它和服务是每个代理创建吊舱(启用了内部网络负载均衡器)我们所有的生产商都在aws。
$ kubectl get svc -n kafka
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kafka-0-external LoadBalancer 10.100.185.40 ac5b7fccb69bc4738b2e498995e65de2-9d6b81206f5d1d7d.elb.us-east-2.amazonaws.com 31090:30366/TCP 10m
kafka-1-external LoadBalancer 10.100.192.249 ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com 31090:31063/TCP 10m
kafka-2-external LoadBalancer 10.100.80.80 a36dc44c757f4429b81163ab651a7012-e94e40584210b988.elb.us-east-2.amazonaws.com 31090:32700/TCP 10m
kafka-cp-kafka ClusterIP 10.100.163.158 <none> 9092/TCP 10m
kafka-cp-kafka-connect ClusterIP 10.100.139.66 <none> 8083/TCP 10m
kafka-cp-kafka-headless ClusterIP None <none> 9092/TCP 10m
kafka-cp-kafka-rest ClusterIP 10.100.146.106 <none> 8082/TCP 10m
kafka-cp-schema-registry ClusterIP 10.100.103.114 <none> 8081/TCP 10m
kafka-cp-zookeeper NodePort 10.100.22.195 <none> 2181:32724/TCP 10m
kafka-cp-zookeeper-headless ClusterIP None <none> 2888/TCP,3888/TCP 10m
现在我想通过生产和消费来测试。我在同一个vpc中启动了一个新的ec2示例。我可以得到元数据,但我不能生产和消费。
ubuntu@ip-192-168-87-196:~/kafka_2.11-2.3.1/bin$ kafkacat -b ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090 -L
Metadata for all topics (from broker -1: ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090/bootstrap):
3 brokers:
broker 0 at kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
broker 2 at kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
broker 1 at kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090
8 topics:
topic "test" with 25 partitions:
partition 0, leader 1, replicas: 1,2,0, isrs: 1,0,2
partition 5, leader 0, replicas: 0,2,1, isrs: 1,0,2
partition 10, leader 2, replicas: 2,1,0, isrs: 1,0,2
当我试图产生这个错误
ubuntu@ip-192-168-87-196:~/kafka_2.11-2.3.1/bin$ kafkacat -b ae035d93de7874c49bc2402d5c174403-65cdb5cda161fa89.elb.us-east-2.amazonaws.com:31090 -C -t test
% ERROR: Local: Host resolution failure: kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/0: Failed to resolve 'kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution
% ERROR: Local: Host resolution failure: kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/2: Failed to resolve 'kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution
% ERROR: Local: Host resolution failure: kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090/1: Failed to resolve 'kafka-cp-kafka-1.kafka-cp-kafka-headless.kafka.svc.cluster.local:31090': Temporary failure in name resolution
这些是我的听众
$ kubectl logs kafka-cp-kafka-2 -n kafka -c cp-kafka-broker | grep -i listeners
+ export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090
advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090
advertised.listeners = PLAINTEXT://kafka-cp-kafka-2.kafka-cp-kafka-headless.kafka:9092,EXTERNAL://:31090
listeners = PLAINTEXT://0.0.0.0:9092,EXTERNAL://0.0.0.0:31090
我试了几天,想要一些指导。让我知道如果有人有什么要分享的,我错过了什么?
1条答案
按热度按时间p8ekf7hl1#
如果您在k8s群集外部运行kafka客户端,则必须为kafka\u播发的\u侦听器使用群集外部可见的外部ip或主机名:
Kafka的听众=plaintext://kafka-cp-kafka-0.kafka-cp-kafka-headless.kafka:9092,external://ac5b7fccb69bc4738b2e498995e65de2-9d6b81206f5d1d7d.elb.us-east-2.amazonaws.com:30366