出于测试目的,我尝试在本地minikube上创建一个kafka集群。集群必须可以从kubernetes外部访问。
当我生产/消费豆荚里面没有问题,一切正常。
当我用本地机器生产时
bin/kafka-console-producer.sh --topic mytopic --broker-list 192.168.99.100:32767
其中192.168.99.100是我的minikube ip,32767是kafka服务的节点端口。
我收到以下错误消息:
>testmessage
>[2018-04-30 11:55:04,604] ERROR Error when sending message to topic ams_stream with key: null, value: 11 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for ams_stream-0: 1506 ms has passed since batch creation plus linger time
当我使用本地机器时,会收到以下警告:
[2018-04-30 10:22:30,680] WARN Connection to node 2 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-04-30 10:23:46,057] WARN Connection to node 8 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-04-30 10:25:01,542] WARN Connection to node 2 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-04-30 10:26:17,008] WARN Connection to node 5 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
经纪人的身份证是对的,看来我至少可以联系到经纪人
编辑:
我认为问题可能是,该服务是路由我“随机”到我的任何经纪人,但他需要路由我到领导的主题。这就是问题所在吗?有人知道解决这个问题的方法吗?
其他信息:
我用的是伍斯特/Kafka和数码仙境/Zookeeper的图片
我开始使用dellemc教程(以及defuze.org上的链接教程)
这不适合我,所以我在kafka-service.yml(1)和kafka-cluster.yml(2)中做了一些更改
Kafka服务.yml
添加了固定节点端口
已从选择器中删除id
Kafka群集.yml
向规范中添加了副本
已从标签中删除id
将代理id更改为由ip中的最后一个数字生成
将不推荐使用的值播发的\u主机\u名称/播发的\u端口替换为
k8s内部通信的侦听器(pod ip:9092)
用于与kubernetes外部的应用程序通信的播发的\u侦听器(minikube ip:node-port)
1-Kafka服务.yml:
---
apiVersion: v1
kind: Service
metadata:
name: kafka-service
labels:
name: kafka
spec:
type: NodePort
ports:
- port: 9092
nodePort: 32767
targetPort: 9092
protocol: TCP
selector:
app: kafka
type: LoadBalancer
2-kafka-cluster.yml:
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: kafka-b
spec:
replicas: 3
template:
metadata:
labels:
app: kafka
spec:
containers:
- name: kafka
image: wurstmeister/kafka
ports:
- containerPort: 9092
env:
- name: HOSTNAME_COMMAND
value: "ifconfig |grep 'addr:172' |cut -d':' -f 2 |cut -d ' ' -f 1"
- name: KAFKA_ZOOKEEPER_CONNECT
value: zk1:2181
- name: BROKER_ID_COMMAND
value: "ifconfig |grep 'inet addr:172' | cut -d'.' -f '4' | cut -d' ' -f '1'"
- name: KAFKA_ADVERTISED_LISTENERS
value: "INTERNAL://192.168.99.100:32767"
- name: KAFKA_LISTENERS
value: "INTERNAL://_{HOSTNAME_COMMAND}:9092"
- name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
value: "INTERNAL:PLAINTEXT"
- name: KAFKA_INTER_BROKER_LISTENER_NAME
value: "INTERNAL"
- name: KAFKA_CREATE_TOPICS
value: mytopic:1:3
暂无答案!
目前还没有任何答案,快来回答吧!