我想要达到的目标:
我们有一个内部Kafka集群。我想在openshift中设置ksqldb,并将其连接到本地kafka集群的代理。
问题是:
当我尝试使用命令“/usr/bin/ksql-server-start/etc/ksqldb/ksql-server.properties”启动ksqldb服务器时,会收到错误消息:
[2020-05-14 15:47:48,519] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:60)
io.confluent.ksql.util.KsqlServerException: Could not get Kafka cluster configuration!
at io.confluent.ksql.services.KafkaClusterUtil.getConfig(KafkaClusterUtil.java:90)
at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.isKafkaAuthorizerEnabled(KsqlAuthorizationValidatorFactory.java:81)
at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.create(KsqlAuthorizationValidatorFactory.java:51)
at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:624)
at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:544)
at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:98)
at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:56)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1589471268517) timed out at 1589471268518 after 1 attempt(s)
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at io.confluent.ksql.services.KafkaClusterUtil.getConfig(KafkaClusterUtil.java:60)
... 6 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1589471268517) timed out at 1589471268518 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
我的配置:
我根据以下图像设置dockerfile:https://hub.docker.com/r/confluentinc/ksqldb-server,端口9092、9093、8080、8082和443打开。
我的服务是这样的:
kind: Service
apiVersion: v1
metadata:
name: social-media-dev
namespace: abc
selfLink: xyz
uid: xyz
resourceVersion: '1'
creationTimestamp: '2020-05-14T09:47:15Z'
labels:
app: social-media-dev
annotations:
openshift.io/generated-by: OpenShiftNewApp
spec:
ports:
- name: social-media-dev
protocol: TCP
port: 9092
targetPort: 9092
nodePort: 31364
selector:
app: social-media-dev
deploymentconfig: social-media-dev
clusterIP: XX.XX.XXX.XXX
type: LoadBalancer
externalIPs:
- XXX.XX.XXX.XXX
sessionAffinity: None
externalTrafficPolicy: Cluster
status:
loadBalancer:
ingress:
- ip: XX.XX.XXX.XXX
我的ksql-server.properties文件包含以下信息:
听众:http://0.0.0.0:8082
bootstrap.servers:x.x.x.x:9092,x.x.x.y:9092,x.x.x.z:9092
到目前为止,我尝试了:
我试着从我的豆荚里连接到一个经纪人,结果成功了: (timeout 1 bash -c '</dev/tcp/X.X.X.X/9092 && echo PORT OPEN || echo PORT CLOSED') 2>/dev/null
结果:端口打开
我也和监听器一起玩了,但是错误消息变短了,只是信息“couldnotgetkafka clusterconfiguration!”没有超时错误。
我尝试将loadbalancer交换到nodeport,但也没有成功。
你知道我下一步可以试试什么吗?
更新:升级到cloudera cdh6后,cloudera kafka集群现在也可以与kafka streams一起工作。因此,我现在能够从openshift中的ksqldb集群连接到本地kafka集群。
1条答案
按热度按时间qlvxas9a1#
更新:升级到cloudera cdh6后,cloudera kafka集群现在也可以与kafka streams一起工作。因此,我现在能够从openshift中的ksqldb集群连接到本地kafka集群。