如何将openshift上的ksqldb集群连接到本地kafka集群

yi0zb3m4  于 2021-06-05  发布在  Kafka
关注(0)|答案(1)|浏览(436)

我想要达到的目标:
我们有一个内部Kafka集群。我想在openshift中设置ksqldb,并将其连接到本地kafka集群的代理。
问题是:
当我尝试使用命令“/usr/bin/ksql-server-start/etc/ksqldb/ksql-server.properties”启动ksqldb服务器时,会收到错误消息:

[2020-05-14 15:47:48,519] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:60)
io.confluent.ksql.util.KsqlServerException: Could not get Kafka cluster configuration!
        at io.confluent.ksql.services.KafkaClusterUtil.getConfig(KafkaClusterUtil.java:90)
        at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.isKafkaAuthorizerEnabled(KsqlAuthorizationValidatorFactory.java:81)
        at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.create(KsqlAuthorizationValidatorFactory.java:51)
        at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:624)
        at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:544)
        at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:98)
        at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:56)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1589471268517) timed out at 1589471268518 after 1 attempt(s)
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
        at io.confluent.ksql.services.KafkaClusterUtil.getConfig(KafkaClusterUtil.java:60)
        ... 6 more
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1589471268517) timed out at 1589471268518 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

我的配置:
我根据以下图像设置dockerfile:https://hub.docker.com/r/confluentinc/ksqldb-server,端口9092、9093、8080、8082和443打开。
我的服务是这样的:

kind: Service
apiVersion: v1
metadata:
  name: social-media-dev
  namespace: abc
  selfLink: xyz
  uid: xyz
  resourceVersion: '1'
  creationTimestamp: '2020-05-14T09:47:15Z'
  labels:
    app: social-media-dev
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
spec:
  ports:
    - name: social-media-dev
      protocol: TCP
      port: 9092
      targetPort: 9092
      nodePort: 31364
  selector:
    app: social-media-dev
    deploymentconfig: social-media-dev
  clusterIP: XX.XX.XXX.XXX
  type: LoadBalancer
  externalIPs:
    - XXX.XX.XXX.XXX
  sessionAffinity: None
  externalTrafficPolicy: Cluster
status:
  loadBalancer:
    ingress:
      - ip: XX.XX.XXX.XXX

我的ksql-server.properties文件包含以下信息:
听众:http://0.0.0.0:8082
bootstrap.servers:x.x.x.x:9092,x.x.x.y:9092,x.x.x.z:9092
到目前为止,我尝试了:
我试着从我的豆荚里连接到一个经纪人,结果成功了: (timeout 1 bash -c '</dev/tcp/X.X.X.X/9092 && echo PORT OPEN || echo PORT CLOSED') 2>/dev/null 结果:端口打开
我也和监听器一起玩了,但是错误消息变短了,只是信息“couldnotgetkafka clusterconfiguration!”没有超时错误。
我尝试将loadbalancer交换到nodeport,但也没有成功。
你知道我下一步可以试试什么吗?
更新:升级到cloudera cdh6后,cloudera kafka集群现在也可以与kafka streams一起工作。因此,我现在能够从openshift中的ksqldb集群连接到本地kafka集群。

qlvxas9a

qlvxas9a1#

更新:升级到cloudera cdh6后,cloudera kafka集群现在也可以与kafka streams一起工作。因此,我现在能够从openshift中的ksqldb集群连接到本地kafka集群。

相关问题