spring cloud data flow>stream>source在k8s中部署但在本地部署时抛出未知的\u主题\u或\u分区

lh80um4z  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(295)

我正试图运行一个“hello,world”spring云数据流,它基于http://cloud.spring.io/spring-cloud-dataflow/. 我能够创建一个简单的源和接收器,并使用kafka在本地scdf服务器上运行它,所以在这里之前一切都是正确的。
现在,我正尝试根据上列出的说明将其部署到我的私有云中http://docs.spring.io/spring-cloud-dataflow-server-kubernetes/docs/current-snapshot/reference/htmlsingle/#_getting_started. 使用这个部署,我可以部署一个简单的“time | log”开箱即用流,没有任何问题,但是我的示例失败了。
具体版本有:
docker版本1.13.1,内部版本092cba3
hyperkube 1.5.5版
scdf 1.2.0.2平方米
zookeeper 3.4.9-1757313,于2016年8月23日06:50 gmt建成
Kafka0.10.1.1
源工件日志为:
2017-04-06t11:05:07.429204866z 2017-04-06 11:05:07428信息主发送线程(10.0.0.181:2181)o.a.z.clientcnxn:876 - 根据10.0.0.181/10.0.0.181:2181建立插座连接,启动会话2017-04-06t11:05:07.440381666z 2017-04-06 11:05:07,439信息主发送线程(10.0.0.181:2181)o.a.z.clientcnxn:1299 - 在服务器10.0.0.181/10.0.0.181:2181上完成会话建立,sessionid=0x15b155ef61e014a,协商超时=10000 2017-04-06t11:05:07.740130495z 2017-04-06 11:05:07,737信息主o.a.k.c.p.producerconfig:180 - 产品配置值:2017-04-06t11:05:07.740160464z acks=1 2017-04-06t11:05:07.740163408z batch.size=16384 2017-04-06t11:05:07.740165226z block.on.buffer.full=false 2017-04-06t11:05:07.740166942z bootstrap.servers=[10.0.0.213:9092]2017-04-06t11:05:07.740168741z buffer.memory=335544322017-04-06t11:05:07.740170545z client.id=2017-04-06t11:05:07.740172245z compression.type=none 2017-04-06t11:05:07.740173971z connections.max.idle.ms=540000 2017-04-06t11:05:07.740175706z interceptor.classes=null 2017-04-06t11:05:07.744179899z reconnect.backoff.ms=50 2017-04-06t11:05:07.744181600z request.timeout.ms=300002017-04-06t11:05:07.744183356z retries=0 2017-04-06t11:05:07.744185083z retry.backoff.ms=100 2017-04-06t11:05:07.744186754z sasl.kerberos.kinit.cmd=/usr/bin/kinit 2017-04-06t11:05:07.744188494z sasl.kerberos.min.time.before.relogin=60000 2017-04-06t11:05:07.744190205z sasl.kerberos.service.name=null 2017-04-06t11:05:07.7441916Zsasl.kerberos.ticket.renew.jitter=0.05 2017-04-06t11:05:07.744193763z sasl.kerberos.ticket.renew.window.factor=0.8 2017-04-06t11:05:07.744195432z sasl.mechanism=gssapi 2017-04-06t11:05:07.744197163z security.protocol=plaintext 2017-04-06t11:05:07.744198789z send.buffer.bytes=131072 2017-04-06t11:05:07.744200522z ssl.cipher.suites=null2017-04-06t11:05:07.744202328z ssl.enabled.protocols=[tlsv1.2,tlsv1.1,tlsv1]2017-04-06t11:05:07.744204161z ssl.endpoint.identification.algorithm=null 2017-04-06t11:05:07.744205837z ssl.key.password=null 2017-04-06t11:05:07.744207544z ssl.keymanager.algorithm=sunx509 2017-04-06t11:05:07.744212464z ssl.keystore.location=null 2017-04-06t11:05:07.744214272z ssl.keystore.password=null2017-04-06t11:05:07.744216025z ssl.keystore.type=jks 2017-04-06t11:05:07.744217647z ssl.protocol=tls 2017-04-06t11:05:07.744219234z ssl.provider=null 2017-04-06t11:05:07.744220987z ssl.secure.random.implementation=null 2017-04-06t11:05:07.744222666z ssl.trustmanager.algorithm=pkix 2017-04-06t11:05:07.744224359z ssl.truststore.location=null 2017-04-06t11:05:07.744226022z ssl.truststore.password=null 2017-04-06t11:05:07.744228171z ssl.truststore.type=jks 2017-04-06t11:05:07.744230006z timeout.ms=30000 2017-04-06t11:05:07.744231705z value.serializer=class org.apache.kafka.common.serialization.bytearryserializer 2017-04-06t11:05:07.744233544z 2017-04-06t11:05:07.837193978z2017-04-06 11:05:07834警告主o.a.k.c.p.producerconfig:188 - 提供了配置“key.deserializer”,但不是已知配置。2017-04-06t11:05:07.837221870z 2017-04-06 11:05:07835警告主o.a.k.c.p.producerconfig:188 - 提供了配置“value.deserializer”,但不是已知配置。2017-04-06t11:05:07.929207703z 2017-04-06 11:05:07926主要信息o.a.k.c.u.appinfoparser:83 - Kafka版本:0.10.1.1 2017-04-06t11:05:07.929239636z 2017-04-06 11:05:07927主要信息o.a.k.c.u.appinfoparser:84 - Kafka委员会:f10ef2720b03b247 2017-04-06t11:05:08.228817026z 2017-04-06 11:05:08,228警告Kafka生产者网络线程|生产者-1 o.a.k.c.networkclient:600 - 获取相关id为0的元数据时出错:{output=unknown\u topic\u or\u partition}2017-04-06t11:05:08.436574800z 2017-04-06 11:05:08,435警告Kafka生产者网络线程|生产者-1 o.a.k.c.networkclient:600 - 获取相关id为1的元数据时出错:{output=unknown\u topic\u或\u partition}
zookepeer日志包括:
2017-04-06t11:04:38.000953447z 2017-04-06 11:04:38000[myid:]-信息[processthread](sid:0 cport:2181)::preprequestprocessor@487]-已处理sessionid的会话终止:0x15b155ef61e0148 2017-04-06t11:05:04.939356606z 2017-04-06 11:05:04,938[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:nioservercnxnfactory@192]-接受插座连接,从/10.1.98.5:48180 2017-04-06t11:05:04.940666418z 2017-04-06 11:05:04,939[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:zookeeperserver@928]-客户试图建立新会话,电话:10.1.98.5:48180 2017-04-06t11:05:04.943859474z 2017-04-06 11:05:04,943[myid:]-信息[syncthread:0:zookeeperserver@673]-为客户端建立会话0x15b155ef61e0149,协商超时10000/10.1.98.5:48180 2017-04-06t11:05:07.325929074z 2017-04-06 11:05:07,325[myid:]-信息[processthread](sid:0 cport:2181)::preprequestprocessor@487]-已处理sessionid的会话终止:0x15b155ef61e0149 2017-04-06t11:05:07.342876962z 2017-04-06 11:05:07,341[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:nioservercnxn@1008]-客户端的闭合插座连接/10.1.98.5:48180,会话ID为0x15b155ef61e0149 2017-04-06t11:05:07.429909440z 2017-04-06 11:05:07,429[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:nioservercnxnfactory@192]-接受插座连接,从/10.1.98.5:48182 2017-04-06t11:05:07.429933377z 2017-04-06 11:05:07,429[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:zookeeperserver@928]-客户试图建立新会话,电话:10.1.98.5:48182 2017-04-06t11:05:07.441158222z 2017-04-06 11:05:07,439[myid:]-信息[syncthread:0:zookeeperserver@673]-为客户端建立会话0x15b155ef61e014a,协商超时10000/10.1.98.5:48182 2017-04-06t11:05:29.695276997z 2017-04-06 11:05:29,694[myid:]-警告[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:nioservercnxn@357]-捕获到流结束异常2017-04-06t11:05:29.695325790z endofstreamexception:无法从客户端会话ID 0x15b155ef61e014a读取其他数据,可能客户端已关闭socket 2017-04-06t11:05:29.695328912z,位于org.apache.zookeeper.server.nioservercnxn.doio(nioservercnxn)。java:228)2017-04-06t11:05:29.695331119z网址:org.apache.zookeeper.server.nioservercnxnfactory.run(nioservercnxnfactory)。java:203)2017-04-06t11:05:29.695333009z at java.lang.thread.run(线程。java:745) 2017-04-06t11:05:29.696333706z 2017-04-06 11:05:29696[myid:]-信息[nioservercxn。factory:0.0.0.0/0.0.0.0:2181:nioservercnxn@1008]-客户端的封闭套接字连接
我在Kafka找不到任何日志在那一刻的例外。
源类的代码段是

package xxxx;

import java.text.SimpleDateFormat;
import java.util.Date;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.messaging.Source;
import org.springframework.context.annotation.Bean;
import org.springframework.integration.annotation.InboundChannelAdapter;
import org.springframework.integration.core.MessageSource;
import org.springframework.messaging.support.GenericMessage;

@SpringBootApplication
@EnableBinding(Source.class)
public class HelloNitesApplication
{
    public static void main(String[] args)
    {
        SpringApplication.run(HelloNitesApplication.class, args);
    }

    @Bean
    @InboundChannelAdapter(value = Source.OUTPUT)
    public MessageSource<String> timerMessageSource()
    {
        return () -> new GenericMessage<>("Hello " + new SimpleDateFormat().format(new Date()));
    }

所以,包含流源的豆荚在一个循环中不断崩溃。

gk7wooem

gk7wooem1#

问题似乎是我的实现忽略了属性“spring.cloud.stream.bindings.output.destination=”,我在执行之前删除了主题“output”,正如我期望它在属性指定的主题中写入的那样。
在我重新部署了所有内容之后,源代码会随着主题的正确创建而工作,尽管在“output”主题中插入了消息,而不是我定义的属性指定的消息。

相关问题