ksql-sink主题x

juzqafwq 于 2021-06-06 发布在 Kafka

关注(0)|答案(1)|浏览(452)

我正在尝试基于sourcetopic1和sourcetopic2在targettopic1中创建数据。两个源主题应该具有相同的事件结构。创建第一个目标流，然后尝试将数据从另一个源流插入当前流。有什么建议吗？

ksql> CREATE STREAM sourceTopic1Stream (category varchar, source varchar, type varchar, id varchar, payload varchar) WITH (KAFKA_TOPIC='sourceTopic1', VALUE_FORMAT='json');

 Message
----------------
 Stream created
----------------
ksql> CREATE STREAM sourceTopic2Stream (category varchar, source varchar, type varchar, id varchar, payload varchar) WITH (KAFKA_TOPIC='sourceTopic2', VALUE_FORMAT='json');

 Message
----------------
 Stream created
----------------
ksql> CREATE STREAM targetTopic1Stream WITH (kafka_topic='targetTopic1', partitions=3) AS select 'sourceTopic1' topicname, category, source, type, id, payload from sourceTopic1Stream where id like 'myid%';

 Message
----------------------------
 Stream created and running
----------------------------
ksql> INSERT INTO targetTopic1Stream SELECT 'sourceTopic2' topicname, category, source, type, id, payload FROM sourceTopic2Stream where id like 'myid%';
io.confluent.ksql.util.KsqlException: Sink topic TARGETTOPIC1STREAM does not exist in th e metastore.

ksql> show topics;

 Kafka Topic        | Registered | Partitions | Partition Replicas | Consumers | ConsumerGroups
------------------------------------------------------------------------------------------------
 _confluent-metrics | false      | 12         | 1                  | 0         | 0
 _schemas           | false      | 1          | 1                  | 0         | 0
 sourceTopic1       | true       | 3          | 1                  | 3         | 1
 sourceTopic2       | true       | 3          | 1                  | 0         | 0
 targetTopic1       | true       | 3          | 1                  | 0         | 0
------------------------------------------------------------------------------------------------
ksql> show streams;

 Stream Name        | Kafka Topic  | Format
--------------------------------------------
 SOURCETOPIC2STREAM | sourceTopic2 | JSON
 TARGETTOPIC1STREAM | targetTopic1 | JSON
 SOURCETOPIC1STREAM | sourceTopic1 | JSON
--------------------------------------------
ksql>

apache-kafka ksqldb

来源：https://stackoverflow.com/questions/53165123/ksql-sink-topic-xxxx-does-not-exist-in-th-e-metastore

1条答案

按热度按时间

qmelpv7a1#

这是ksql中的一个bug。我写在这里：https://github.com/confluentinc/ksql/issues/2123
解决方法是不指定 kafka_topic 在你的 CREATE STREAM … AS :

ksql>
ksql> CREATE STREAM TargetStream WITH (partitions=3) AS select 'sourceTopic1' topicname, category, source, type, id, payload from sourceTopic1Stream where id like 'myid%';

 Message
----------------------------
 Stream created and running
----------------------------
ksql> INSERT INTO TargetStream SELECT 'sourceTopic2' AS topicname, * FROM sourceTopic2Stream where id like 'myid%';

 Message
-------------------------------
 Insert Into query is running.
-------------------------------
ksql> SELECT * FROM TargetStream;
1541496149897 | null | sourceTopic2 | Foo2 | bar: | x | myid2 | asdf
1541496141671 | null | sourceTopic1 | Foo1 | bar: | x | myid1 | asdf

赞(0）回复(0）举报 2021-06-07

我来回答

ksql-sink主题x

1条答案

相关问题

热门标签

最新问答