knative事件kafkasource问题

snvhrwxg  于 2021-06-05  发布在  Kafka
关注(0)|答案(0)|浏览(226)

我对knative eventing有一个问题,我希望能得到你的帮助,因为我对它还不熟悉。我使用的是knativev0.10.0、minikubev1.5.2、docker v19.03.1和kafka 2.3.1(用作事件源)。我使用kafkajava客户端api和spring boot开发了一个kafka生产者和一个kafka消费者。它们在管道中工作,因此我有一个cronjobsource,它每x分钟唤醒一个生产者,生产者将一些事件发送到一个kafka主题,然后我配置了一个kafkasource,它侦听该主题并应该唤醒消费者。实际上,kafkasource什么都不做,即使一切都配置得很好,我试着记录kafkasource,但它没有错误或成功的日志。如果我不通过kafkasource手动唤醒生产者和消费者,他们都可以很好地工作。
这是我的kafkasource yaml文件配置:

apiVersion: sources.eventing.knative.dev/v1alpha1
kind: KafkaSource
metadata:
  name: ods-datamartextractor-kafka-source
spec:
  consumerGroup: ods-dataconverter-consumer-group-id
  bootstrapServers: my-cluster-kafka-bootstrap.kafka:9092
  topics: datamartextractor-topic
  sink:
    ref:
        apiVersion: serving.knative.dev/v1
        kind: Service
        name: ods-dataconverter-service

这是我的Kafka制作人代码:

@Service
public class ODSDataMartExtractor {

public void sendEvent(DataMart dataMart) {
        String event = dataMart.toString();
        logger.info(String.format("#### -> Producing message -> " + event));
        ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send("datamartextractor-topic", event);
        future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
            @Override
            public void onSuccess(SendResult<String, String> result) {
                System.out.println("Sent message=[" + event + "] with offset=[" + result.getRecordMetadata().offset() + "]");
            }
            @Override
            public void onFailure(Throwable ex) {
                System.out.println("Unable to send message=[" + event + "] due to : " + ex.getMessage());
            }
        });
    }
}

Kafka制作人配置:

@Configuration
public class KafkaProducerConfig {

    @Bean
    public ProducerFactory<String, String> producerFactory() {
        Map<String, Object> configProps = new HashMap<>();
        configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "my-cluster-kafka-bootstrap.kafka:9092");
        configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        return new DefaultKafkaProducerFactory<>(configProps);
    }

    @Bean
    public KafkaTemplate<String, String> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }
}

Kafka消费代码:

@Service
public class ODSDataConverter {

@KafkaListener(topics = "datamartextractor-topic", groupId = "ods-dataconverter-consumer-group-id")
    public void consumeEvent(String event) throws IOException {
        logger.info(String.format("#### -> Consumed message -> " + event));
    }
}

Kafka消费者配置:

@EnableKafka
@Configuration
public class KafkaConsumerConfig {

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "my-cluster-kafka-bootstrap.kafka:9092");
        props.put(ConsumerConfig.GROUP_ID_CONFIG, "ods-dataconverter-consumer-group-id");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        return new DefaultKafkaConsumerFactory<>(props);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

谢谢您!

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题