Kafka身上嵌着Spark依赖关系问题

ssm49v7z  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(396)

我正在尝试使用spark streaming 2.0.0和kafka 0.10。我习惯了我的集成测试https://github.com/manub/scalatest-embedded-kafka 但是我在启动服务器时遇到了一些问题。当我尝试使用spark 2.2.0时,它是有效的。

<dependency>
        <groupId>net.manub</groupId>
        <artifactId>scalatest-embedded-kafka_2.11</artifactId>
        <version>${embedded-kafka.version}</version> -->I tried many versions.
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
        <version>2.0.2</version>
    </dependency>

An exception or error caused a run to abort: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time; 
java.lang.NoSuchMethodError: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
    at net.manub.embeddedkafka.EmbeddedKafkaSupport$class.startKafka(EmbeddedKafka.scala:467)
    at net.manub.embeddedkafka.EmbeddedKafka$.startKafka(EmbeddedKafka.scala:38)
    at net.manub.embeddedkafka.EmbeddedKafka$.start(EmbeddedKafka.scala:55)
    at iris.orange.ScalaTest$$anonfun$1.apply$mcV$sp(ScalaTest.scala:10)

这似乎是一个依赖的问题,但我没有去工作。我选择了一个使用相同Kafka版本的嵌入式Kafka。

3ks5zfa0

3ks5zfa01#

您需要使用正确版本的 spark-streaming-kafka https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10_2.10/2.0.0

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka-0-10_2.10</artifactId>
    <version>2.0.0</version>
</dependency>

相关问题