为什么我会得到 java.lang.NoClassDefFoundError spark提交时出错？stacktrace如下所示
[appadm@elk01 spark]$ bin/spark-submit --class "com.ipponusa.SparkStringConsumer" --master localhost:9092 samples/my-app-1.0-SNAPSHORT.jar 1
Hello
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
at kafka.utils.Pool.<init>(Pool.scala:26)
at kafka.consumer.FetchRequestAndResponseStatsRegistry$.<init>(FetchRequestAndResponseStats.scala:61)
at kafka.consumer.FetchRequestAndResponseStatsRegistry$.<clinit>(FetchRequestAndResponseStats.scala)
at kafka.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:44)
at org.apache.spark.streaming.kafka.KafkaCluster.connect(KafkaCluster.scala:52)
at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$org$apache$spark$streaming$kafka$KafkaCluster$$withBrokers$1.apply(KafkaCluster.scala:345)
at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$org$apache$spark$streaming$kafka$KafkaCluster$$withBrokers$1.apply(KafkaCluster.scala:342)
1条答案
按热度按时间f45qwnt81#
您必须使用与您编写spark代码的scala版本相同的版本。检查它们，可能需要降级scala版本。