我正在使用eclipseide使用yarn客户机模式向集群提交远程spark作业。
但是在eclipseide中得到下面的错误消息。
java.lang.NoClassDefFoundError: com/thoughtworks/paranamer/BytecodeReadingParanamer
com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.<init>(BeanIntrospector.scala:40)
com.fasterxml.jackson.module.scala.introspect.BeanIntrospector$.<clinit>(BeanIntrospector.scala)
com.fasterxml.jackson.module.scala.introspect.ScalaPropertiesCollector.<init>(ScalaPropertiesCollector.scala:22)
com.fasterxml.jackson.module.scala.introspect.ScalaClassIntrospector$.constructPropertyCollector(ScalaClassIntrospector.scala:24)
com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.collectProperties(BasicClassIntrospector.java:142)
com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forSerialization(BasicClassIntrospector.java:68)
com.fasterxml.jackson.databind.introspect.BasicClassIntrospector.forSerialization(BasicClassIntrospector.java:11)
com.fasterxml.jackson.databind.SerializationConfig.introspect(SerializationConfig.java:490)
com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:133)
com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:873)
com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:833)
com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:387)
com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:478)
com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:97)
com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:2718)
com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2210)
org.apache.spark.rdd.RDDOperationScope.toJson(RDDOperationScope.scala:51)
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:144)
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
org.apache.spark.SparkContext.withScope(SparkContext.scala:714)
org.apache.spark.SparkContext.textFile(SparkContext.scala:830)
org.apache.spark.api.java.JavaSparkContext.textFile(JavaSparkContext.scala:181)
com.nuevora.core.spark.commons.CommonFunctions.getJavaRDDFromFile(CommonFunctions.java:61)
com.nuevora.core.spark.UpdateDataset.modifyInputDataset(UpdateDataset.java:103)
com.nuevora.controllers.FormsValidatorServlet.service(FormsValidatorServlet.java:3070)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
执行器日志没有帮助。但job的最终状态是“成功”
需要帮助解决此错误吗
1条答案
按热度按时间jv2fixgn1#
我通过添加cloudera发行版附带的paranamer jar解决了这个问题