spark:stackoverflowerr正在尝试将JavaRDD转换为Dataframe

bkkx9g8r  于 2021-06-10  发布在  Hbase
关注(0)|答案(0)|浏览(250)

我正在使用spark 2.0并尝试将JavaRDD转换为Dataframe。
下面是我正在使用的代码,我的bean有嵌套的bean。

JavaRDD<XXX> mappedRDD = hbaseRDD.map(s->{

 //final long serialVersionUID = -2021713021648730786L;

XXX xx=new XXX();

 return xx;

});
Dataset<Row> df = sparkSession.createDataFrame(mappedRDD, XXX.class);

我有以下错误

Exception in thread "main" java.lang.StackOverflowError
    at sun.reflect.generics.repository.GenericDeclRepository.getTypeParameters(GenericDeclRepository.java:84)
    at java.lang.Class.getTypeParameters(Class.java:715)
    at org.spark_project.guava.reflect.Types$ParameterizedTypeImpl.<init>(Types.java:288)
    at org.spark_project.guava.reflect.Types.newParameterizedType(Types.java:98)
    at org.spark_project.guava.reflect.TypeToken.toGenericType(TypeToken.java:917)
    at org.spark_project.guava.reflect.TypeToken.getSupertype(TypeToken.java:401)
    at org.apache.spark.sql.catalyst.JavaTypeInference$.elementType(JavaTypeInference.scala:132)
    at org.apache.spark.sql.catalyst.JavaTypeInference$.org$apache$spark$sql$catalyst$JavaTypeInference$$inferDataType(JavaTypeInference.scala:101)
    at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:117)
    at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:115)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186)
    at org.apache.spark.sql.catalyst.JavaTypeInference$.org$apache$spark$sql$catalyst$JavaTypeInference$$inferDataType(JavaTypeInference.scala:115)
    at org.apache.spark.sql.catalyst.JavaTypeInference$.org$apache$spark$sql$catalyst$JavaTypeInference$$inferDataType(JavaTypeInference.scala:101)
    at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:117)
    at org.apache.spark.sql.catalyst.JavaTypeInference$$anonfun$2.apply(JavaTypeInference.scala:115)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题