配置单元2.3.2中kryo库中的stackoverflow错误

xt0899hw  于 2021-06-26  发布在  Hive
关注(0)|答案(0)|浏览(764)

我正在将emr作业从emr 3.7.0迁移到5.13.0。我们在配置单元脚本中使用brickhouse udf函数。在emr5.13.0上,我在kryo库中遇到了stackoverflow错误。深入研究后,我发现这个问题(链接)在2015年被修复到kryo图书馆。我在Hive2.3.2中遇到以下异常。

Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Unable to create serializer "org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicReference
Serialization trace:
_tableInfo (org.codehaus.jackson.sym.BytesToNameCanonicalizer)
_rootByteSymbols (org.codehaus.jackson.JsonFactory)
jsonFactory (brickhouse.udf.json.ToJsonUDF)
genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator)

然后

Caused by: java.lang.StackOverflowError
at java.util.HashMap.hash(HashMap.java:339)
at java.util.HashMap.get(HashMap.java:557)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:61)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
at org.apache.hive.com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)

此堆栈跟踪包含作为kryo库修复程序的一部分删除的代码(链接)。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题