我在ApacheFlink1.8.1中有一个项目,使用Scala2.11和Java8。我以前使用maven进行编译和所有的依赖关系管理,但改用gradle。。。这就引出了下面这个问题:
j.l.ClassNotFoundException: om.tinker.my.project.ProjectPayload
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
... 3 frames excluded
at c.e.k.u.DefaultClassResolver.readName(DefaultClassResolver.java:172)
... 15 common frames omitted
Wrapped by: c.e.kryo.KryoException: Unable to find class: om.tinker.my.project.ProjectPayload
Serialization trace:
eventOutputTag (com.my.project.contexts.ProjectContext)
at c.e.k.u.DefaultClassResolver.readName(DefaultClassResolver.java:178)
at c.e.k.u.DefaultClassResolver.readClass(DefaultClassResolver.java:147)
at c.e.kryo.Kryo.readClass(Kryo.java:674)
at c.e.k.s.ReflectField.read(ReflectField.java:107)
at c.e.k.s.FieldSerializer.read(FieldSerializer.java:122)
at c.e.kryo.Kryo.readClassAndObject(Kryo.java:793)
at o.a.f.a.j.t.r.k.KryoSerializer.deserialize(KryoSerializer.java:346)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:202)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:46)
at o.a.f.r.p.NonReusingDeserializationDelegate.read(NonReusin...
首先,错误消息缺少“c”。类路径应该是“com.tinker.my.project.projectpayload”。。。我用那个代码检查了文件,我的import语句中没有丢失的c。。。
我还编辑了flink conf文件以使用父级优先策略。。。
进一步的背景信息:我有另一个文件叫做 ProjectContext
它有一个 ArrayList<ProjectPayload>
. 它还有 eventOutputTag
(如序列化跟踪中所述)。。。当我发表评论时 ArrayList<ProjectPayload>
它的创始者/创始者,一切正常!
当我把示例变量及其getters/setters放回 ProjectContext
,那么 ClassNotFoundException
发生。。。
此外,我还添加了大量的print语句,创建了projectpayload的示例,并很好地将其注销。
编辑(2020年6月30日)###
鉴于此序列化问题,我添加了以下代码: env.getConfig.registerTypeWithKryoSerializer(classOf[ProjectPayload], classOf[JavaSerializer[ProjectPayload]])
现在我有一个尴尬的(但类似的)错误:
"j.l.ClassNotFoundException: \u0005sr\u00008com.tinker.my.project.ProjectPayload+\"v
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
... 3 frames excluded
at c.e.k.u.DefaultClassResolver.readName(DefaultClassResolver.java:172)
... 15 common frames omitted
Wrapped by: c.e.kryo.KryoException: Unable to find class: \u0005sr\u00008com.tinker.my.project.ProjectPayload+\"v
Serialization trace:
allMyPayloads (com.tinker.my.project.ProjectContext)
at c.e.k.u.DefaultClassResolver.readName(DefaultClassResolver.java:178)
at c.e.k.u.DefaultClassResolver.readClass(DefaultClassResolver.java:147)
at c.e.kryo.Kryo.readClass(Kryo.java:674)
at c.e.k.s.ReflectField.read(ReflectField.java:107)
at c.e.k.s.FieldSerializer.read(FieldSerializer.java:122)
at c.e.kryo.Kryo.readClassAndObject(Kryo.java:793)
at o.a.f.a.j.t.r.k.KryoSerializer.deserialize(KryoSerializer.java:346)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:202)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:46)
at o.a.f.r.p.NonReusingDeserializationDelegate....
结果是 \u0005
是unicode字符“inquiry”。以及 \u00008
导致谷歌搜索结果乱七八糟。。。稍后再汇报
编辑(2020年7月1日)####一些进展:我正在初始化 ArrayList<ProjectPayload>
内部 ProjectContext
. 当我删除了初始化,将其移到外部,然后设置arraylist值时,我的代码走得更远了。然后它抱怨 HashMap<String, String>
示例变量——我最终删除了它,因为它没有被使用。
这让我想到 IndexOutOfBoundsException
:
j.l.IndexOutOfBoundsException: Index: 93, Size: 9
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at c.e.k.u.MapReferenceResolver.getReadObject(MapReferenceResolver.java:62)
at c.e.kryo.Kryo.readReferenceOrNull(Kryo.java:838)
at c.e.kryo.Kryo.readObjectOrNull(Kryo.java:761)
at c.e.k.s.ReflectField.read(ReflectField.java:120)
... 12 common frames omitted
Wrapped by: c.e.kryo.KryoException: java.lang.IndexOutOfBoundsException: Index: 93, Size: 9
Serialization trace:
fooBarStr (com.tinker.my.project.contexts.ProjectContext)
at c.e.k.s.ReflectField.read(ReflectField.java:133)
at c.e.k.s.FieldSerializer.read(FieldSerializer.java:122)
at c.e.kryo.Kryo.readClassAndObject(Kryo.java:793)
at o.a.f.a.j.t.r.k.KryoSerializer.deserialize(KryoSerializer.java:346)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:202)
at o.a.f.s.r.s.StreamElementSerializer.deserialize(StreamElementSerializer.java:46)
at o.a.f.r.p.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55)
at o.a.f.r.i.n.a.s.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRec...
关于kryo的github问题:https://github.com/esotericsoftware/kryo/issues/456
1条答案
按热度按时间6psbrbz91#
试试这个:
并确保您正在导入
org.apache.flink.api.java.typeutils.runtime.kryo.JavaSerializer
https://ci.apache.org/projects/flink/flink-docs-stable/dev/custom_serializers.html#issue-使用kryos javaserializer