运行storm拓扑时invalidclassexception本地类不兼容

91zkwejq  于 2021-06-24  发布在  Storm
关注(0)|答案(1)|浏览(326)

在本地模式下运行时，运行单个拓扑的storm应用程序运行正常，但在分布式环境下运行时，我的工作日志中会出现这些异常。这是同一个例外，但它涉及两个不同的类。第一个是指storm.trident.topology.masterbatchcoordinator：

  1. java.lang.RuntimeException: java.io.InvalidClassException: storm.trident.topology.MasterBatchCoordinator; local class incompatible: stream classdesc serialVersionUID = -5647458900661444882, local class serialVersionUID = 7308128536282303612
  2.     at backtype.storm.utils.Utils.deserialize(Utils.java:68) ~[storm-core-0.9.0-wip21.jar:na]
  3.     at backtype.storm.utils.Utils.getSetComponentObject(Utils.java:200) ~[storm-core-0.9.0-wip21.jar:na]
  4.     at backtype.storm.daemon.task$get_task_object.invoke(task.clj:58) ~[storm-core-0.9.0-wip21.jar:na]
  5.     at backtype.storm.daemon.task$mk_task_data$fn__3164.invoke(task.clj:165) ~[storm-core-0.9.0-wip21.jar:na]
  6.     at backtype.storm.util$assoc_apply_self.invoke(util.clj:752) ~[storm-core-0.9.0-wip21.jar:na]
  7.     at backtype.storm.daemon.task$mk_task_data.invoke(task.clj:158) ~[storm-core-0.9.0-wip21.jar:na]
  8.     at backtype.storm.daemon.task$mk_task.invoke(task.clj:169) ~[storm-core-0.9.0-wip21.jar:na]
  9.     at backtype.storm.daemon.executor$mk_executor$fn__3352.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
  10.     at clojure.core$map$fn__4087.invoke(core.clj:2432) ~[clojure-1.4.0.jar:na]
  11.     at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
  12.     at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
  13.     at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
  14.     at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
  15.     at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.4.0.jar:na]
  16.     at clojure.core.protocols$fn__5875.invoke(protocols.clj:54) ~[clojure-1.4.0.jar:na]
  17.     at clojure.core.protocols$fn__5828$G__5823__5841.invoke(protocols.clj:13) ~[clojure-1.4.0.jar:na]
  18.     at clojure.core$reduce.invoke(core.clj:6030) ~[clojure-1.4.0.jar:na]
  19.     at clojure.core$into.invoke(core.clj:6077) ~[clojure-1.4.0.jar:na]
  20.     at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
  21.     at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863$iter__5868__5872$fn__5873.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
  22.     at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
  23.     at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
  24.     at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.4.0.jar:na]
  25.     at clojure.lang.RT.next(RT.java:587) ~[clojure-1.4.0.jar:na]
  26.     at clojure.core$next.invoke(core.clj:64) ~[clojure-1.4.0.jar:na]
  27.     at clojure.core$dorun.invoke(core.clj:2726) ~[clojure-1.4.0.jar:na]
  28.     at clojure.core$doall.invoke(core.clj:2741) ~[clojure-1.4.0.jar:na]
  29.     at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
  30.     at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
  31.     at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
  32.     at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
  33.     at backtype.storm.daemon.worker$fn__5862$mk_worker__5918.doInvoke(worker.clj:329) ~[storm-core-0.9.0-wip21.jar:na]
  34.     at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
  35.     at backtype.storm.daemon.worker$_main.invoke(worker.clj:439) ~[storm-core-0.9.0-wip21.jar:na]
  36.     at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
  37.     at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
  38.     at backtype.storm.daemon.worker.main(Unknown Source) ~[storm-core-0.9.0-wip21.jar:na]
  39. Caused by: java.io.InvalidClassException: storm.trident.topology.MasterBatchCoordinator; local class incompatible: stream classdesc serialVersionUID = -5647458900661444882, local class serialVersionUID = 7308128536282303612
  40.     at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) ~[na:1.7.0_25]
  41.     at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) ~[na:1.7.0_25]
  42.     at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) ~[na:1.7.0_25]
  43.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) ~[na:1.7.0_25]
  44.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) ~[na:1.7.0_25]
  45.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) ~[na:1.7.0_25]
  46.     at backtype.storm.utils.Utils.deserialize(Utils.java:64) ~[storm-core-0.9.0-wip21.jar:na]
  47.     ... 36 common frames omitted

第二个是backtype.storm.topology.basicboltexecutor：

  1. java.lang.RuntimeException: java.io.InvalidClassException: backtype.storm.topology.BasicBoltExecutor; local class incompatible: stream classdesc serialVersionUID = -2269346678248157705, local class serialVersionUID = 1040841720700290343
  2.     at backtype.storm.utils.Utils.deserialize(Utils.java:68) ~[storm-core-0.9.0-wip21.jar:na]
  3.     at backtype.storm.utils.Utils.getSetComponentObject(Utils.java:200) ~[storm-core-0.9.0-wip21.jar:na]
  4.     at backtype.storm.daemon.task$get_task_object.invoke(task.clj:58) ~[storm-core-0.9.0-wip21.jar:na]
  5.     at backtype.storm.daemon.task$mk_task_data$fn__3164.invoke(task.clj:165) ~[storm-core-0.9.0-wip21.jar:na]
  6.     at backtype.storm.util$assoc_apply_self.invoke(util.clj:752) ~[storm-core-0.9.0-wip21.jar:na]
  7.     at backtype.storm.daemon.task$mk_task_data.invoke(task.clj:158) ~[storm-core-0.9.0-wip21.jar:na]
  8.     at backtype.storm.daemon.task$mk_task.invoke(task.clj:169) ~[storm-core-0.9.0-wip21.jar:na]
  9.     at backtype.storm.daemon.executor$mk_executor$fn__3352.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
  10.     at clojure.core$map$fn__4087.invoke(core.clj:2432) ~[clojure-1.4.0.jar:na]
  11.     at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
  12.     at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
  13.     at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
  14.     at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
  15.     at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.4.0.jar:na]
  16.     at clojure.core.protocols$fn__5875.invoke(protocols.clj:54) ~[clojure-1.4.0.jar:na]
  17.     at clojure.core.protocols$fn__5828$G__5823__5841.invoke(protocols.clj:13) ~[clojure-1.4.0.jar:na]
  18.     at clojure.core$reduce.invoke(core.clj:6030) ~[clojure-1.4.0.jar:na]
  19.     at clojure.core$into.invoke(core.clj:6077) ~[clojure-1.4.0.jar:na]
  20.     at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
  21.     at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863$iter__5868__5872$fn__5873.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
  22.     at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
  23.     at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
  24.     at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
  25.     at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
  26.     at clojure.core$dorun.invoke(core.clj:2725) ~[clojure-1.4.0.jar:na]
  27.     at clojure.core$doall.invoke(core.clj:2741) ~[clojure-1.4.0.jar:na]
  28.     at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
  29.     at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
  30.     at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
  31.     at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
  32.     at backtype.storm.daemon.worker$fn__5862$mk_worker__5918.doInvoke(worker.clj:329) ~[storm-core-0.9.0-wip21.jar:na]
  33.     at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
  34.     at backtype.storm.daemon.worker$_main.invoke(worker.clj:439) ~[storm-core-0.9.0-wip21.jar:na]
  35.     at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
  36.     at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
  37.     at backtype.storm.daemon.worker.main(Unknown Source) ~[storm-core-0.9.0-wip21.jar:na]
  38. Caused by: java.io.InvalidClassException: backtype.storm.topology.BasicBoltExecutor; local class incompatible: stream classdesc serialVersionUID = -2269346678248157705, local class serialVersionUID = 1040841720700290343
  39.     at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) ~[na:1.7.0_25]
  40.     at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) ~[na:1.7.0_25]
  41.     at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) ~[na:1.7.0_25]
  42.     at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) ~[na:1.7.0_25]
  43.     at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) ~[na:1.7.0_25]
  44.     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) ~[na:1.7.0_25]
  45.     at backtype.storm.utils.Utils.deserialize(Utils.java:64) ~[storm-core-0.9.0-wip21.jar:na]
  46.     ... 35 common frames omitted
  47. 2013-09-19 07:42:19 b.s.util [INFO] Halting process: ("Error on initialization")

它不断出现在日志中，我的拓扑结构显然不起作用。我知道这个异常是在发现类的串行版本与类描述符的串行版本不同时引发的，但是我不知道是什么导致了它以及如何修复它。
以下是我的pom依赖项：

  1. <properties>
  2.     <org.springframework-version>3.2.3.RELEASE</org.springframework-version>
  3.     <storm-version>0.9.0-wip17</storm-version>
  4.     <storm-kafka-version>0.9.0-wip16a-scala292</storm-kafka-version>
  5.     <kafka-version>0.7.1-SNAPSHOT</kafka-version>
  6.     <cassandra.unit.version>1.0.3.1</cassandra.unit.version>
  7.     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  8. </properties>
  10. <repositories>
  11.     <repository>
  12.         <id>github-releases</id>
  13.         <url>http://oss.sonatype.org/content/repositories/github-releases/</url>
  14.     </repository>
  15.     <repository>
  16.         <id>clojars.org</id>
  17.         <url>http://clojars.org/repo</url>
  18.     </repository>
  19.     <repository>
  20.         <id>twitter4j</id>
  21.         <url>http://twitter4j.org/maven2</url>
  22.     </repository>
  23. </repositories>
  25. <dependencies>
  26.     <dependency>
  27.         <groupId>org.springframework</groupId>
  28.         <artifactId>spring-core</artifactId>
  29.         <version>${org.springframework-version}</version>
  30.         <exclusions>
  31.             <!-- Exclude Commons Logging in favor of SLF4j -->
  32.             <exclusion>
  33.                 <groupId>commons-logging</groupId>
  34.                 <artifactId>commons-logging</artifactId>
  35.             </exclusion>
  36.         </exclusions>
  37.     </dependency>
  38.     <dependency>
  39.         <groupId>storm</groupId>
  40.         <artifactId>storm</artifactId>
  41.         <version>${storm-version}</version>
  42.         <scope>provided</scope>
  43.     </dependency>
  44.     <dependency>
  45.         <groupId>storm</groupId>
  46.         <artifactId>storm-kafka</artifactId>
  47.         <version>${storm-kafka-version}</version>
  48.     </dependency>
  49.     <dependency>
  50.         <groupId>org.clojars.smallrivers</groupId>
  51.         <artifactId>kafka</artifactId>
  52.         <version>${kafka-version}</version>
  53.     </dependency>
  54.     <dependency>
  55.         <groupId>org.twitter4j</groupId>
  56.         <artifactId>twitter4j-core</artifactId>
  57.         <version>2.2.6-SNAPSHOT</version>
  58.     </dependency>
  59.     <dependency>
  60.         <groupId>org.twitter4j</groupId>
  61.         <artifactId>twitter4j-stream</artifactId>
  62.         <version>2.2.6-SNAPSHOT</version>
  63.     </dependency>
  64.     <dependency>
  65.         <groupId>com.googlecode.json-simple</groupId>
  66.         <artifactId>json-simple</artifactId>
  67.         <version>1.1</version>
  68.     </dependency>
  69.     <dependency>
  70.         <groupId>com.google.guava</groupId>
  71.         <artifactId>guava</artifactId>
  72.         <version>13.0.1</version>
  73.     </dependency>
  74.     <dependency>
  75.         <groupId>commons-collections</groupId>
  76.         <artifactId>commons-collections</artifactId>
  77.         <version>3.2.1</version>
  78.     </dependency>
  79.     <dependency>
  80.         <groupId>me.prettyprint</groupId>
  81.         <artifactId>hector-core</artifactId>
  82.         <version>1.0-4</version>
  83.     </dependency>
  84.     <!-- Tests -->
  85.     <dependency>
  86.         <groupId>junit</groupId>
  87.         <artifactId>junit</artifactId>
  88.         <version>4.11</version>
  89.     </dependency>
  90.     <dependency>
  91.         <groupId>org.yaml</groupId>
  92.         <artifactId>snakeyaml</artifactId>
  93.         <version>1.12</version>
  94.         <scope>test</scope>
  95.     </dependency>
  96.     <dependency>
  97.         <groupId>org.cassandraunit</groupId>
  98.         <artifactId>cassandra-unit</artifactId>
  99.         <version>${cassandra.unit.version}</version>
  100.         <scope>test</scope>
  101.     </dependency>
  102.     <dependency>
  103.         <groupId>org.mockito</groupId>
  104.         <artifactId>mockito-all</artifactId>
  105.         <version>1.9.5</version>
  106.         <scope>test</scope>
  107.     </dependency>
  108. </dependencies>

有什么想法吗？我在这里迷路了：\

serializationapache-stormtrident

1条答案

按热度按时间
ulmd4ohb

ulmd4ohb1#

有没有可能你混合了不同的风暴版本？错误日志引用了storm版本0.9.0-wip21，但storm版本0.9.0-wip17列在您的日志中的项目依赖项中 pom.xml 文件。集群上的所有组件都应该使用相同的storm版本。

赞(0）分享  回复(0）举报  2021-06-24
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com