我试图用spark和sparklingwater创建一个简单的scala应用程序 build.sbt
文件:
name := "test"
version := "0.1"
scalaVersion := "2.11.12"
logLevel := Level.Error
lazy val versions = new {
val spark = "2.4.5"
val sparklingWater = "3.30.0.5-1-2.4"
val play = "2.7.5"
val scala = "2.11.12"
}
compileOrder in Compile := CompileOrder.JavaThenScala
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.11" % versions.spark,
"org.apache.spark" % "spark-mllib_2.11" % versions.spark,
"org.apache.spark" % "spark-sql_2.11" % versions.spark,
"org.apache.spark" % "spark-repl_2.11" % versions.spark,
"ai.h2o" % "sparkling-water-package_2.11" % versions.sparklingWater,
"org.scala-lang" % "scala-compiler" % versions.scala,
"org.scala-lang" % "scala-library" % versions.scala,
"org.scala-lang" % "scala-reflect" % versions.scala,
)
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
但是,当我尝试使用 H2OContext.getOrCreate()
,sbt抛出 Caused by: scala.reflect.internal.FatalError: package scala does not have a member Int
当我使用 run
sbt控制台中的命令。但是,当我直接运行main方法时,代码运行得很好。
以下是堆栈跟踪:
20/07/01 18:11:44 INFO H2OContext: Connecting to H2O cluster.
20/07/01 18:11:44 INFO RestApiUtils: H2O node http://192.168.10.81:54321/3/CloudLock successfully responded for the POST.
20/07/01 18:11:44 INFO RestApiUtils: H2O node http://192.168.10.81:54321/3/verifyWebOpen successfully responded for the GET.
20/07/01 18:11:44 INFO RestApiUtils: H2O node http://192.168.10.81:54321/3/verifyVersion?referenced_version=3.30.0.5 successfully responded for the GET.
20/07/01 18:11:44 INFO RestApiUtils: H2O node http://192.168.10.81:54321/3/Cloud successfully responded for the GET.
20/07/01 18:11:44 INFO RestApiUtils: H2O node http://192.168.10.81:54321/99/Rapids successfully responded for the POST.
20/07/01 18:11:44 INFO SignalUtils: Registered signal handler for INT
[error] (run-main-0) java.lang.ExceptionInInitializerError
[error] java.lang.ExceptionInInitializerError
[error] at com.demo.DeepLearningDemo.main(DeepLearningDemo.scala)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:498)
[error] Caused by: scala.reflect.internal.FatalError: package scala does not have a member Int
[error] at scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1182)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1199)
[error] at scala.reflect.internal.Definitions$ValueClassDefinitions$class.valueClassSymbol(Definitions.scala:77)
[error] at scala.reflect.internal.Definitions$ValueClassDefinitions$class.IntClass(Definitions.scala:114)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.IntClass$lzycompute(Definitions.scala:153)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.IntClass(Definitions.scala:153)
[error] at scala.reflect.internal.Definitions$ValueClassDefinitions$class.IntTpe(Definitions.scala:127)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.IntTpe$lzycompute(Definitions.scala:153)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.IntTpe(Definitions.scala:153)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.sig2type$1(ClassfileParser.scala:630)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.sig2type$1(ClassfileParser.scala:727)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.scala$tools$nsc$symtab$classfile$ClassfileParser$$sigToType(ClassfileParser.scala:773)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser$ConstantPool.getType(ClassfileParser.scala:289)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.parseMethod(ClassfileParser.scala:568)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.scala$tools$nsc$symtab$classfile$ClassfileParser$$queueLoad$1(ClassfileParser.scala:483)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser$$anonfun$parseClass$1.apply$mcV$sp(ClassfileParser.scala:493)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.parseClass(ClassfileParser.scala:498)
[error] at scala.tools.nsc.symtab.classfile.ClassfileParser.parse(ClassfileParser.scala:139)
[error] at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader$$anonfun$doComplete$2.apply$mcV$sp(SymbolLoaders.scala:347)
[error] at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader$$anonfun$doComplete$2.apply(SymbolLoaders.scala:347)
[error] at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader$$anonfun$doComplete$2.apply(SymbolLoaders.scala:347)
[error] at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:235)
[error] at scala.tools.nsc.symtab.SymbolLoaders$ClassfileLoader.doComplete(SymbolLoaders.scala:347)
[error] at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:211)
[error] at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535)
[error] at scala.reflect.internal.Symbols$Symbol.initialize(Symbols.scala:1680)
[error] at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
[error] at scala.tools.nsc.Global$Run.<init>(Global.scala:1242)
[error] at scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139)
[error] at scala.tools.nsc.interpreter.IMain.global$lzycompute(IMain.scala:168)
[error] at scala.tools.nsc.interpreter.IMain.global(IMain.scala:167)
[error] at scala.tools.nsc.interpreter.IMain$naming$.<init>(IMain.scala:196)
[error] at scala.tools.nsc.interpreter.IMain.naming$lzycompute(IMain.scala:195)
[error] at scala.tools.nsc.interpreter.IMain.naming(IMain.scala:195)
[error] at ai.h2o.sparkling.repl.H2OIMain.<init>(H2OIMain.scala:35)
[error] at ai.h2o.sparkling.repl.H2OIMain$.createInterpreter(H2OIMain.scala:48)
[error] at ai.h2o.sparkling.repl.H2OInterpreter.createInterpreter(H2OInterpreter.scala:42)
[error] at ai.h2o.sparkling.repl.BaseH2OInterpreter.initializeInterpreter(BaseH2OInterpreter.scala:110)
[error] at ai.h2o.sparkling.repl.BaseH2OInterpreter.<init>(BaseH2OInterpreter.scala:300)
[error] at ai.h2o.sparkling.repl.H2OInterpreter.<init>(H2OInterpreter.scala:39)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet.ai$h2o$sparkling$backend$api$scalainterpreter$ScalaInterpreterServlet$$createInterpreterInPool(ScalaInterpreterServlet.scala:101)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet$$anonfun$initializeInterpreterPool$1.apply(ScalaInterpreterServlet.scala:95)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet$$anonfun$initializeInterpreterPool$1.apply(ScalaInterpreterServlet.scala:94)
[error] at scala.collection.immutable.Range.foreach(Range.scala:160)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet.initializeInterpreterPool(ScalaInterpreterServlet.scala:94)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet.<init>(ScalaInterpreterServlet.scala:48)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet$.getServlet(ScalaInterpreterServlet.scala:145)
[error] at ai.h2o.sparkling.backend.api.ServletRegister$class.register(ServletRegister.scala:30)
[error] at ai.h2o.sparkling.backend.api.scalainterpreter.ScalaInterpreterServlet$.register(ScalaInterpreterServlet.scala:142)
[error] at water.webserver.jetty9.SparklingWaterJettyHelper.createServletContextHandler(SparklingWaterJettyHelper.scala:41)
[error] at water.webserver.jetty9.SparklingWaterJettyHelper.startServer(SparklingWaterJettyHelper.scala:90)
[error] at ai.h2o.sparkling.backend.utils.ProxyStarter$.startFlowProxy(ProxyStarter.scala:42)
[error] at org.apache.spark.h2o.H2OContext.<init>(H2OContext.scala:98)
[error] at org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:481)
[error] at org.apache.spark.h2o.H2OContext$.getOrCreate(H2OContext.scala:519)
[error] at com.demo.DemoWrapper$class.$init$(DemoWrapper.scala:69)
[error] at com.demo.DeepLearningDemo$.<init>(DeepLearningDemo.scala:9)
[error] at com.demo.DeepLearningDemo$.<clinit>(DeepLearningDemo.scala)
[error] at com.demo.DeepLearningDemo.main(DeepLearningDemo.scala)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:498)
[error] stack trace is suppressed; run last Compile / bgRun for the full output
与这个问题相关的其他答案似乎指向scala和play框架之间的问题(在我的应用程序中,我并不直接依赖于这个框架)。有办法解决这个问题吗?这是scala或起泡水的问题吗?
暂无答案!
目前还没有任何答案,快来回答吧!