我正在使用 Log4j
与 Scala 2.12.12
以及 Spark-Core 3.0.1
但是,当我将库依赖项更改为不将spark core打包到胖jar中时,在尝试运行它时会出现以下错误:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/Logger
at com.some.package.name.Utils$.setup(Utils.scala:207)
at com.some.package.name.Main$.main(Main.scala:9)
at com.some.package.name.Main.main(Main.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Logger
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
编译成功,如果我删除 provided
从dependencies行中删除子句一切正常。我的 build.sbt
具体如下:
scalaVersion := "2.12.12"
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "3.0.1" % "provided",
"org.apache.logging.log4j" % "log4j-api" % "2.13.3",
"org.apache.logging.log4j" % "log4j-core" % "2.13.3",
"org.scalatest" %% "scalatest" % "3.2.0" % "test",
"com.holdenkarau" %% "spark-testing-base" % "3.0.1_1.0.0" % Test)
如果我删除写入记录器的代码 SparkContext
表示为产生错误的行。
暂无答案!
目前还没有任何答案,快来回答吧!