在sbt中运行fat jar和获取路径未找到异常

jm81lzqq  于 2021-07-14  发布在  Spark
关注(0)|答案(0)|浏览(327)

我有一个scala项目,它引用了在src/main/resources/myfile.csv中创建的文件。
我创建了一个胖jar,下面是我的build.sbt

name := "my-project"

version := "0.1"
scalaVersion := "2.11.12"

libraryDependencies ++= Seq(

  "org.apache.spark" %% "spark-core" % "2.4.7",
  "org.apache.spark" %% "spark-sql" % "2.4.7"
)

assemblyMergeStrategy in assembly := {
  case PathList("META-INF","services",xs @ _*) => MergeStrategy.filterDistinctLines
  case PathList("META-INF",xs @ _*) => MergeStrategy.discard
  case "application.conf" => MergeStrategy.concat
  case _ => MergeStrategy.first
}

但是当我试着运行我的胖jar时,我得到了以下错误
线程“main”org.apache.spark.sql.analysisexception中出现异常:路径不存在:file:/path/to/myproject/target/scala-2.11/src/main/resources/myfile.csv
我使用以下命令来运行jar

java -cp myProject.jar com.organization.myProject

有没有办法在build.sbt文件中包含我的文件/文件夹路径?
更新
我修改了build.sbt如下

name := "my-project"

    version := "0.1"
    scalaVersion := "2.11.12"

    libraryDependencies ++= Seq(

      "org.apache.spark" %% "spark-core" % "2.4.7",
      "org.apache.spark" %% "spark-sql" % "2.4.7"
    )

   scalaSource in Compile := baseDirectory.value / "main"

    assemblyMergeStrategy in assembly := {
      case PathList("META-INF","services",xs @ _*) => MergeStrategy.filterDistinctLines
      case PathList("META-INF",xs @ _*) => MergeStrategy.discard
      case "application.conf" => MergeStrategy.concat
      case _ => MergeStrategy.first
    }

但它仍然给路径找不到例外。有人有主意吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题