为什么我的build.sbt要寻找hadoop流媒体的2.11版本?

kg7wmglp  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(340)

我正在做一个关于apachespark的pluralsight课程,有一次他们要求我们建立一个对hadoop流的依赖关系。我已经把它添加到我的 build.sbt 但我得到的结果出乎意料:

构建.sbt

name := "SparkPlayground"

version := "1.0"

scalaVersion := "2.11.8"

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.0.0" % "provided"
libraryDependencies += "com.github.scala-incubator.io" %% "scala-io-core" % "0.4.3"
libraryDependencies += "com.github.scala-incubator.io" %% "scala-io-file" % "0.4.3"
libraryDependencies += "org.apache.hadoop" %% "hadoop-streaming" % "2.7.0"

错误消息

SBT 'SparkPlayground' project refresh failed
    Error:Error:Error while importing SBT project:<br/>...<br/><pre>[info] Resolving org.scala-sbt#task-system;0.13.8 ...

[info] Resolving org.scala-sbt#tasks;0.13.8 ...

[info] Resolving org.scala-sbt#tracking;0.13.8 ...

[info] Resolving org.scala-sbt#cache;0.13.8 ...

[info] Resolving org.scala-sbt#testing;0.13.8 ...

[info] Resolving org.scala-sbt#test-agent;0.13.8 ...

[info] Resolving org.scala-sbt#test-interface;1.0 ...

[info] Resolving org.scala-sbt#main-settings;0.13.8 ...

[info] Resolving org.scala-sbt#apply-macro;0.13.8 ...

[info] Resolving org.scala-sbt#command;0.13.8 ...

[info] Resolving org.scala-sbt#logic;0.13.8 ...

[info] Resolving org.scala-sbt#precompiled-2_8_2;0.13.8 ...

[info] Resolving org.scala-sbt#precompiled-2_9_2;0.13.8 ...

[info] Resolving org.scala-sbt#precompiled-2_9_3;0.13.8 ...

[trace] Stack trace suppressed: run 'last *:update' for the full output.

[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.

[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-streaming_2.11;2.6.0: not found

[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-streaming_2.11;2.6.0: not found

[error] Total time: 13 s, completed Sep 5, 2016 2:05:47 AM

从错误消息来看,出于某种原因,sbt似乎在寻找hadoop-streaming\u2.11,但我不知道这2.11是从哪里来的。我对scala和sbt还很陌生,所以我猜我在什么地方打了个愚蠢的错

dojqjjoe

dojqjjoe1#

如果使用groupid%%artifactid%revision而不是groupid%artifactid%revision(区别是groupid后面的双精度%%),sbt会将项目的scala版本添加到项目名称中
来自sbt手册。
所以你应该在这里用%。

相关问题