hbase测试实用程序的sbt依赖关系管理问题

我正在尝试使用scalatest执行单元测试，使用hbase测试实用程序对开发代码进行本地测试。在sbt中设置hbase测试实用程序是目前的难题。编译时，出现以下错误：

[warn]  module not found: org.apache.hbase#${compat.module};1.2.1
[warn] ==== local: tried
[warn]   /root/.ivy2/local/org.apache.hbase/${compat.module}/1.2.1/ivys/ivy.xml
[warn] ==== public: tried
[warn]   https://repo1.maven.org/maven2/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== Akka Repository: tried
[warn]   http://repo.akka.io/releases/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== scala-tools: tried
[warn]   https://oss.sonatype.org/content/groups/scala-tools/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== cloudera-repos: tried
[warn]   https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/apache/hbase/${compat.module}/1.2.1/${compat.module}-1.2.1.pom
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.apache.hbase#${compat.module};1.2.1: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]      org.apache.hbase:${compat.module}:1.2.1
[warn]        +- org.apache.hbase:hbase-testing-util:1.2.1 (/workspace/spark/etl/built.sbt#L30-62)

[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.hbase#${compat.module};1.2.1: not found
[error] Total time: 32 s, completed Apr 29, 2016 9:25:27 AM

我的build.sbt文件如下：

val hbaseVersion = "1.2.1"
val sparkVersion = "1.6.1"
val hadoopVersion = "2.7.1"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion,
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-mllib" % sparkVersion ,
  "org.apache.hbase" % "hbase" % hbaseVersion,
  "org.apache.hbase" % "hbase-server" % hbaseVersion,
  "org.apache.hbase" % "hbase-server" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-client" % hbaseVersion,
  "org.apache.hbase" % "hbase-common" % hbaseVersion,
  "org.apache.hbase" % "hbase-common" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-annotations" % hbaseVersion,
  "org.apache.hbase" % "hbase-testing-util" % hbaseVersion % "test",
  "org.apache.hadoop" % "hadoop-minicluster" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-mapreduce-client-jobclient" % hadoopVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion classifier "tests",
  "org.apache.hbase" % "hbase-hadoop-compat" % hbaseVersion,
  "org.apache.hbase" % "hbase-hadoop-compat" % hbaseVersion classifier "tests",
  "org.apache.hbase" % "hbase-hadoop2-compat" % hbaseVersion,
  "org.apache.hbase" % "hbase-hadoop2-compat" % hbaseVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-common" % hadoopVersion,
  "org.apache.hadoop" % "hadoop-common" % hadoopVersion classifier "tests",
  "org.apache.hadoop" % "hadoop-annotations" % hadoopVersion,
  "org.scalatest" %% "scalatest" % "2.2.6" % "test" ,
  //"org.scalacheck" %% "scalacheck" % "1.12.5" % "test",
  "com.cloudera.sparkts" % "sparkts" % "0.3.0",
  "com.ecwid.consul" % "consul-api" % "1.1.9",
  "joda-time" % "joda-time" % "2.7"
)

resolvers ++= Seq(
  "Akka Repository" at "http://repo.akka.io/releases/",
  "scala-tools" at "https://oss.sonatype.org/content/groups/scala-tools",
  "cloudera-repos" at "https://repository.cloudera.com/artifactory/cloudera-repos/",
  "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
)

有人知道为什么会发生这种失败吗？

很抱歉延迟了回复。我无法让它按原样工作，所以我更改了如下版本：

val sparkVersion = "1.6.1"
val hbaseVersion = "1.2.0-cdh5.7.0"
val hadoopVersion = "2.6.0-cdh5.7.0"

这导致了更多的头痛。由于引用了较旧的库方法，我不得不将guava的版本更改为较早的版本，因此这是必需的：

"com.google.guava" % "guava" % "14.0" force()

（我认为16.0版本就可以了）
不得不评论如下：

// "com.cloudera" % "spark-hbase" % "0.0.2-clabs",

（原来q里没有）
最后，看起来原来的问题是一个需要解决的bug，请看这里，感谢david portabella的参考：
https://issues.apache.org/jira/browse/hbase-15925
并用1.2.2版本修复

hbase测试实用程序的sbt依赖关系管理问题

1条答案

相关问题

热门标签

最新问答