无法解析org.apache的依赖关系hadoop:hadoop-aws:2.7.6

eh57zj3b  于 2021-05-27  发布在  Spark
关注(0)|答案(2)|浏览(386)

当我键入以下命令让我的java spark从amazons3读取csv文件时

spark-submit --class myApp --packages org.apache.hadoop:hadoop-aws:2.7.6 --master local ./target/spark_pratice.jar

错误消息

::::::::::::::::::::::::::::::::::::::::::::::

                ::              FAILED DOWNLOADS            ::

                :: ^ see resolution messages for details  ^ ::

                ::::::::::::::::::::::::::::::::::::::::::::::

                :: javax.activation#activation;1.1!activation.jar

                :: asm#asm;3.2!asm.jar

                :: org.apache.avro#avro;1.7.4!avro.jar

                :: com.thoughtworks.paranamer#paranamer;2.3!paranamer.jar

                :: org.xerial.snappy#snappy-java;1.0.4.1!snappy-java.jar(bundle)

                :: io.netty#netty;3.6.2.Final!netty.jar(bundle)

                ::::::::::::::::::::::::::::::::::::::::::::::
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [
    download failed: javax.activation#activation;1.1!activation.jar, 
    download failed: asm#asm;3.2!asm.jar, 
    download failed: org.apache.avro#avro;1.7.4!avro.jar, 
    download failed: com.thoughtworks.paranamer#paranamer;2.3!paranamer.jar, 
    download failed: org.xerial.snappy#snappy-java;1.0.4.1!snappy-java.jar(bundle), 
    download failed: io.netty#netty;3.6.2.Final!netty.jar(bundle)
]

我不知道如何解决这个问题。我需要更新什么吗?Maven?或者别的什么?

k3fezbri

k3fezbri1#

我也面临同样的问题&我已经删除了 .ivy2/cache , ivy2/jars 以及 .m2/repository/ 目录和执行 spark-shell --package 命令它开始工作。

7ivaypg9

7ivaypg92#

最后,我尝试从maven repository maven repository下载丢失的jar文件
我把它们放到了.m2/repository的正确存储库中

相关问题