java.lang.illegalargumentexception:源文件列表和源路径都存在

7nbnzgx9  于 2021-05-29  发布在  Hadoop
关注(0)|答案(2)|浏览(700)

我试图复制文件从hdfs到s3使用 distcp 通过执行以下命令

hadoop distcp -fs.s3a.access.key=AccessKey -fs.s3a.secret.key=SecrerKey \
s3n://testbdr/test2 hdfs://hostname:portnumber/tmp/test

但我得到以下错误:

17/09/05 02:59:30 ERROR tools.DistCp: Invalid arguments:
java.lang.IllegalArgumentException: Both source file listing and source paths present
        at org.apache.hadoop.tools.OptionsParser.parseSourceAndTargetPaths(OptionsParser.java:341)
        at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:89)
        at org.apache.hadoop.tools.DistCp.run(DistCp.java:112)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.tools.DistCp.main(DistCp.java:436)
Invalid arguments: Both source file listing and source paths present
usage: distcp OPTIONS [source_path...] <target_path>
              OPTIONS
 -append                       Reuse existing data in target files and
                               append new data to them if possible
 -async                        Should distcp execution be blocking
fquxozlt

fquxozlt1#

要传递配置参数,必须使用 -D ```
hadoop distcp -Dfs.s3a.access.key=AccessKey -Dfs.s3a.secret.key=SecrerKey
s3n://testbdr/test2 hdfs://hostname:portnumber/tmp/test

lrl1mhuk

lrl1mhuk2#

旧命令

hadoop distcp-dfs.s3a.access.key=accesskey-dfs.s3a.secret.key=secretkey\s3n://testbdr/test2hdfs://hostname:端口号/tmp/测试

修正指令

hadoop distcp-dfs.s3n.awsaccesskeyid=accesskey-dfs.s3n.awssecretaccesskey=secretkey\s3n://testbdr/test2hdfs://hostname:端口号/tmp/测试

相关问题