错误解析参数，amazon aws emr

hsgswve4 于 2021-06-02 发布在 Hadoop

关注(0)|答案(1)|浏览(653)

我正在尝试创建一个step by linux控制台：

aws emr add-steps --cluster-id j-XXXXXXXXXX --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ 
Args=["--s3Endpoint,s3-eu-west-1.amazonaws.com","--src,s3://folder-name/logs/j-XXXXXXXXXX/node/","--dest,hdfs:///output","--srcPattern,.*[a-zA-Z,]+"]

我跳过以下错误
分析参数“-steps”时出错：应为“，”，收到的输入为“+”
我怎么能修好它？
我正在寻找一个解决方案，上传多个文件到s3和s3distcp的Hive收集亚马逊电子病历。还有别的办法吗？
我还有另一个问题：现在我正在创建一个ssh隧道来连接到hive，我如何连接php？
目前，我已经解决了删除“src模式”的错误，但给我另一个错误，我包括下面的图像
图像错误
这就是出现的错误

INFO Synchronously wait child process to complete : hadoop jar /var/lib/aws/emr/step-runner/hadoop- 
INFO waitProcessCompletion ended with exit code 1 : hadoop jar
/var/lib/aws/emr/step-runner/hadoop-
INFO total process run time: 2 seconds
2016-07-12T14:26:48.744Z INFO Step created jobs:
2016-07-12T14:26:48.744Z WARN Step failed with exitCode 1 and took 2 seconds

谢谢！！！

hadoop Hive amazon-emr amazon-s3 s3distcp

来源：https://stackoverflow.com/questions/38325770/error-parsing-parameter-amazon-aws-emr

1条答案

按热度按时间

u2nhd7ah1#

尝试json配置

[
    {
        "Name":"S3DistCp step",
        "Args":["s3-dist-cp","--s3Endpoint=s3.amazonaws.com","--src=s3://mybucket/logs/j-3GYXXXXXX9IOJ/node/","--dest=hdfs:///output","--srcPattern=.*[a-zA-Z,]+"],
        "ActionOnFailure":"CONTINUE",
        "Type":"CUSTOM_JAR",
        "Jar":"command-runner.jar"        
    }
]
``` `aws emr add-steps --cluster-id j-3GYXXXXXX9IOK --steps file://./myStep.json` http://docs.aws.amazon.com/emr/latest/releaseguide/usingemr_s3distcp.html#usingemr_s3distcp.step

赞(0）回复(0）举报 2021-06-02

我来回答

错误解析参数，amazon aws emr

1条答案

相关问题

热门标签

最新问答