我正在尝试使用aws sdk ruby for amazon elasticmapreduce服务运行一个step,该服务使用hadoop,虽然我可以创建集群和step,但当使用web界面手动设置时,该步骤总是失败
emr = Aws::EMR::Client.new
cluster_id = "*******"
resp = emr.add_job_flow_steps({
job_flow_id: cluster_id, # required
steps: [ # required
{
name: "TestStep", # required
action_on_failure: "CANCEL_AND_WAIT", # accepts TERMINATE_JOB_FLOW, TERMINATE_CLUSTER, CANCEL_AND_WAIT, CONTINUE
hadoop_jar_step: { # required
jar: 'command-runner.jar',
args:[
"-files",
"s3://source123/mapper.py,s3://source123/source_reducer.py",
"-mapper",
"mapper.py",
"-reducer",
"source_reducer.py",
"-input",
"s3://source123/input/",
"-output",
"s3://source123/output/"
]
},
},
],
})
我得到的错误是 Cannot run program "-files" (in directory "."): error=2, No such file or directory
有什么线索吗?
1条答案
按热度按时间pw9qyyiw1#
似乎添加hadoop流媒体的工作原理如下