hadoopmapreduce如何将字符串参数传递给Map器

3phpmpom  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(316)

我一直在ssh上使用hadoop mapreduce命令,如下所示:

hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.0.1.jar -file /nfs_home/appers/user1/mapper.py -file /nfs_home/appers/user1/reducer.py -mapper '/usr/lib/python_2.7.3/bin/python mapper.py' -reducer '/usr/lib/python_2.7.3/bin/python reducer.py' -input /ccexp/data/test_xml/0901282-510179094535002-oozie-oozi-W/extractOut//.xml -output /user/ccexptest/output/user1/MRoutput

但是,我正在扩展mapper.py脚本的功能,并希望在启动mapreduce作业时将字符串参数传递给mapper。如何编辑上面的mr命令,以便将字符串作为Map器的参数包含进来?

c86crjj0

c86crjj01#

可以将任意命令行输入传递到 -mapper 以及 -reducer ,为了你的案子

-mapper '/usr/lib/python_2.7.3/bin/python mapper.py myargs'

相关问题