我一直在ssh上使用hadoop mapreduce命令,如下所示:
hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.0.1.jar -file /nfs_home/appers/user1/mapper.py -file /nfs_home/appers/user1/reducer.py -mapper '/usr/lib/python_2.7.3/bin/python mapper.py' -reducer '/usr/lib/python_2.7.3/bin/python reducer.py' -input /ccexp/data/test_xml/0901282-510179094535002-oozie-oozi-W/extractOut//.xml -output /user/ccexptest/output/user1/MRoutput
但是,我正在扩展mapper.py脚本的功能,并希望在启动mapreduce作业时将字符串参数传递给mapper。如何编辑上面的mr命令,以便将字符串作为Map器的参数包含进来?
1条答案
按热度按时间c86crjj01#
可以将任意命令行输入传递到
-mapper
以及-reducer
,为了你的案子