我正在Windows7上使用cygwin尝试在Hadoop1.2.1中设置一个节点。我遵循这个教程。我能够很好地创建输入目录,并将.xml文件复制到输入目录。问题似乎是当我执行 $ bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs [a-z.]+'
它在命令行中抛出“error:could not find or load main class work\work”。我已经检查了源代码(如下所列,看起来像python),并且显示了一个main方法。我还尝试了原始命令行调用的变体,例如, $ bin/hadoop jar hadoop-examples-*.jar main input output 'dfs [a-z.]+'
等等。
我的问题是:为什么hadoop没有阅读这个主要方法?我怎样才能理解这个主要方法呢?当cygwin说“work/work”时,它告诉我什么?源代码是用python编写并编译成.jar格式的这一事实有什么意义吗?
from org.apache.hadoop.fs import Path
from org.apache.hadoop.io import *
from org.apache.hadoop.mapred import *
import sys
import getopt
class WordCountMap(Mapper, MapReduceBase):
one = IntWritable(1)
def map(self, key, value, output, reporter):
for w in value.toString().split():
output.collect(Text(w), self.one)
class Summer(Reducer, MapReduceBase):
def reduce(self, key, values, output, reporter):
sum = 0
while values.hasNext():
sum += values.next().get()
output.collect(key, IntWritable(sum))
def printUsage(code):
print "wordcount [-m <maps>] [-r <reduces>] <input> <output>"
sys.exit(code)
def main(args):
conf = JobConf(WordCountMap);
conf.setJobName("wordcount");
conf.setOutputKeyClass(Text);
conf.setOutputValueClass(IntWritable);
conf.setMapperClass(WordCountMap);
conf.setCombinerClass(Summer);
conf.setReducerClass(Summer);
try:
flags, other_args = getopt.getopt(args[1:], "m:r:")
except getopt.GetoptError:
printUsage(1)
if len(other_args) != 2:
printUsage(1)
for f,v in flags:
if f == "-m":
conf.setNumMapTasks(int(v))
elif f == "-r":
conf.setNumReduceTasks(int(v))
conf.setInputPath(Path(other_args[0]))
conf.setOutputPath(Path(other_args[1]))
JobClient.runJob(conf);
if __name__ == "__main__":
main(sys.argv)
暂无答案!
目前还没有任何答案,快来回答吧!