在inputformat中使用combinefileinputformatinput时,我得到了错误示例化异常。
Configuration conf = new Configuration();
conf.set("fs.default.name", "hdfs://master:54310");
Job job = new Job(conf, "WordCount");
job.setJarByClass(WordCount.class);
job.setMapperClass(WordCountMapper.class);
job.setReducerClass(WordCountReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setInputFormatClass(CombineFileInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
FileInputFormat.addInputPath(job, new Path("/user/hduser/CombineFileInputFormatInput"));
FileOutputFormat.setOutputPath(job, new Path("/user/hduser/CombineFileInputFormatOutputNew2"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
1条答案
按热度按时间kyvafyod1#
CombineFileInputFormat
是抽象类,必须对其进行扩展并实现createrecordreader方法:然后更换
job.setInputFormatClass(..)
使用扩展类的名称进行调用。