org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.close()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(5.8k)|赞(0)|评价(0)|浏览(144)

本文整理了Java中org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.close()方法的一些代码示例,展示了MultipleOutputs.close()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。MultipleOutputs.close()方法的具体详情如下:
包路径:org.apache.hadoop.mapreduce.lib.output.MultipleOutputs
类名称:MultipleOutputs
方法名:close

MultipleOutputs.close介绍

[英]Closes all the opened outputs. This should be called from cleanup method of map/reduce task. If overridden subclasses must invoke super.close() at the end of their close()
[中]关闭所有打开的输出。这应该从map/reduce任务的清理方法中调用。如果被重写,子类必须在其close()末尾调用super.close()

代码示例

代码示例来源:origin: apache/kylin

  1. private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
  2. // output written to baseDir/colName/colName.rldict-r-00000 (etc)
  3. String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
  4. try (ByteArrayOutputStream baos = new ByteArrayOutputStream(); DataOutputStream outputStream = new DataOutputStream(baos);) {
  5. outputStream.writeUTF(dict.getClass().getName());
  6. dict.write(outputStream);
  7. mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(), new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
  8. }
  9. mos.close();
  10. }
  11. }

代码示例来源:origin: apache/kylin

  1. @Override
  2. public void doCleanup(Context context) throws IOException, InterruptedException {
  3. mos.close();
  4. Path outputDirBase = new Path(context.getConfiguration().get(FileOutputFormat.OUTDIR), PathNameCuboidBase);
  5. FileSystem fs = FileSystem.get(context.getConfiguration());
  6. if (!fs.exists(outputDirBase)) {
  7. fs.mkdirs(outputDirBase);
  8. SequenceFile
  9. .createWriter(context.getConfiguration(),
  10. SequenceFile.Writer.file(new Path(outputDirBase, "part-m-00000")),
  11. SequenceFile.Writer.keyClass(Text.class), SequenceFile.Writer.valueClass(Text.class))
  12. .close();
  13. }
  14. }

代码示例来源:origin: apache/kylin

  1. @Override
  2. public void doCleanup(Context context) throws IOException, InterruptedException {
  3. mos.close();
  4. Path outputDirBase = new Path(context.getConfiguration().get(FileOutputFormat.OUTDIR), PathNameCuboidBase);
  5. FileSystem fs = FileSystem.get(context.getConfiguration());
  6. if (!fs.exists(outputDirBase)) {
  7. fs.mkdirs(outputDirBase);
  8. SequenceFile
  9. .createWriter(context.getConfiguration(),
  10. SequenceFile.Writer.file(new Path(outputDirBase, "part-m-00000")),
  11. SequenceFile.Writer.keyClass(Text.class), SequenceFile.Writer.valueClass(Text.class))
  12. .close();
  13. }
  14. }

代码示例来源:origin: apache/kylin

  1. @Override
  2. protected void doCleanup(Context context) throws IOException, InterruptedException {
  3. if (isStatistics) {
  4. //output the hll info;
  5. List<Long> allCuboids = Lists.newArrayList();
  6. allCuboids.addAll(cuboidHLLMap.keySet());
  7. Collections.sort(allCuboids);
  8. logMapperAndCuboidStatistics(allCuboids); // for human check
  9. outputStatistics(allCuboids);
  10. } else {
  11. //dimension col
  12. if (cubeDesc.listDimensionColumnsExcludingDerived(true).contains(col)) {
  13. outputDimRangeInfo();
  14. }
  15. // dic col
  16. if (buildDictInReducer) {
  17. Dictionary<String> dict = builder.build();
  18. outputDict(col, dict);
  19. }
  20. }
  21. mos.close();
  22. }

代码示例来源:origin: pl.edu.icm.coansys/coansys-io-input

  1. @Override
  2. public void cleanup(Context context) throws IOException, InterruptedException {
  3. mos.close();
  4. }
  5. }

代码示例来源:origin: jasonTangxd/recommendSys

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. // multipleOutputs=null;//本地会失败
  4. multipleOutputs.close();
  5. }
  6. }

代码示例来源:origin: linkedin/dynamometer

  1. @Override
  2. public void cleanup(Context context) throws IOException, InterruptedException {
  3. multiOutputs.close();
  4. multiOutputs = null;
  5. }

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. public void cleanup(Context context) throws IOException,
  3. InterruptedException {
  4. if (debugOut != null) {
  5. debugOut.close();
  6. }
  7. }

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. public void cleanup(Context context) throws IOException,
  3. InterruptedException {
  4. if (debugOut != null) {
  5. debugOut.close();
  6. }
  7. }
  8. protected void process(Context context, Fact fact, Derivation d,

代码示例来源:origin: geftimov/hadoop-map-reduce-patterns

  1. protected void cleanup(Context context) throws IOException,
  2. InterruptedException {
  3. // Close multiple outputs!
  4. mos.close();
  5. }
  6. }

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. protected void cleanup(final Context context) throws IOException, InterruptedException {
  3. if (mos != null) {
  4. mos.close();
  5. }
  6. }

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. public void cleanup(Context context) throws IOException,
  3. InterruptedException {
  4. if (mout != null) {
  5. mout.close();
  6. }
  7. log.info("Most input triples stored at one time by any reasoner: "
  8. + maxStored + " (reasoner for node: " + maxNode + ")");
  9. }
  10. @Override

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. public void cleanup(Context context) throws IOException,
  3. InterruptedException {
  4. mout.close();
  5. log.info("Input records processed: " + totalInput);
  6. log.info("Distinct facts: " + totalFacts);
  7. log.info("Output facts: " + totalOutput);
  8. }
  9. @Override

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. public void cleanup(Context context)
  2. throws IOException, InterruptedException {
  3. mos.close();
  4. }
  5. }

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. public void cleanup(Context context)
  2. throws IOException, InterruptedException {
  3. mos.close();
  4. }
  5. }

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. public void cleanup(Context context)
  2. throws IOException, InterruptedException {
  3. mos.close();
  4. }
  5. }

代码示例来源:origin: openimaj/openimaj

  1. @Override
  2. protected void cleanup(Context context) throws IOException,
  3. InterruptedException
  4. {
  5. super.cleanup(context);
  6. mos.close();
  7. }
  8. }

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. public void cleanup(Context context)
  2. throws IOException, InterruptedException {
  3. mos.close();
  4. }
  5. }

代码示例来源:origin: apache/incubator-rya

  1. @Override
  2. protected void cleanup(Context context) throws IOException,
  3. InterruptedException {
  4. if (debugOut != null) {
  5. debugOut.close();
  6. }
  7. // Perform schema-level reasoning
  8. schema.closure();
  9. // Output the complete schema
  10. context.write(NullWritable.get(), schema);
  11. }
  12. }

代码示例来源:origin: pl.edu.icm.coansys/coansys-io-input

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. writeDecisions(NullWritable.get(), makeDecisionsDuringCleanup());
  4. mos.close();
  5. super.cleanup(context);
  6. }
  7. }

相关文章