org.apache.hadoop.mapreduce.Mapper.cleanup()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(5.3k)|赞(0)|评价(0)|浏览(155)

本文整理了Java中org.apache.hadoop.mapreduce.Mapper.cleanup()方法的一些代码示例,展示了Mapper.cleanup()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Mapper.cleanup()方法的具体详情如下:
包路径:org.apache.hadoop.mapreduce.Mapper
类名称:Mapper
方法名:cleanup

Mapper.cleanup介绍

[英]Called once at the end of the task.
[中]任务结束时调用一次。

代码示例

代码示例来源:origin: apache/hbase

  1. @Override
  2. protected void
  3. cleanup(Mapper<WALKey, WALEdit, ImmutableBytesWritable, Mutation>.Context context)
  4. throws IOException, InterruptedException {
  5. super.cleanup(context);
  6. }

代码示例来源:origin: thinkaurelius/titan

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. super.cleanup(context);
  4. job.workerIterationEnd(metrics);
  5. }

代码示例来源:origin: apache/ignite

  1. /** {@inheritDoc} */
  2. @Override protected void cleanup(Context ctx) throws IOException, InterruptedException {
  3. super.cleanup(ctx);
  4. HadoopErrorSimulator.instance().onMapCleanup();
  5. }

代码示例来源:origin: apache/phoenix

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. super.cleanup(context);
  4. if (connection != null) {
  5. try {
  6. connection.close();
  7. } catch (SQLException e) {
  8. LOG.error("Error {} while closing connection in the PhoenixIndexMapper class ",
  9. e.getMessage());
  10. }
  11. }
  12. }
  13. }

代码示例来源:origin: apache/phoenix

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. super.cleanup(context);
  4. if (connection != null) {
  5. try {
  6. processBatch(context);
  7. connection.close();
  8. if (outputConn != null) {
  9. outputConn.close();
  10. }
  11. } catch (SQLException e) {
  12. LOG.error("Error while closing connection in the PhoenixIndexMapper class ", e);
  13. throw new IOException(e);
  14. }
  15. }
  16. }

代码示例来源:origin: apache/phoenix

  1. super.cleanup(context);
  2. } catch (SQLException e) {
  3. LOG.error(" Error {} while read/write of a record ", e.getMessage());

代码示例来源:origin: com.aliyun.phoenix/ali-phoenix-core

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. super.cleanup(context);
  4. if (outputStream != null) {
  5. outputStream.close();
  6. }
  7. }
  8. }

代码示例来源:origin: org.apache.mahout/mahout-utils

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. for (Map.Entry<Integer, WeightedVectorWritable> entry : mostDistantPoints.entrySet()) {
  4. context.write(new IntWritable(entry.getKey()), entry.getValue());
  5. }
  6. super.cleanup(context);
  7. }

代码示例来源:origin: aseldawy/spatialhadoop2

  1. @Override
  2. protected void cleanup(
  3. Mapper<Rectangle, Iterable<S>, IntWritable, Triangulation>.Context context)
  4. throws IOException, InterruptedException {
  5. super.cleanup(context);
  6. writer.close(context);
  7. }
  8. }

代码示例来源:origin: ukwa/webarchive-discovery

  1. @Override
  2. protected void cleanup(
  3. Mapper<Path, BytesWritable, Text, Text>.Context context)
  4. throws IOException, InterruptedException {
  5. log.debug("Cleaning up and emitting final result...");
  6. super.cleanup(context);
  7. this.emit(context);
  8. }

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. protected void cleanup(Context context)
  2. throws IOException, InterruptedException {
  3. mapCleanup = true;
  4. super.cleanup(context);
  5. }
  6. }

代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

  1. @SuppressWarnings({ "rawtypes", "unchecked" })
  2. @Override
  3. protected void cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)
  4. throws IOException, InterruptedException {
  5. mapCleanup = true;
  6. super.cleanup(context);
  7. }

代码示例来源:origin: twitter/hraven

  1. @Override
  2. protected void cleanup(
  3. Mapper<JobFile, FileStatus, ImmutableBytesWritable, Put>.Context context)
  4. throws IOException, InterruptedException {
  5. if (hbaseConnection != null) {
  6. hbaseConnection.close();
  7. }
  8. super.cleanup(context);
  9. }

代码示例来源:origin: locationtech/geowave

  1. @Override
  2. protected void cleanup(
  3. final Mapper<GeoWaveInputKey, SimpleFeature, AvroKey<AvroSimpleFeatureCollection>, NullWritable>.Context context)
  4. throws IOException, InterruptedException {
  5. super.cleanup(context);
  6. writeRemainingAvroBatches(context);
  7. }

代码示例来源:origin: locationtech/geowave

  1. @Override
  2. protected void cleanup(final Context context) throws IOException, InterruptedException {
  3. osmProvider.close();
  4. super.cleanup(context);
  5. }

代码示例来源:origin: org.apache.hadoop/hadoop-distcp

  1. @Override
  2. protected void cleanup(Context context)
  3. throws IOException, InterruptedException {
  4. super.cleanup(context);
  5. long secs = (System.currentTimeMillis() - startEpoch) / 1000;
  6. incrementCounter(context, Counter.BANDWIDTH_IN_BYTES,
  7. totalBytesCopied / ((secs == 0 ? 1 : secs)));
  8. }
  9. }

代码示例来源:origin: org.apache.mahout/mahout-core

  1. @Override
  2. protected void cleanup(Context ctx) throws IOException, InterruptedException {
  3. if (weightsPerFeature != null) {
  4. ctx.write(new Text(TrainNaiveBayesJob.WEIGHTS_PER_FEATURE), new VectorWritable(weightsPerFeature));
  5. ctx.write(new Text(TrainNaiveBayesJob.WEIGHTS_PER_LABEL), new VectorWritable(weightsPerLabel));
  6. }
  7. super.cleanup(ctx);
  8. }
  9. }

代码示例来源:origin: org.apache.mahout/mahout-core

  1. @Override
  2. protected void cleanup(Context ctx) throws IOException, InterruptedException {
  3. ctx.write(new Text(TrainNaiveBayesJob.LABEL_THETA_NORMALIZER),
  4. new VectorWritable(trainer.retrievePerLabelThetaNormalizer()));
  5. super.cleanup(ctx);
  6. }
  7. }

代码示例来源:origin: MKLab-ITI/multimedia-indexing

  1. @Override
  2. protected void cleanup(Context context) throws IOException, InterruptedException {
  3. super.cleanup(context);
  4. downloader.shutDown();
  5. vectorizer.shutDown();
  6. }

代码示例来源:origin: com.moz.fiji.examples.phonebook/fiji-phonebook

  1. /** {@inheritDoc} */
  2. @Override
  3. protected void cleanup(Context hadoopContext) throws IOException, InterruptedException {
  4. ResourceUtils.closeOrLog(mTableWriter);
  5. ResourceUtils.releaseOrLog(mTable);
  6. ResourceUtils.releaseOrLog(mFiji);
  7. super.cleanup(hadoopContext);
  8. }
  9. }

相关文章