org.apache.hadoop.mapred.lib.MultipleOutputs.<init>()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(7.3k)|赞(0)|评价(0)|浏览(147)

本文整理了Java中org.apache.hadoop.mapred.lib.MultipleOutputs.<init>()方法的一些代码示例，展示了MultipleOutputs.<init>()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。MultipleOutputs.<init>()方法的具体详情如下：
包路径：org.apache.hadoop.mapred.lib.MultipleOutputs
类名称：MultipleOutputs
方法名：<init>

MultipleOutputs.<init>介绍

[英]Creates and initializes multiple named outputs support, it should be instantiated in the Mapper/Reducer configure method.
[中]创建并初始化多个命名输出支持，它应该在Mapper/Reducer配置方法中实例化。

代码示例

代码示例来源：origin: org.apache.hadoop/hadoop-mapred-test

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: org.apache.hadoop/hadoop-mapred-test

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: uk.bl.wa.discovery/warc-hadoop-recordreaders

@Override
public void configure(JobConf job) {
  super.configure(job);
  mos = new MultipleOutputs(job);
}

代码示例来源：origin: lintool/Mr.LDA

public void configure(JobConf conf) {
 multipleOutputs = new MultipleOutputs(conf);
 numberOfLanguages = conf.getInt(Settings.PROPERTY_PREFIX + "model.languages", 0);
 index = new int[numberOfLanguages];
}

代码示例来源：origin: ukwa/webarchive-discovery

@Override
public void configure(JobConf job) {
  super.configure(job);
  mos = new MultipleOutputs(job);
}

代码示例来源：origin: ukwa/webarchive-discovery

@Override
public void configure(JobConf job) {
  super.configure(job);
  mos = new MultipleOutputs(job);
}

代码示例来源：origin: uk.bl.wa.discovery/warc-hadoop-recordreaders

@Override
public void configure(JobConf job) {
  super.configure(job);
  mos = new MultipleOutputs(job);
}

代码示例来源：origin: lintool/Mr.LDA

public void configure(JobConf conf) {
 multipleOutputs = new MultipleOutputs(conf);
 learning = conf.getBoolean(Settings.PROPERTY_PREFIX + "model.train", Settings.LEARNING_MODE);
 // System.out.println("======================================================================");
 // System.out.println("Available processors (cores): " +
 // Runtime.getRuntime().availableProcessors());
 // long maxMemory = Runtime.getRuntime().maxMemory();
 // System.out.println("Maximum memory (bytes): " + (maxMemory == Long.MAX_VALUE ? "no limit" :
 // maxMemory));
 // System.out.println("Free memory (bytes): " + Runtime.getRuntime().freeMemory());
 // System.out.println("Total memory (bytes): " + Runtime.getRuntime().totalMemory());
 // System.out.println("======================================================================");
}

代码示例来源：origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: org.apache.mahout/mahout-core

protected void setup() {
 int r = Integer.parseInt(jobConf.get(PROP_AROWBLOCK_SIZE));
 int k = Integer.parseInt(jobConf.get(PROP_K));
 int p = Integer.parseInt(jobConf.get(PROP_P));
 kp = k + p;
 yLookahead = Lists.newArrayListWithCapacity(kp);
 qSolver = new GivensThinSolver(r, kp);
 outputs = new MultipleOutputs(new JobConf(jobConf));
 closeables.addFirst(new Closeable() {
  @Override
  public void close() throws IOException {
   outputs.close();
  }
 });
}

代码示例来源：origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient

public void configure(JobConf conf) {
 mos = new MultipleOutputs(conf);
}

代码示例来源：origin: org.apache.mahout/mahout-mrlegacy

protected void setup() {
 int r = Integer.parseInt(jobConf.get(PROP_AROWBLOCK_SIZE));
 int k = Integer.parseInt(jobConf.get(PROP_K));
 int p = Integer.parseInt(jobConf.get(PROP_P));
 kp = k + p;
 yLookahead = Lists.newArrayListWithCapacity(kp);
 qSolver = new GivensThinSolver(r, kp);
 outputs = new MultipleOutputs(new JobConf(jobConf));
 closeables.addFirst(new Closeable() {
  @Override
  public void close() throws IOException {
   outputs.close();
  }
 });
}

代码示例来源：origin: org.apache.mahout/mahout-mr

protected void setup() {
 int r = Integer.parseInt(jobConf.get(PROP_AROWBLOCK_SIZE));
 int k = Integer.parseInt(jobConf.get(PROP_K));
 int p = Integer.parseInt(jobConf.get(PROP_P));
 kp = k + p;
 yLookahead = Lists.newArrayListWithCapacity(kp);
 qSolver = new GivensThinSolver(r, kp);
 outputs = new MultipleOutputs(new JobConf(jobConf));
 closeables.addFirst(new Closeable() {
  @Override
  public void close() throws IOException {
   outputs.close();
  }
 });
}

代码示例来源：origin: 01org/graphbuilder

@Override
public void configure(JobConf job) {
 super.configure(job);
 this.mos = new MultipleOutputs(job);
 this.hashRawVid = job.getBoolean("hashRawVid", true);
}

代码示例来源：origin: org.apache.mahout/mahout-core

@Override
protected void setup(Context context) throws IOException,
 InterruptedException {
 Configuration conf = context.getConfiguration();
 blockHeight = conf.getInt(PROP_OUTER_PROD_BLOCK_HEIGHT, -1);
 outputBBt = conf.getBoolean(PROP_OUPTUT_BBT_PRODUCTS, false);
 if (outputBBt) {
  int k = conf.getInt(QJob.PROP_K, -1);
  int p = conf.getInt(QJob.PROP_P, -1);
  Validate.isTrue(k > 0, "invalid k parameter");
  Validate.isTrue(p >= 0, "invalid p parameter");
  mBBt = new UpperTriangular(k + p);
 }
 String xiPathStr = conf.get(PROP_XI_PATH);
 if (xiPathStr != null) {
  xi = SSVDHelper.loadAndSumUpVectors(new Path(xiPathStr), conf);
  if (xi == null) {
   throw new IOException(String.format("unable to load mean path xi from %s.",
                     xiPathStr));
  }
 }
 if (outputBBt || xi != null) {
  outputs = new MultipleOutputs(new JobConf(conf));
  closeables.addFirst(new IOUtils.MultipleOutputsCloseableAdapter(outputs));
 }
}

代码示例来源：origin: org.apache.mahout/mahout-mr

@Override
protected void setup(Context context) throws IOException,
 InterruptedException {
 Configuration conf = context.getConfiguration();
 blockHeight = conf.getInt(PROP_OUTER_PROD_BLOCK_HEIGHT, -1);
 outputBBt = conf.getBoolean(PROP_OUPTUT_BBT_PRODUCTS, false);
 if (outputBBt) {
  int k = conf.getInt(QJob.PROP_K, -1);
  int p = conf.getInt(QJob.PROP_P, -1);
  Validate.isTrue(k > 0, "invalid k parameter");
  Validate.isTrue(p >= 0, "invalid p parameter");
  mBBt = new UpperTriangular(k + p);
 }
 String xiPathStr = conf.get(PROP_XI_PATH);
 if (xiPathStr != null) {
  xi = SSVDHelper.loadAndSumUpVectors(new Path(xiPathStr), conf);
  if (xi == null) {
   throw new IOException(String.format("unable to load mean path xi from %s.",
                     xiPathStr));
  }
 }
 if (outputBBt || xi != null) {
  outputs = new MultipleOutputs(new JobConf(conf));
  closeables.addFirst(new IOUtils.MultipleOutputsCloseableAdapter(outputs));
 }
}

代码示例来源：origin: org.apache.mahout/mahout-mrlegacy

@Override
protected void setup(Context context) throws IOException,
 InterruptedException {
 Configuration conf = context.getConfiguration();
 blockHeight = conf.getInt(PROP_OUTER_PROD_BLOCK_HEIGHT, -1);
 outputBBt = conf.getBoolean(PROP_OUPTUT_BBT_PRODUCTS, false);
 if (outputBBt) {
  int k = conf.getInt(QJob.PROP_K, -1);
  int p = conf.getInt(QJob.PROP_P, -1);
  Validate.isTrue(k > 0, "invalid k parameter");
  Validate.isTrue(p >= 0, "invalid p parameter");
  mBBt = new UpperTriangular(k + p);
 }
 String xiPathStr = conf.get(PROP_XI_PATH);
 if (xiPathStr != null) {
  xi = SSVDHelper.loadAndSumUpVectors(new Path(xiPathStr), conf);
  if (xi == null) {
   throw new IOException(String.format("unable to load mean path xi from %s.",
                     xiPathStr));
  }
 }
 if (outputBBt || xi != null) {
  outputs = new MultipleOutputs(new JobConf(conf));
  closeables.addFirst(new IOUtils.MultipleOutputsCloseableAdapter(outputs));
 }
}

代码示例来源：origin: lintool/Mr.LDA

public void configure(JobConf conf) {
 multipleOutputs = new MultipleOutputs(conf);
 numberOfLanguages = conf.getInt(Settings.PROPERTY_PREFIX + "model.languages", 0);
 for (int languageIndex = 0; languageIndex < numberOfLanguages; languageIndex++) {
  docContent.add(new HMapSIW());
 }
}

内容来源于网络，如有侵权，请联系作者删除！

MultipleOutputs

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com