org.apache.tez.dag.api.Vertex.setConf()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(2.0k)|赞(0)|评价(0)|浏览(89)

本文整理了Java中org.apache.tez.dag.api.Vertex.setConf()方法的一些代码示例,展示了Vertex.setConf()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Vertex.setConf()方法的具体详情如下:
包路径:org.apache.tez.dag.api.Vertex
类名称:Vertex
方法名:setConf

Vertex.setConf介绍

[英]This is currently used to setup additional configuration parameters which will be available in the Vertex specific configuration used in the AppMaster. This API would be used for properties which are used by the Tez framework while executing this vertex as part of a larger DAG. As an example, the number of attempts for a task.

A vertex inherits it's Configuration from the DAG, and can override properties for this Vertex only using this method

Currently, properties which are used by the task runtime, such as the task to AM heartbeat interval, cannot be changed using this method.

Note: This API does not add any configuration to runtime components such as InputInitializers, OutputCommitters, Inputs and Outputs.
[中]这目前用于设置附加配置参数,这些参数将在AppMaster中使用的特定于顶点的配置中可用。该API将用于Tez框架在作为更大DAG的一部分执行该顶点时使用的属性。例如,任务的尝试次数。
顶点从DAG继承其配置,并且只能使用此方法覆盖该顶点的属性
当前,无法使用此方法更改任务运行时使用的属性,例如任务到AM心跳间隔。
注意:此API不会向运行时组件(如InputInitializers、OutputCommitter、Inputs和Outputs)添加任何配置。

代码示例

代码示例来源:origin: apache/hive

wx.setConf(TEZ_MEMORY_RESERVE_FRACTION, Double.toString(frac));

代码示例来源:origin: apache/drill

wx.setConf(TEZ_MEMORY_RESERVE_FRACTION, Double.toString(frac));

代码示例来源:origin: org.apache.pig/pig

/**
 * Set config with Scope.Vertex in TezConfiguration on the vertex
 *
 * @param vertex Vertex on which config is to be set
 * @param isMapVertex Whether map or reduce vertex. i.e root or intermediate/leaf vertex
 * @param conf Config that contains the tez or equivalent mapreduce settings.
 */
public static void setVertexConfig(Vertex vertex, boolean isMapVertex,
    Configuration conf) {
  Map<String, String> configMapping = isMapVertex ? mrMapParamToTezVertexParamMap
      : mrReduceParamToTezVertexParamMap;
  for (Entry<String, String> dep : configMapping.entrySet()) {
    String value = conf.get(dep.getValue(), conf.get(dep.getKey()));
    if (value != null) {
      vertex.setConf(dep.getValue(), value);
      LOG.debug("Setting " + dep.getValue() + " to " + value
          + " for the vertex " + vertex.getName());
    }
  }
}

相关文章