org.apache.tez.dag.api.Vertex.setExecutionContext()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(3.4k)|赞(0)|评价(0)|浏览(81)

本文整理了Java中org.apache.tez.dag.api.Vertex.setExecutionContext()方法的一些代码示例,展示了Vertex.setExecutionContext()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Vertex.setExecutionContext()方法的具体详情如下:
包路径:org.apache.tez.dag.api.Vertex
类名称:Vertex
方法名:setExecutionContext

Vertex.setExecutionContext介绍

[英]Sets the execution context for this Vertex - i.e. the Task Scheduler, ContainerLauncher and TaskCommunicator to be used. Also whether the vertex will be executed within the AM. If partially specified, the default components in Tez will be used - which may or may not work with the custom context.
[中]设置此顶点的执行上下文,即要使用的任务调度器、ContainerLauncher和TaskCommunicator。以及顶点是否将在AM内执行。如果部分指定,将使用Tez中的默认组件,这可能适用于自定义上下文,也可能不适用于自定义上下文。

代码示例

代码示例来源:origin: apache/drill

private Vertex createVertex(JobConf conf, ReduceWork reduceWork,
  LocalResource appJarLr, List<LocalResource> additionalLr, FileSystem fs,
  Path mrScratchDir, Context ctx) throws Exception {
 // set up operator plan
 conf.set(Utilities.INPUT_NAME, reduceWork.getName());
 Utilities.setReduceWork(conf, reduceWork, mrScratchDir, false);
 // create the directories FileSinkOperators need
 Utilities.createTmpDirs(conf, reduceWork);
 VertexExecutionContext vertexExecutionContext = createVertexExecutionContext(reduceWork);
 // create the vertex
 Vertex reducer = Vertex.create(reduceWork.getName(),
   ProcessorDescriptor.create(ReduceTezProcessor.class.getName()).
     setUserPayload(TezUtils.createUserPayloadFromConf(conf)),
   reduceWork.isAutoReduceParallelism() ?
     reduceWork.getMaxReduceTasks() :
     reduceWork.getNumReduceTasks(), getContainerResource(conf));
 reducer.setTaskEnvironment(getContainerEnvironment(conf, false));
 reducer.setExecutionContext(vertexExecutionContext);
 reducer.setTaskLaunchCmdOpts(getContainerJavaOpts(conf));
 Map<String, LocalResource> localResources = new HashMap<String, LocalResource>();
 localResources.put(getBaseName(appJarLr), appJarLr);
 for (LocalResource lr: additionalLr) {
  localResources.put(getBaseName(lr), lr);
 }
 reducer.addTaskLocalFiles(localResources);
 return reducer;
}

代码示例来源:origin: apache/hive

private Vertex createVertex(JobConf conf, ReduceWork reduceWork, FileSystem fs,
  Path mrScratchDir, Context ctx, Map<String, LocalResource> localResources)
    throws Exception {
 // set up operator plan
 conf.set(Utilities.INPUT_NAME, reduceWork.getName());
 Utilities.setReduceWork(conf, reduceWork, mrScratchDir, false);
 // create the directories FileSinkOperators need
 Utilities.createTmpDirs(conf, reduceWork);
 VertexExecutionContext vertexExecutionContext = createVertexExecutionContext(reduceWork);
 // create the vertex
 Vertex reducer = Vertex.create(reduceWork.getName(),
   ProcessorDescriptor.create(ReduceTezProcessor.class.getName()).
     setUserPayload(TezUtils.createUserPayloadFromConf(conf)),
   reduceWork.isAutoReduceParallelism() ?
     reduceWork.getMaxReduceTasks() :
     reduceWork.getNumReduceTasks(), getContainerResource(conf));
 reducer.setTaskEnvironment(getContainerEnvironment(conf, false));
 reducer.setExecutionContext(vertexExecutionContext);
 reducer.setTaskLaunchCmdOpts(getContainerJavaOpts(conf));
 reducer.addTaskLocalFiles(localResources);
 return reducer;
}

代码示例来源:origin: apache/drill

map.setExecutionContext(executionContext);
map.setTaskLaunchCmdOpts(getContainerJavaOpts(conf));

代码示例来源:origin: apache/hive

map.setExecutionContext(executionContext);
map.setTaskLaunchCmdOpts(getContainerJavaOpts(conf));

代码示例来源:origin: org.apache.tez/tez-examples

private void setVertexExecutionContext(Vertex vertex, VertexExecutionContext executionContext) {
 if (executionContext != null) {
  vertex.setExecutionContext(executionContext);
 }
}

相关文章