org.apache.tez.dag.api.Vertex.getProcessorDescriptor()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(4.1k)|赞(0)|评价(0)|浏览(108)

本文整理了Java中org.apache.tez.dag.api.Vertex.getProcessorDescriptor()方法的一些代码示例,展示了Vertex.getProcessorDescriptor()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Vertex.getProcessorDescriptor()方法的具体详情如下:
包路径:org.apache.tez.dag.api.Vertex
类名称:Vertex
方法名:getProcessorDescriptor

Vertex.getProcessorDescriptor介绍

[英]Get the vertex task processor descriptor
[中]

代码示例

代码示例来源:origin: apache/hive

public TaskSpec constructTaskSpec(DAG dag, String vertexName, int numSplits, ApplicationId appId, int index) {
 Vertex vertex = dag.getVertex(vertexName);
 ProcessorDescriptor processorDescriptor = vertex.getProcessorDescriptor();
 List<RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor>> inputs =
   vertex.getInputs();
 List<RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor>> outputs =
   vertex.getOutputs();
 Preconditions.checkState(inputs.size() == 1);
 Preconditions.checkState(outputs.size() == 1);
 List<InputSpec> inputSpecs = new ArrayList<>();
 for (RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor> input : inputs) {
  InputSpec inputSpec = new InputSpec(input.getName(), input.getIODescriptor(), 1);
  inputSpecs.add(inputSpec);
 }
 List<OutputSpec> outputSpecs = new ArrayList<>();
 for (RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor> output : outputs) {
  OutputSpec outputSpec = new OutputSpec(output.getName(), output.getIODescriptor(), 1);
  outputSpecs.add(outputSpec);
 }
 TezDAGID dagId = TezDAGID.getInstance(appId, 0);
 TezVertexID vertexId = TezVertexID.getInstance(dagId, 0);
 TezTaskID taskId = TezTaskID.getInstance(vertexId, index);
 TezTaskAttemptID taskAttemptId = TezTaskAttemptID.getInstance(taskId, 0);
 return new TaskSpec(taskAttemptId, dag.getName(), vertexName, numSplits, processorDescriptor, inputSpecs, outputSpecs, null);
}

代码示例来源:origin: apache/drill

public TaskSpec constructTaskSpec(DAG dag, String vertexName, int numSplits, ApplicationId appId, int index) {
 Vertex vertex = dag.getVertex(vertexName);
 ProcessorDescriptor processorDescriptor = vertex.getProcessorDescriptor();
 List<RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor>> inputs =
   vertex.getInputs();
 List<RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor>> outputs =
   vertex.getOutputs();
 Preconditions.checkState(inputs.size() == 1);
 Preconditions.checkState(outputs.size() == 1);
 List<InputSpec> inputSpecs = new ArrayList<>();
 for (RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor> input : inputs) {
  InputSpec inputSpec = new InputSpec(input.getName(), input.getIODescriptor(), 1);
  inputSpecs.add(inputSpec);
 }
 List<OutputSpec> outputSpecs = new ArrayList<>();
 for (RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor> output : outputs) {
  OutputSpec outputSpec = new OutputSpec(output.getName(), output.getIODescriptor(), 1);
  outputSpecs.add(outputSpec);
 }
 TezDAGID dagId = TezDAGID.getInstance(appId, 0);
 TezVertexID vertexId = TezVertexID.getInstance(dagId, 0);
 TezTaskID taskId = TezTaskID.getInstance(vertexId, index);
 TezTaskAttemptID taskAttemptId = TezTaskAttemptID.getInstance(taskId, 0);
 return new TaskSpec(taskAttemptId, dag.getName(), vertexName, numSplits, processorDescriptor, inputSpecs, outputSpecs, null);
}

代码示例来源:origin: org.apache.tez/tez-mapreduce

if (stageConf.getBoolean(TezRuntimeConfiguration.TEZ_RUNTIME_CONVERT_USER_PAYLOAD_TO_HISTORY_TEXT,
  TezRuntimeConfiguration.TEZ_RUNTIME_CONVERT_USER_PAYLOAD_TO_HISTORY_TEXT_DEFAULT)) {
 vertex.getProcessorDescriptor().setHistoryText(TezUtils.convertToHistoryText(stageConf));
 LOG.debug("Adding vertex to DAG" + ", vertexName="
   + vertex.getName() + ", processor="
   + vertex.getProcessorDescriptor().getClassName() + ", parallelism="
   + vertex.getParallelism() + ", javaOpts=" + vertex.getTaskLaunchCmdOpts()
   + ", resources=" + vertex.getTaskResource()

代码示例来源:origin: org.apache.pig/pig

if (v != null && tezVertexStatsMap.containsKey(v.getName())) {
  TezVertexStats vertexStats = entry.getValue();
  UserPayload payload = v.getProcessorDescriptor().getUserPayload();
  Configuration conf = TezUtils.createConfFromUserPayload(payload);
  vertexStats.setConf(conf);

代码示例来源:origin: org.apache.tez/tez-api

.convertToDAGPlan(vertex.getProcessorDescriptor()));

相关文章