本文整理了Java中org.apache.tez.dag.api.Vertex.getProcessorDescriptor()
方法的一些代码示例,展示了Vertex.getProcessorDescriptor()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Vertex.getProcessorDescriptor()
方法的具体详情如下:
包路径:org.apache.tez.dag.api.Vertex
类名称:Vertex
方法名:getProcessorDescriptor
[英]Get the vertex task processor descriptor
[中]
代码示例来源:origin: apache/hive
public TaskSpec constructTaskSpec(DAG dag, String vertexName, int numSplits, ApplicationId appId, int index) {
Vertex vertex = dag.getVertex(vertexName);
ProcessorDescriptor processorDescriptor = vertex.getProcessorDescriptor();
List<RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor>> inputs =
vertex.getInputs();
List<RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor>> outputs =
vertex.getOutputs();
Preconditions.checkState(inputs.size() == 1);
Preconditions.checkState(outputs.size() == 1);
List<InputSpec> inputSpecs = new ArrayList<>();
for (RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor> input : inputs) {
InputSpec inputSpec = new InputSpec(input.getName(), input.getIODescriptor(), 1);
inputSpecs.add(inputSpec);
}
List<OutputSpec> outputSpecs = new ArrayList<>();
for (RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor> output : outputs) {
OutputSpec outputSpec = new OutputSpec(output.getName(), output.getIODescriptor(), 1);
outputSpecs.add(outputSpec);
}
TezDAGID dagId = TezDAGID.getInstance(appId, 0);
TezVertexID vertexId = TezVertexID.getInstance(dagId, 0);
TezTaskID taskId = TezTaskID.getInstance(vertexId, index);
TezTaskAttemptID taskAttemptId = TezTaskAttemptID.getInstance(taskId, 0);
return new TaskSpec(taskAttemptId, dag.getName(), vertexName, numSplits, processorDescriptor, inputSpecs, outputSpecs, null);
}
代码示例来源:origin: apache/drill
public TaskSpec constructTaskSpec(DAG dag, String vertexName, int numSplits, ApplicationId appId, int index) {
Vertex vertex = dag.getVertex(vertexName);
ProcessorDescriptor processorDescriptor = vertex.getProcessorDescriptor();
List<RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor>> inputs =
vertex.getInputs();
List<RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor>> outputs =
vertex.getOutputs();
Preconditions.checkState(inputs.size() == 1);
Preconditions.checkState(outputs.size() == 1);
List<InputSpec> inputSpecs = new ArrayList<>();
for (RootInputLeafOutput<InputDescriptor, InputInitializerDescriptor> input : inputs) {
InputSpec inputSpec = new InputSpec(input.getName(), input.getIODescriptor(), 1);
inputSpecs.add(inputSpec);
}
List<OutputSpec> outputSpecs = new ArrayList<>();
for (RootInputLeafOutput<OutputDescriptor, OutputCommitterDescriptor> output : outputs) {
OutputSpec outputSpec = new OutputSpec(output.getName(), output.getIODescriptor(), 1);
outputSpecs.add(outputSpec);
}
TezDAGID dagId = TezDAGID.getInstance(appId, 0);
TezVertexID vertexId = TezVertexID.getInstance(dagId, 0);
TezTaskID taskId = TezTaskID.getInstance(vertexId, index);
TezTaskAttemptID taskAttemptId = TezTaskAttemptID.getInstance(taskId, 0);
return new TaskSpec(taskAttemptId, dag.getName(), vertexName, numSplits, processorDescriptor, inputSpecs, outputSpecs, null);
}
代码示例来源:origin: org.apache.tez/tez-mapreduce
if (stageConf.getBoolean(TezRuntimeConfiguration.TEZ_RUNTIME_CONVERT_USER_PAYLOAD_TO_HISTORY_TEXT,
TezRuntimeConfiguration.TEZ_RUNTIME_CONVERT_USER_PAYLOAD_TO_HISTORY_TEXT_DEFAULT)) {
vertex.getProcessorDescriptor().setHistoryText(TezUtils.convertToHistoryText(stageConf));
LOG.debug("Adding vertex to DAG" + ", vertexName="
+ vertex.getName() + ", processor="
+ vertex.getProcessorDescriptor().getClassName() + ", parallelism="
+ vertex.getParallelism() + ", javaOpts=" + vertex.getTaskLaunchCmdOpts()
+ ", resources=" + vertex.getTaskResource()
代码示例来源:origin: org.apache.pig/pig
if (v != null && tezVertexStatsMap.containsKey(v.getName())) {
TezVertexStats vertexStats = entry.getValue();
UserPayload payload = v.getProcessorDescriptor().getUserPayload();
Configuration conf = TezUtils.createConfFromUserPayload(payload);
vertexStats.setConf(conf);
代码示例来源:origin: org.apache.tez/tez-api
.convertToDAGPlan(vertex.getProcessorDescriptor()));
内容来源于网络,如有侵权,请联系作者删除!