org.apache.tez.dag.api.Vertex.getParallelism()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(2.3k)|赞(0)|评价(0)|浏览(131)

本文整理了Java中org.apache.tez.dag.api.Vertex.getParallelism()方法的一些代码示例,展示了Vertex.getParallelism()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Vertex.getParallelism()方法的具体详情如下:
包路径:org.apache.tez.dag.api.Vertex
类名称:Vertex
方法名:getParallelism

Vertex.getParallelism介绍

[英]Get the specified number of tasks specified to run in this vertex. It may be -1 if the parallelism is defined at runtime. Parallelism may change at runtime
[中]获取指定在此顶点中运行的指定任务数。如果并行度是在运行时定义的,那么它可能是-1。并行性可能会在运行时发生变化

代码示例

代码示例来源:origin: org.apache.tez/tez-api

if (vertex.getParallelism() > -1) {
 newKnownTasksVertices.add(vertex);
  if (e.getEdgeProperty().getDataMovementType() == DataMovementType.ONE_TO_ONE) {
   Vertex outVertex = e.getOutputVertex();
   if (outVertex.getParallelism() == -1) {
    LOG.info("Inferring parallelism for vertex: "
      + outVertex.getName() + " to be " + v.getParallelism()
      + " from 1-1 connection with vertex " + v.getName());
    outVertex.setParallelism(v.getParallelism());
    newKnownTasksVertices.add(outVertex);
 if (inputVertex.getParallelism() != outputVertex.getParallelism()) {
  if (outputVertex.getParallelism() != -1) {
   throw new TezUncheckedException(
     "1-1 Edge. Destination vertex parallelism must match source vertex. "
if (vertex.getParallelism() == -1) {
 boolean hasInputInitializer = false;
 if (vertex.getDataSources() != null && !vertex.getDataSources().isEmpty()) {
 if (vertex.getInputVertices()!= null && !vertex.getInputVertices().isEmpty()) {
  for (Vertex srcVertex : vertex.getInputVertices()) {
   if (srcVertex.getParallelism() == -1) {
    has1to1UninitedSources = true;
    break;

代码示例来源:origin: org.apache.tez/tez-mapreduce

+ vertex.getName() + ", processor="
+ vertex.getProcessorDescriptor().getClassName() + ", parallelism="
+ vertex.getParallelism() + ", javaOpts=" + vertex.getTaskLaunchCmdOpts()
+ ", resources=" + vertex.getTaskResource()

代码示例来源:origin: org.apache.pig/pig

continue;
vertexStats.accumulateStats(status, v.getParallelism());
if(vertexStats.getInputs() != null && !vertexStats.getInputs().isEmpty()) {
  inputs.addAll(vertexStats.getInputs());

代码示例来源:origin: org.apache.tez/tez-api

vertexLRs.put(TezConstants.TEZ_PB_BINARY_CONF_NAME, binaryConfig);
int vertexParallelism = vertex.getParallelism();
VertexLocationHint vertexLocationHint = vertex.getLocationHint();
if (dataSources.size() == 1) {

相关文章