org.apache.hadoop.yarn.util.resource.Resources.componentwiseMax()方法的使用及代码示例

x33g5p2x  于2022-01-29 转载在 其他  
字(9.2k)|赞(0)|评价(0)|浏览(96)

本文整理了Java中org.apache.hadoop.yarn.util.resource.Resources.componentwiseMax方法的一些代码示例，展示了Resources.componentwiseMax的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Resources.componentwiseMax方法的具体详情如下：
包路径：org.apache.hadoop.yarn.util.resource.Resources
类名称：Resources
方法名：componentwiseMax

Resources.componentwiseMax介绍

暂无

代码示例

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

public void setHeadroom(Resource globalLimit) {
 this.resourceLimit = Resources.componentwiseMax(globalLimit,
   Resources.none());
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

/**
 * Get the maximum capacity across specified time instances. The search-space
 * is specified using the starting value, tick, and the periodic interval for
 * search. Maximum resource allocation across tick, tick + period, tick + 2 *
 * period,..., tick + n * period .. is returned.
 *
 * @param tick the starting time instance
 * @param period interval at which capacity is evaluated
 * @return maximum resource allocation
 */
public Resource getMaximumPeriodicCapacity(long tick, long period) {
 Resource maxCapacity = ZERO_RESOURCE;
 readLock.lock();
 try {
  if (!cumulativeCapacity.isEmpty()) {
   Long lastKey = cumulativeCapacity.lastKey();
   for (long t = tick; t <= lastKey; t = t + period) {
    maxCapacity = Resources.componentwiseMax(maxCapacity,
      cumulativeCapacity.floorEntry(t).getValue());
   }
  }
  return maxCapacity;
 } finally {
  readLock.unlock();
 }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

@Override
public Resource getMaxShare() {
 Resource maxResource = maxShare.getResource(scheduler.getClusterResource());
 // Max resource should be greater than or equal to min resource
 Resource result = Resources.componentwiseMax(maxResource, minShare);
 if (!Resources.equals(maxResource, result)) {
  LOG.warn(String.format("Queue %s has max resources %s less than "
    + "min resources %s", getName(), maxResource, minShare));
 }
 return result;
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

public synchronized Resource getTotalPendingResourcesConsideringUserLimit(
  Resource resources) {
 Map<String, Resource> userNameToHeadroom = new HashMap<String, Resource>();
 Resource pendingConsideringUserLimit = Resource.newInstance(0, 0);
 for (FiCaSchedulerApp app : activeApplications) {
  String userName = app.getUser();
  if (!userNameToHeadroom.containsKey(userName)) {
   User user = getUser(userName);
   Resource headroom = Resources.subtract(
     computeUserLimit(app, resources, minimumAllocation, user, null),
     user.getUsed());
   // Make sure none of the the components of headroom is negative.
   headroom = Resources.componentwiseMax(headroom, Resources.none());
   userNameToHeadroom.put(userName, headroom);
  }
  Resource minpendingConsideringUserLimit =
    Resources.componentwiseMin(userNameToHeadroom.get(userName),
                  app.getTotalPendingRequests());
  Resources.addTo(pendingConsideringUserLimit, minpendingConsideringUserLimit);
  Resources.subtractFrom(userNameToHeadroom.get(userName),
              minpendingConsideringUserLimit);
 }
 return pendingConsideringUserLimit;
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

public synchronized Resource getTotalPendingResourcesConsideringUserLimit(
  Resource resources) {
 Map<String, Resource> userNameToHeadroom = new HashMap<String, Resource>();
 Resource pendingConsideringUserLimit = Resource.newInstance(0, 0);
 for (FiCaSchedulerApp app : activeApplications) {
  String userName = app.getUser();
  if (!userNameToHeadroom.containsKey(userName)) {
   User user = getUser(userName);
   Resource headroom = Resources.subtract(
     computeUserLimit(app, resources, minimumAllocation, user, null),
     user.getUsed());
   // Make sure none of the the components of headroom is negative.
   headroom = Resources.componentwiseMax(headroom, Resources.none());
   userNameToHeadroom.put(userName, headroom);
  }
  Resource minpendingConsideringUserLimit =
    Resources.componentwiseMin(userNameToHeadroom.get(userName),
                  app.getTotalPendingRequests());
  Resources.addTo(pendingConsideringUserLimit, minpendingConsideringUserLimit);
  Resources.subtractFrom(userNameToHeadroom.get(userName),
              minpendingConsideringUserLimit);
 }
 return pendingConsideringUserLimit;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

user.getUsed(partition));
 headroom = Resources.componentwiseMax(headroom, Resources.none());
 userNameToHeadroom.put(userName, headroom);
   app.getAppAttemptResourceUsage().getReserved(partition));
pending = Resources.componentwiseMax(pending, Resources.none());

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

Resource offer(Resource avail, ResourceCalculator rc,
  Resource clusterResource) {
 Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
         Resources.subtract(maxCapacity, idealAssigned),
         Resource.newInstance(0, 0));
 // remain = avail - min(avail, (max - assigned), (current + pending - assigned))
 Resource accepted = 
   Resources.min(rc, clusterResource, 
     absMaxCapIdealAssignedDelta,
   Resources.min(rc, clusterResource, avail, Resources.subtract(
     Resources.add(current, pending), idealAssigned)));
 Resource remain = Resources.subtract(avail, accepted);
 Resources.addTo(idealAssigned, accepted);
 return remain;
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

Resource offer(Resource avail, ResourceCalculator rc,
  Resource clusterResource) {
 Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
         Resources.subtract(maxCapacity, idealAssigned),
         Resource.newInstance(0, 0));
 // remain = avail - min(avail, (max - assigned), (current + pending - assigned))
 Resource accepted = 
   Resources.min(rc, clusterResource, 
     absMaxCapIdealAssignedDelta,
   Resources.min(rc, clusterResource, avail, Resources.subtract(
     Resources.add(current, pending), idealAssigned)));
 Resource remain = Resources.subtract(avail, accepted);
 Resources.addTo(idealAssigned, accepted);
 return remain;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

unassigned = Resources.componentwiseMax(unassigned, Resources.none());

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

@Test
public void testComponentwiseMax() {
 assertEquals(createResource(2, 2),
   componentwiseMax(createResource(1, 1), createResource(2, 2)));
 assertEquals(createResource(2, 2),
   componentwiseMax(createResource(2, 2), createResource(1, 1)));
 assertEquals(createResource(2, 2),
   componentwiseMax(createResource(1, 2), createResource(2, 1)));
 assertEquals(createResource(2, 2, 2),
   componentwiseMax(createResource(1, 1, 1), createResource(2, 2, 2)));
 assertEquals(createResource(2, 2, 2),
   componentwiseMax(createResource(2, 2, 2), createResource(1, 1)));
 assertEquals(createResource(2, 2, 3),
   componentwiseMax(createResource(1, 2, 2), createResource(2, 1, 3)));
 assertEquals(createResource(2, 2, 1),
   componentwiseMax(createResource(2, 2, 0), createResource(2, 1, 1)));
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

isA(String.class), eq(true))).thenReturn(Resources.componentwiseMax(
Resources.subtract(pending[i],
  reserved[i] == null ? Resources.none() : reserved[i]),

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

Resource offer(Resource avail, ResourceCalculator rc,
  Resource clusterResource, boolean considersReservedResource,
  boolean allowQueueBalanceAfterAllSafisfied) {
 Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
   Resources.subtract(getMax(), idealAssigned),
   Resource.newInstance(0, 0));
 accepted = Resources.componentwiseMax(accepted, Resources.none());

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private Resource getResourceToIncrease(UpdateContainerRequest updateReq,
  RMContainer rmContainer) {
 if (updateReq.getContainerUpdateType() ==
   ContainerUpdateType.PROMOTE_EXECUTION_TYPE) {
  return rmContainer.getContainer().getResource();
 }
 if (updateReq.getContainerUpdateType() ==
   ContainerUpdateType.INCREASE_RESOURCE) {
  //       This has to equal the Resources in excess of fitsIn()
  //       for container increase and is equal to the container total
  //       resource for Promotion.
  Resource maxCap = Resources.componentwiseMax(updateReq.getCapability(),
    rmContainer.getContainer().getResource());
  return Resources.add(maxCap,
    Resources.negate(rmContainer.getContainer().getResource()));
 }
 return null;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private static Resource getMaxAvailableResourceToQueue(
  final ResourceCalculator rc, RMNodeLabelsManager nlm, CSQueue queue,
  Resource cluster) {
 Set<String> nodeLabels = queue.getNodeLabelsForQueue();
 Resource totalAvailableResource = Resources.createResource(0, 0);
 for (String partition : nodeLabels) {
  // Calculate guaranteed resource for a label in a queue by below logic.
  // (total label resource) * (absolute capacity of label in that queue)
  Resource queueGuaranteedResource = queue.getEffectiveCapacity(partition);
  // Available resource in queue for a specific label will be calculated as
  // {(guaranteed resource for a label in a queue) -
  // (resource usage of that label in the queue)}
  // Finally accumulate this available resource to get total.
  Resource available = (Resources.greaterThan(rc, cluster,
    queueGuaranteedResource,
    queue.getQueueResourceUsage().getUsed(partition))) ? Resources
    .componentwiseMax(Resources.subtractFrom(queueGuaranteedResource,
      queue.getQueueResourceUsage().getUsed(partition)), Resources
      .none()) : Resources.none();
  Resources.addTo(totalAvailableResource, available);
 }
 return totalAvailableResource;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

.componentwiseMax(toObtainAfterPreemption, Resources.none()),
Resources.componentwiseMax(toObtainByPartition, Resources.none()));

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

container.lastConfirmedResource = Resources.componentwiseMax(
   nmContainerResource, container.lastConfirmedResource);
} else {

内容来源于网络，如有侵权，请联系作者删除！

resources

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com