io.prestosql.spi.Node.getHostAndPort()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(13.1k)|赞(0)|评价(0)|浏览(222)

本文整理了Java中io.prestosql.spi.Node.getHostAndPort()方法的一些代码示例，展示了Node.getHostAndPort()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Node.getHostAndPort()方法的具体详情如下：
包路径：io.prestosql.spi.Node
类名称：Node
方法名：getHostAndPort

Node.getHostAndPort介绍

暂无

代码示例

代码示例来源：origin: prestosql/presto

  1. private static List<HostAddress> getAddressesForNodes(Map<String, Node> nodeMap, Iterable<String> nodeIdentifiers)
  2. {
  3.   ImmutableList.Builder<HostAddress> nodes = ImmutableList.builder();
  4.   for (String id : nodeIdentifiers) {
  5.     Node node = nodeMap.get(id);
  6.     if (node != null) {
  7.       nodes.add(node.getHostAndPort());
  8.     }
  9.   }
  10.   return nodes.build();
  11. }

代码示例来源：origin: prestosql/presto

  1. @Inject
  2. public MemoryPageSinkProvider(MemoryPagesStore pagesStore, NodeManager nodeManager)
  3. {
  4.   this(pagesStore, requireNonNull(nodeManager, "nodeManager is null").getCurrentNode().getHostAndPort());
  5. }

代码示例来源：origin: io.prestosql/presto-memory

  1. @Inject
  2. public MemoryPageSinkProvider(MemoryPagesStore pagesStore, NodeManager nodeManager)
  3. {
  4.   this(pagesStore, requireNonNull(nodeManager, "nodeManager is null").getCurrentNode().getHostAndPort());
  5. }

代码示例来源：origin: prestosql/presto

  1. public synchronized Map<String, Optional<MemoryInfo>> getWorkerMemoryInfo()
  2. {
  3.   Map<String, Optional<MemoryInfo>> memoryInfo = new HashMap<>();
  4.   for (Entry<String, RemoteNodeMemory> entry : nodes.entrySet()) {
  5.     // workerId is of the form "node_identifier [node_host]"
  6.     String workerId = entry.getKey() + " [" + entry.getValue().getNode().getHostAndPort().getHostText() + "]";
  7.     memoryInfo.put(workerId, entry.getValue().getInfo());
  8.   }
  9.   return memoryInfo;
  10. }

代码示例来源：origin: io.prestosql/presto-main

  1. public synchronized Map<String, Optional<MemoryInfo>> getWorkerMemoryInfo()
  2. {
  3.   Map<String, Optional<MemoryInfo>> memoryInfo = new HashMap<>();
  4.   for (Entry<String, RemoteNodeMemory> entry : nodes.entrySet()) {
  5.     // workerId is of the form "node_identifier [node_host]"
  6.     String workerId = entry.getKey() + " [" + entry.getValue().getNode().getHostAndPort().getHostText() + "]";
  7.     memoryInfo.put(workerId, entry.getValue().getInfo());
  8.   }
  9.   return memoryInfo;
  10. }

代码示例来源：origin: prestosql/presto

  1. private ConnectorSplit createBucketSplit(int bucketNumber, Set<ShardNodes> shards)
  2.   {
  3.     // Bucket splits contain all the shards for the bucket
  4.     // and run on the node assigned to the bucket.
  5.     String nodeId = bucketToNode.get().get(bucketNumber);
  6.     Node node = nodesById.get(nodeId);
  7.     if (node == null) {
  8.       throw new PrestoException(NO_NODES_AVAILABLE, "Node for bucket is offline: " + nodeId);
  9.     }
  10.     Set<UUID> shardUuids = shards.stream()
  11.         .map(ShardNodes::getShardUuid)
  12.         .collect(toSet());
  13.     HostAddress address = node.getHostAndPort();
  14.     return new RaptorSplit(connectorId, shardUuids, bucketNumber, address, effectivePredicate, transactionId);
  15.   }
  16. }

代码示例来源：origin: io.prestosql/presto-tpcds

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     TpcdsTableHandle tableHandle = ((TpcdsTableLayoutHandle) layout).getTable();
  6.     Set<Node> nodes = nodeManager.getRequiredWorkerNodes();
  7.     checkState(!nodes.isEmpty(), "No TPCDS nodes available");
  9.     int totalParts = nodes.size() * splitsPerNode;
  10.     int partNumber = 0;
  12.     // Split the data using split and skew by the number of nodes available.
  13.     ImmutableList.Builder<ConnectorSplit> splits = ImmutableList.builder();
  14.     for (Node node : nodes) {
  15.       for (int i = 0; i < splitsPerNode; i++) {
  16.         splits.add(new TpcdsSplit(tableHandle, partNumber, totalParts, ImmutableList.of(node.getHostAndPort()), noSexism));
  17.         partNumber++;
  18.       }
  19.     }
  20.     return new FixedSplitSource(splits.build());
  21.   }
  22. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     TpcdsTableHandle tableHandle = ((TpcdsTableLayoutHandle) layout).getTable();
  6.     Set<Node> nodes = nodeManager.getRequiredWorkerNodes();
  7.     checkState(!nodes.isEmpty(), "No TPCDS nodes available");
  9.     int totalParts = nodes.size() * splitsPerNode;
  10.     int partNumber = 0;
  12.     // Split the data using split and skew by the number of nodes available.
  13.     ImmutableList.Builder<ConnectorSplit> splits = ImmutableList.builder();
  14.     for (Node node : nodes) {
  15.       for (int i = 0; i < splitsPerNode; i++) {
  16.         splits.add(new TpcdsSplit(tableHandle, partNumber, totalParts, ImmutableList.of(node.getHostAndPort()), noSexism));
  17.         partNumber++;
  18.       }
  19.     }
  20.     return new FixedSplitSource(splits.build());
  21.   }
  22. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     TpchTableLayoutHandle tableLayoutHandle = (TpchTableLayoutHandle) layout;
  5.     TpchTableHandle tableHandle = tableLayoutHandle.getTable();
  7.     Set<Node> nodes = nodeManager.getRequiredWorkerNodes();
  9.     int totalParts = nodes.size() * splitsPerNode;
  10.     int partNumber = 0;
  12.     // Split the data using split and skew by the number of nodes available.
  13.     ImmutableList.Builder<ConnectorSplit> splits = ImmutableList.builder();
  14.     for (Node node : nodes) {
  15.       for (int i = 0; i < splitsPerNode; i++) {
  16.         splits.add(new TpchSplit(tableHandle, partNumber, totalParts, ImmutableList.of(node.getHostAndPort()), tableLayoutHandle.getPredicate()));
  17.         partNumber++;
  18.       }
  19.     }
  20.     return new FixedSplitSource(splits.build());
  21.   }
  22. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     InformationSchemaTableLayoutHandle handle = (InformationSchemaTableLayoutHandle) layout;
  6.     List<HostAddress> localAddress = ImmutableList.of(nodeManager.getCurrentNode().getHostAndPort());
  8.     ConnectorSplit split = new InformationSchemaSplit(handle.getTable(), handle.getPrefixes(), localAddress);
  10.     return new FixedSplitSource(ImmutableList.of(split));
  11.   }
  12. }

代码示例来源：origin: io.prestosql/presto-tpch

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     TpchTableLayoutHandle tableLayoutHandle = (TpchTableLayoutHandle) layout;
  5.     TpchTableHandle tableHandle = tableLayoutHandle.getTable();
  7.     Set<Node> nodes = nodeManager.getRequiredWorkerNodes();
  9.     int totalParts = nodes.size() * splitsPerNode;
  10.     int partNumber = 0;
  12.     // Split the data using split and skew by the number of nodes available.
  13.     ImmutableList.Builder<ConnectorSplit> splits = ImmutableList.builder();
  14.     for (Node node : nodes) {
  15.       for (int i = 0; i < splitsPerNode; i++) {
  16.         splits.add(new TpchSplit(tableHandle, partNumber, totalParts, ImmutableList.of(node.getHostAndPort()), tableLayoutHandle.getPredicate()));
  17.         partNumber++;
  18.       }
  19.     }
  20.     return new FixedSplitSource(splits.build());
  21.   }
  22. }

代码示例来源：origin: io.prestosql/presto-main

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     InformationSchemaTableLayoutHandle handle = (InformationSchemaTableLayoutHandle) layout;
  6.     List<HostAddress> localAddress = ImmutableList.of(nodeManager.getCurrentNode().getHostAndPort());
  8.     ConnectorSplit split = new InformationSchemaSplit(handle.getTable(), handle.getPrefixes(), localAddress);
  10.     return new FixedSplitSource(ImmutableList.of(split));
  11.   }
  12. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     LocalFileTableLayoutHandle layoutHandle = (LocalFileTableLayoutHandle) layout;
  5.     LocalFileTableHandle tableHandle = layoutHandle.getTable();
  7.     TupleDomain<LocalFileColumnHandle> effectivePredicate = layoutHandle.getConstraint()
  8.         .transform(LocalFileColumnHandle.class::cast);
  10.     List<ConnectorSplit> splits = nodeManager.getAllNodes().stream()
  11.         .map(node -> new LocalFileSplit(node.getHostAndPort(), tableHandle.getSchemaTableName(), effectivePredicate))
  12.         .collect(Collectors.toList());
  14.     return new FixedSplitSource(splits);
  15.   }
  16. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorTableLayoutHandle layout, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     JmxTableLayoutHandle jmxLayout = (JmxTableLayoutHandle) layout;
  5.     JmxTableHandle tableHandle = jmxLayout.getTable();
  6.     TupleDomain<ColumnHandle> predicate = jmxLayout.getConstraint();
  8.     //TODO is there a better way to get the node column?
  9.     Optional<JmxColumnHandle> nodeColumnHandle = tableHandle.getColumnHandles().stream()
  10.         .filter(jmxColumnHandle -> jmxColumnHandle.getColumnName().equals(NODE_COLUMN_NAME))
  11.         .findFirst();
  12.     checkState(nodeColumnHandle.isPresent(), "Failed to find %s column", NODE_COLUMN_NAME);
  14.     List<ConnectorSplit> splits = nodeManager.getAllNodes().stream()
  15.         .filter(node -> {
  16.           NullableValue value = NullableValue.of(createUnboundedVarcharType(), utf8Slice(node.getNodeIdentifier()));
  17.           return predicate.overlaps(fromFixedValues(ImmutableMap.of(nodeColumnHandle.get(), value)));
  18.         })
  19.         .map(node -> new JmxSplit(tableHandle, ImmutableList.of(node.getHostAndPort())))
  20.         .collect(toList());
  22.     return new FixedSplitSource(splits);
  23.   }
  24. }

代码示例来源：origin: prestosql/presto

  1. HostAddress address = nodeManager.getCurrentNode().getHostAndPort();
  2. ConnectorSplit split = new SystemSplit(tableHandle.getConnectorId(), tableHandle, address, constraint);
  3. return new FixedSplitSource(ImmutableList.of(split));
  4. splits.add(new SystemSplit(tableHandle.getConnectorId(), tableHandle, node.getHostAndPort(), constraint));

代码示例来源：origin: prestosql/presto

  1. @Test
  2. public void testScheduleLocal()
  3. {
  4.   Split split = new Split(CONNECTOR_ID, TestingTransactionHandle.create(), new TestSplitLocal());
  5.   Set<Split> splits = ImmutableSet.of(split);
  6.   Map.Entry<Node, Split> assignment = Iterables.getOnlyElement(nodeSelector.computeAssignments(splits, ImmutableList.copyOf(taskMap.values())).getAssignments().entries());
  7.   assertEquals(assignment.getKey().getHostAndPort(), split.getAddresses().get(0));
  8.   assertEquals(assignment.getValue(), split);
  9. }

代码示例来源：origin: io.prestosql/presto-main

  1. HostAddress address = nodeManager.getCurrentNode().getHostAndPort();
  2. ConnectorSplit split = new SystemSplit(tableHandle.getConnectorId(), tableHandle, address, constraint);
  3. return new FixedSplitSource(ImmutableList.of(split));
  4. splits.add(new SystemSplit(tableHandle.getConnectorId(), tableHandle, node.getHostAndPort(), constraint));

代码示例来源：origin: io.prestosql/presto-main

  1. @Test
  2. public void testScheduleLocal()
  3. {
  4.   Split split = new Split(CONNECTOR_ID, TestingTransactionHandle.create(), new TestSplitLocal());
  5.   Set<Split> splits = ImmutableSet.of(split);
  6.   Map.Entry<Node, Split> assignment = Iterables.getOnlyElement(nodeSelector.computeAssignments(splits, ImmutableList.copyOf(taskMap.values())).getAssignments().entries());
  7.   assertEquals(assignment.getKey().getHostAndPort(), split.getAddresses().get(0));
  8.   assertEquals(assignment.getValue(), split);
  9. }

代码示例来源：origin: prestosql/presto

  1. private ConnectorSplit createSplit(BucketShards bucketShards)
  2. {
  3.   if (bucketShards.getBucketNumber().isPresent()) {
  4.     return createBucketSplit(bucketShards.getBucketNumber().getAsInt(), bucketShards.getShards());
  5.   }
  6.   verify(bucketShards.getShards().size() == 1, "wrong shard count for non-bucketed table");
  7.   ShardNodes shard = getOnlyElement(bucketShards.getShards());
  8.   UUID shardId = shard.getShardUuid();
  9.   Set<String> nodeIds = shard.getNodeIdentifiers();
  10.   List<HostAddress> addresses = getAddressesForNodes(nodesById, nodeIds);
  11.   if (addresses.isEmpty()) {
  12.     if (!backupAvailable) {
  13.       throw new PrestoException(RAPTOR_NO_HOST_FOR_SHARD, format("No host for shard %s found: %s", shardId, nodeIds));
  14.     }
  15.     // Pick a random node and optimistically assign the shard to it.
  16.     // That node will restore the shard from the backup location.
  17.     Set<Node> availableNodes = nodeSupplier.getWorkerNodes();
  18.     if (availableNodes.isEmpty()) {
  19.       throw new PrestoException(NO_NODES_AVAILABLE, "No nodes available to run query");
  20.     }
  21.     Node node = selectRandom(availableNodes);
  22.     shardManager.replaceShardAssignment(tableId, shardId, node.getNodeIdentifier(), true);
  23.     addresses = ImmutableList.of(node.getHostAndPort());
  24.   }
  25.   return new RaptorSplit(connectorId, shardId, addresses, effectivePredicate, transactionId);
  26. }

代码示例来源：origin: prestosql/presto

  1. @Override
  2.   public ConnectorSplitSource getSplits(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorTableLayoutHandle layoutHandle, SplitSchedulingStrategy splitSchedulingStrategy)
  3.   {
  4.     AtopTableLayoutHandle handle = (AtopTableLayoutHandle) layoutHandle;
  6.     AtopTableHandle table = handle.getTableHandle();
  8.     List<ConnectorSplit> splits = new ArrayList<>();
  9.     ZonedDateTime end = ZonedDateTime.now(timeZone);
  10.     for (Node node : nodeManager.getWorkerNodes()) {
  11.       ZonedDateTime start = end.minusDays(maxHistoryDays - 1).withHour(0).withMinute(0).withSecond(0).withNano(0);
  12.       while (start.isBefore(end)) {
  13.         ZonedDateTime splitEnd = start.withHour(23).withMinute(59).withSecond(59).withNano(0);
  14.         Domain splitDomain = Domain.create(ValueSet.ofRanges(Range.range(TIMESTAMP_WITH_TIME_ZONE, 1000 * start.toEpochSecond(), true, 1000 * splitEnd.toEpochSecond(), true)), false);
  15.         if (handle.getStartTimeConstraint().overlaps(splitDomain) && handle.getEndTimeConstraint().overlaps(splitDomain)) {
  16.           splits.add(new AtopSplit(table.getTable(), node.getHostAndPort(), start.toEpochSecond(), start.getZone()));
  17.         }
  18.         start = start.plusDays(1).withHour(0).withMinute(0).withSecond(0).withNano(0);
  19.       }
  20.     }
  22.     return new FixedSplitSource(splits);
  23.   }
  24. }

内容来源于网络，如有侵权，请联系作者删除！

Node

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com