org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperHaServices类的使用及代码示例

x33g5p2x  于2022-02-05 转载在 其他  
字(9.5k)|赞(0)|评价(0)|浏览(121)

本文整理了Java中org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperHaServices类的一些代码示例，展示了ZooKeeperHaServices类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。ZooKeeperHaServices类的具体详情如下：
包路径：org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperHaServices
类名称：ZooKeeperHaServices

ZooKeeperHaServices介绍

[英]An implementation of the HighAvailabilityServices using Apache ZooKeeper. The services store data in ZooKeeper's nodes as illustrated by teh following tree structure:

/flink 
+/cluster_id_1/resource_manager_lock 
|            | 
|            +/job-id-1/job_manager_lock 
|            |         /checkpoints/latest 
|            |                     /latest-1 
|            |                     /latest-2 
|            | 
|            +/job-id-2/job_manager_lock 
|       
+/cluster_id_2/resource_manager_lock 
| 
+/job-id-1/job_manager_lock 
|/checkpoints/latest 
|            /latest-1 
|/persisted_job_graph

The root path "/flink" is configurable via the option HighAvailabilityOptions#HA_ZOOKEEPER_ROOT. This makes sure Flink stores its data under specific subtrees in ZooKeeper, for example to accommodate specific permission.

The "cluster_id" part identifies the data stored for a specific Flink "cluster". This "cluster" can be either a standalone or containerized Flink cluster, or it can be job on a framework like YARN or Mesos (in a "per-job-cluster" mode).

In case of a "per-job-cluster" on YARN or Mesos, the cluster-id is generated and configured automatically by the client or dispatcher that submits the Job to YARN or Mesos.

In the case of a standalone cluster, that cluster-id needs to be configured via HighAvailabilityOptions#HA_CLUSTER_ID. All nodes with the same cluster id will join the same cluster and participate in the execution of the same set of jobs.
[中]使用Apache ZooKeeper实现HighAvailabilityServices。服务将数据存储在ZooKeeper的节点中，如以下树结构所示：

/flink 
+/cluster_id_1/resource_manager_lock 
|            | 
|            +/job-id-1/job_manager_lock 
|            |         /checkpoints/latest 
|            |                     /latest-1 
|            |                     /latest-2 
|            | 
|            +/job-id-2/job_manager_lock 
|       
+/cluster_id_2/resource_manager_lock 
| 
+/job-id-1/job_manager_lock 
|/checkpoints/latest 
|            /latest-1 
|/persisted_job_graph

根路径“/flink”可通过选项HighAvailabilityOptions#HA#u ZOOKEEPER_root进行配置。这确保Flink将其数据存储在ZooKeeper中的特定子树下，例如，以适应特定的权限。
“cluster_id”部分标识为特定Flink“cluster”存储的数据。这个“集群”可以是独立的或集装箱化的Flink集群，也可以是框架上的作业，比如纱线或Mesos（在“每作业集群”模式下）。
如果在Thread或Mesos上有“每个作业群集”，则群集id由向Thread或Mesos提交作业的客户端或调度程序自动生成和配置。
对于独立群集，需要通过HighAvailabilityOptions#HA#u cluster_id配置该群集id。具有相同群集id的所有节点将加入同一群集，并参与执行同一组作业。

代码示例

代码示例来源：origin: com.alibaba.blink/flink-runtime

@Override
public LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID) {
  return ZooKeeperUtils.createLeaderRetrievalService(client, configuration, getPathForJobManager(jobID));
}

代码示例来源：origin: org.apache.flink/flink-runtime

@Override
public LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID, String defaultJobManagerAddress) {
  return getJobManagerLeaderRetriever(jobID);
}

代码示例来源：origin: com.alibaba.blink/flink-runtime

@Override
public void closeAndCleanupAllData() throws Exception {
  LOG.info("Close and clean up all data for ZooKeeperHaServices.");
  Throwable exception = null;
  try {
    blobStoreService.closeAndCleanupAllData();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close and clean up all data of ZooKeeperHaServices.");
  }
}

代码示例来源：origin: com.alibaba.blink/flink-runtime

public static HighAvailabilityServices createAvailableOrEmbeddedServices(
  Configuration config,
  Executor executor) throws Exception {
  HighAvailabilityMode highAvailabilityMode = LeaderRetrievalUtils.getRecoveryMode(config);
  switch (highAvailabilityMode) {
    case NONE:
      return new EmbeddedHaServices(executor);
    case FILESYSTEM:
      throw new UnsupportedOperationException("to be implemented");
    case ZOOKEEPER:
      BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(config);
      return new ZooKeeperHaServices(
        ZooKeeperUtils.startCuratorFramework(config),
        executor,
        config,
        blobStoreService);
    default:
      throw new Exception("High availability mode " + highAvailabilityMode + " is not supported.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime

@Override
public void closeAndCleanupAllData() throws Exception {
  LOG.info("Close and clean up all data for ZooKeeperHaServices.");
  Throwable exception = null;
  try {
    blobStoreService.closeAndCleanupAllData();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close and clean up all data of ZooKeeperHaServices.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.10

public static HighAvailabilityServices createAvailableOrEmbeddedServices(
  Configuration config,
  Executor executor) throws Exception {
  HighAvailabilityMode highAvailabilityMode = LeaderRetrievalUtils.getRecoveryMode(config);
  switch (highAvailabilityMode) {
    case NONE:
      return new EmbeddedHaServices(executor);
    case ZOOKEEPER:
      BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(config);
      return new ZooKeeperHaServices(
        ZooKeeperUtils.startCuratorFramework(config),
        executor,
        config,
        blobStoreService);
    default:
      throw new Exception("High availability mode " + highAvailabilityMode + " is not supported.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

@Override
public void closeAndCleanupAllData() throws Exception {
  LOG.info("Close and clean up all data for ZooKeeperHaServices.");
  Throwable exception = null;
  try {
    blobStoreService.closeAndCleanupAllData();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close and clean up all data of ZooKeeperHaServices.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

@Override
public LeaderElectionService getJobManagerLeaderElectionService(JobID jobID) {
  return ZooKeeperUtils.createLeaderElectionService(client, configuration, getPathForJobManager(jobID));
}

代码示例来源：origin: org.apache.flink/flink-runtime

public static HighAvailabilityServices createAvailableOrEmbeddedServices(
  Configuration config,
  Executor executor) throws Exception {
  HighAvailabilityMode highAvailabilityMode = LeaderRetrievalUtils.getRecoveryMode(config);
  switch (highAvailabilityMode) {
    case NONE:
      return new EmbeddedHaServices(executor);
    case ZOOKEEPER:
      BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(config);
      return new ZooKeeperHaServices(
        ZooKeeperUtils.startCuratorFramework(config),
        executor,
        config,
        blobStoreService);
    case FACTORY_CLASS:
      return createCustomHAServices(config, executor);
    default:
      throw new Exception("High availability mode " + highAvailabilityMode + " is not supported.");
  }
}

代码示例来源：origin: com.alibaba.blink/flink-runtime

@Override
public LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID, String defaultJobManagerAddress) {
  return getJobManagerLeaderRetriever(jobID);
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

@Override
public void close() throws Exception {
  Throwable exception = null;
  try {
    blobStoreService.close();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close the ZooKeeperHaServices.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.10

@Override
public LeaderElectionService getJobManagerLeaderElectionService(JobID jobID) {
  return ZooKeeperUtils.createLeaderElectionService(client, configuration, getPathForJobManager(jobID));
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

public static HighAvailabilityServices createAvailableOrEmbeddedServices(
  Configuration config,
  Executor executor) throws Exception {
  HighAvailabilityMode highAvailabilityMode = LeaderRetrievalUtils.getRecoveryMode(config);
  switch (highAvailabilityMode) {
    case NONE:
      return new EmbeddedHaServices(executor);
    case ZOOKEEPER:
      BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(config);
      return new ZooKeeperHaServices(
        ZooKeeperUtils.startCuratorFramework(config),
        executor,
        config,
        blobStoreService);
    case FACTORY_CLASS:
      return createCustomHAServices(config, executor);
    default:
      throw new Exception("High availability mode " + highAvailabilityMode + " is not supported.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

@Override
public LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID, String defaultJobManagerAddress) {
  return getJobManagerLeaderRetriever(jobID);
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.10

@Override
public void closeAndCleanupAllData() throws Exception {
  Throwable exception = null;
  try {
    blobStoreService.closeAndCleanupAllData();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close and clean up all data of ZooKeeperHaServices.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.11

@Override
public LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID) {
  return ZooKeeperUtils.createLeaderRetrievalService(client, configuration, getPathForJobManager(jobID));
}

代码示例来源：origin: org.apache.flink/flink-runtime_2.10

BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(configuration);
return new ZooKeeperHaServices(
  ZooKeeperUtils.startCuratorFramework(configuration),
  executor,

代码示例来源：origin: org.apache.flink/flink-runtime

@Override
public void close() throws Exception {
  Throwable exception = null;
  try {
    blobStoreService.close();
  } catch (Throwable t) {
    exception = t;
  }
  internalClose();
  if (exception != null) {
    ExceptionUtils.rethrowException(exception, "Could not properly close the ZooKeeperHaServices.");
  }
}

代码示例来源：origin: org.apache.flink/flink-runtime

@Override
public LeaderElectionService getJobManagerLeaderElectionService(JobID jobID) {
  return ZooKeeperUtils.createLeaderElectionService(client, configuration, getPathForJobManager(jobID));
}

代码示例来源：origin: com.alibaba.blink/flink-runtime

BlobStoreService blobStoreService = BlobUtils.createBlobStoreFromConfig(configuration);
return new ZooKeeperHaServices(
  ZooKeeperUtils.startCuratorFramework(configuration),
  executor,

内容来源于网络，如有侵权，请联系作者删除！

ZooKeeperHaServices

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com