org.apache.samza.zk.ZkKeyBuilder.getJobModelPathPrefix()方法的使用及代码示例

x33g5p2x  于2022-02-05 转载在 其他  
字(10.0k)|赞(0)|评价(0)|浏览(80)

本文整理了Java中org.apache.samza.zk.ZkKeyBuilder.getJobModelPathPrefix()方法的一些代码示例，展示了ZkKeyBuilder.getJobModelPathPrefix()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。ZkKeyBuilder.getJobModelPathPrefix()方法的具体详情如下：
包路径：org.apache.samza.zk.ZkKeyBuilder
类名称：ZkKeyBuilder
方法名：getJobModelPathPrefix

ZkKeyBuilder.getJobModelPathPrefix介绍

暂无

代码示例

代码示例来源：origin: org.apache.samza/samza-core_2.11

String getJobModelPath(String jobModelVersion) {
 return String.format("%s/%s", getJobModelPathPrefix(), jobModelVersion);
}

代码示例来源：origin: apache/samza

String getJobModelPath(String jobModelVersion) {
 return String.format("%s/%s", getJobModelPathPrefix(), jobModelVersion);
}

代码示例来源：origin: org.apache.samza/samza-core_2.12

String getJobModelPath(String jobModelVersion) {
 return String.format("%s/%s", getJobModelPathPrefix(), jobModelVersion);
}

代码示例来源：origin: org.apache.samza/samza-core

String getJobModelPath(String jobModelVersion) {
 return String.format("%s/%s", getJobModelPathPrefix(), jobModelVersion);
}

代码示例来源：origin: org.apache.samza/samza-core_2.10

String getJobModelPath(String jobModelVersion) {
 return String.format("%s/%s", getJobModelPathPrefix(), jobModelVersion);
}

代码示例来源：origin: apache/samza

void deleteOldJobModels(int numVersionsToLeave) {
 // read current list of JMs
 String path = keyBuilder.getJobModelPathPrefix();
 LOG.info("About to delete jm path=" + path);
 List<String> znodeIds = zkClient.getChildren(path);
 deleteOldVersionPath(path, znodeIds, numVersionsToLeave, new Comparator<String>() {
  @Override
  public int compare(String o1, String o2) {
   // jm version name format is <num>
   return Integer.valueOf(o1) - Integer.valueOf(o2);
  }
 });
}

代码示例来源：origin: apache/samza

/**
 * Generates the next JobModel version that should be used by a processor group in a rebalancing phase
 * for coordination.
 * @param currentJobModelVersion the current version of JobModel.
 * @return the next JobModel version.
 */
public String getNextJobModelVersion(String currentJobModelVersion) {
 if (currentJobModelVersion == null) {
  return  "1";
 } else {
  /**
   * There's inconsistency between the maximum published jobModel version and value stored in jobModelVersion
   * zookeeper node. Short term fix is to read all published jobModel versions and choose the maximum version. If there's a
   * inconsistency, update the jobModelVersionPath with maximum published jobModelVersion.
   */
  List<String> publishedJobModelVersions = zkClient.getChildren(keyBuilder.getJobModelPathPrefix());
  metrics.reads.inc(publishedJobModelVersions.size());
  String maxPublishedJMVersion = publishedJobModelVersions.stream()
                              .max(Comparator.comparingInt(Integer::valueOf)).orElse("0");
  return Integer.toString(Math.max(Integer.valueOf(currentJobModelVersion), Integer.valueOf(maxPublishedJMVersion)) + 1);
 }
}

代码示例来源：origin: org.apache.samza/samza-core_2.12

void deleteOldJobModels(int numVersionsToLeave) {
 // read current list of JMs
 String path = keyBuilder.getJobModelPathPrefix();
 LOG.info("About to delete jm path=" + path);
 List<String> znodeIds = zkClient.getChildren(path);
 deleteOldVersionPath(path, znodeIds, numVersionsToLeave, new Comparator<String>() {
  @Override
  public int compare(String o1, String o2) {
   // jm version name format is <num>
   return Integer.valueOf(o1) - Integer.valueOf(o2);
  }
 });
}

代码示例来源：origin: org.apache.samza/samza-core_2.11

void deleteOldJobModels(int numVersionsToLeave) {
 // read current list of JMs
 String path = keyBuilder.getJobModelPathPrefix();
 LOG.info("About to delete jm path=" + path);
 List<String> znodeIds = zkClient.getChildren(path);
 deleteOldVersionPath(path, znodeIds, numVersionsToLeave, new Comparator<String>() {
  @Override
  public int compare(String o1, String o2) {
   // jm version name format is <num>
   return Integer.valueOf(o1) - Integer.valueOf(o2);
  }
 });
}

代码示例来源：origin: org.apache.samza/samza-core

void deleteOldJobModels(int numVersionsToLeave) {
 // read current list of JMs
 String path = keyBuilder.getJobModelPathPrefix();
 LOG.info("About to delete jm path=" + path);
 List<String> znodeIds = zkClient.getChildren(path);
 deleteOldVersionPath(path, znodeIds, numVersionsToLeave, new Comparator<String>() {
  @Override
  public int compare(String o1, String o2) {
   // jm version name format is <num>
   return Integer.valueOf(o1) - Integer.valueOf(o2);
  }
 });
}

代码示例来源：origin: org.apache.samza/samza-core_2.10

/**
 * Generates the next JobModel version that should be used by a processor group in a rebalancing phase
 * for coordination.
 * @param currentJobModelVersion the current version of JobModel.
 * @return the next JobModel version.
 */
public String getNextJobModelVersion(String currentJobModelVersion) {
 if (currentJobModelVersion == null) {
  return  "1";
 } else {
  /**
   * There's inconsistency between the maximum published jobModel version and value stored in jobModelVersion
   * zookeeper node. Short term fix is to read all published jobModel versions and choose the maximum version. If there's a
   * inconsistency, update the jobModelVersionPath with maximum published jobModelVersion.
   */
  List<String> publishedJobModelVersions = zkClient.getChildren(keyBuilder.getJobModelPathPrefix());
  metrics.reads.inc(publishedJobModelVersions.size());
  String maxPublishedJMVersion = publishedJobModelVersions.stream()
                              .max(Comparator.comparingInt(Integer::valueOf)).orElse("0");
  return Integer.toString(Math.max(Integer.valueOf(currentJobModelVersion), Integer.valueOf(maxPublishedJMVersion)) + 1);
 }
}

代码示例来源：origin: org.apache.samza/samza-core_2.10

void deleteOldJobModels(int numVersionsToLeave) {
 // read current list of JMs
 String path = keyBuilder.getJobModelPathPrefix();
 LOG.info("About to delete jm path=" + path);
 List<String> znodeIds = zkClient.getChildren(path);
 deleteOldVersionPath(path, znodeIds, numVersionsToLeave, new Comparator<String>() {
  @Override
  public int compare(String o1, String o2) {
   // jm version name format is <num>
   return Integer.valueOf(o1) - Integer.valueOf(o2);
  }
 });
}

代码示例来源：origin: apache/samza

@Test
public void testCleanUpZkJobModels() {
 String root = zkUtils.getKeyBuilder().getJobModelPathPrefix();
 System.out.println("root=" + root);
 zkUtils.getZkClient().createPersistent(root, true);
 // generate multiple version
 for (int i = 101; i < 110; i++) {
  zkUtils.publishJobModel(String.valueOf(i), null);
 }
 // clean all of the versions except 5 most recent ones
 zkUtils.deleteOldJobModels(5);
 Assert.assertEquals(Arrays.asList("105", "106", "107", "108", "109"), zkUtils.getZkClient().getChildren(root));
}

代码示例来源：origin: apache/samza

@Override
public void start() {
 ZkKeyBuilder keyBuilder = zkUtils.getKeyBuilder();
 zkUtils.validateZkVersion();
 zkUtils.validatePaths(new String[]{keyBuilder.getProcessorsPath(), keyBuilder.getJobModelVersionPath(), keyBuilder.getJobModelPathPrefix(), keyBuilder.getTaskLocalityPath()});
 systemAdmins.start();
 leaderElector.tryBecomeLeader();
 zkUtils.subscribeToJobModelVersionChange(new ZkJobModelVersionChangeHandler(zkUtils));
}

代码示例来源：origin: org.apache.samza/samza-core_2.12

@Override
public void start() {
 ZkKeyBuilder keyBuilder = zkUtils.getKeyBuilder();
 zkUtils.validateZkVersion();
 zkUtils.validatePaths(new String[]{keyBuilder.getProcessorsPath(), keyBuilder.getJobModelVersionPath(), keyBuilder
   .getJobModelPathPrefix()});
 startMetrics();
 systemAdmins.start();
 leaderElector.tryBecomeLeader();
 zkUtils.subscribeToJobModelVersionChange(new ZkJobModelVersionChangeHandler(zkUtils));
}

代码示例来源：origin: org.apache.samza/samza-core_2.10

@Override
public void start() {
 ZkKeyBuilder keyBuilder = zkUtils.getKeyBuilder();
 zkUtils.validateZkVersion();
 zkUtils.validatePaths(new String[]{keyBuilder.getProcessorsPath(), keyBuilder.getJobModelVersionPath(), keyBuilder
   .getJobModelPathPrefix()});
 startMetrics();
 systemAdmins.start();
 leaderElector.tryBecomeLeader();
 zkUtils.subscribeToJobModelVersionChange(new ZkJobModelVersionChangeHandler(zkUtils));
}

代码示例来源：origin: org.apache.samza/samza-core

@Override
public void start() {
 ZkKeyBuilder keyBuilder = zkUtils.getKeyBuilder();
 zkUtils.validateZkVersion();
 zkUtils.validatePaths(new String[]{keyBuilder.getProcessorsPath(), keyBuilder.getJobModelVersionPath(), keyBuilder
   .getJobModelPathPrefix()});
 startMetrics();
 systemAdmins.start();
 leaderElector.tryBecomeLeader();
 zkUtils.subscribeToJobModelVersionChange(new ZkJobModelVersionChangeHandler(zkUtils));
}

代码示例来源：origin: org.apache.samza/samza-core_2.11

@Override
public void start() {
 ZkKeyBuilder keyBuilder = zkUtils.getKeyBuilder();
 zkUtils.validateZkVersion();
 zkUtils.validatePaths(new String[]{keyBuilder.getProcessorsPath(), keyBuilder.getJobModelVersionPath(), keyBuilder
   .getJobModelPathPrefix()});
 startMetrics();
 systemAdmins.start();
 leaderElector.tryBecomeLeader();
 zkUtils.subscribeToJobModelVersionChange(new ZkJobModelVersionChangeHandler(zkUtils));
}

代码示例来源：origin: apache/samza

@Test
 public void testJobModelPath() {
  ZkKeyBuilder builder = new ZkKeyBuilder("test");

  Assert.assertEquals("/test/" + ZkKeyBuilder.JOBMODEL_GENERATION_PATH + "/jobModelVersion", builder.getJobModelVersionPath());
  Assert.assertEquals("/test/" + ZkKeyBuilder.JOBMODEL_GENERATION_PATH + "/jobModels", builder.getJobModelPathPrefix());
  String version = "2";
  Assert.assertEquals("/test/" + ZkKeyBuilder.JOBMODEL_GENERATION_PATH + "/jobModels/" + version, builder.getJobModelPath(version));
  Assert.assertEquals("/test/" + ZkKeyBuilder.JOBMODEL_GENERATION_PATH + "/" + ZkKeyBuilder.JOB_MODEL_UPGRADE_BARRIER_PATH + "/versionBarriers", builder.getJobModelVersionBarrierPrefix());
 }
}

代码示例来源：origin: apache/samza

@Test
public void testgetNextJobModelVersion() {
 // Set up the Zk base paths for testing.
 ZkKeyBuilder keyBuilder = new ZkKeyBuilder("test");
 String root = keyBuilder.getRootPath();
 zkClient.deleteRecursive(root);
 zkUtils.validatePaths(new String[]{root, keyBuilder.getJobModelPathPrefix(), keyBuilder.getJobModelVersionPath()});
 String version = "1";
 String oldVersion = "0";
 // Set zkNode JobModelVersion to 1.
 zkUtils.publishJobModelVersion(oldVersion, version);
 Assert.assertEquals(version, zkUtils.getJobModelVersion());
 // Publish JobModel with a higher version (2).
 zkUtils.publishJobModel("2", new JobModel(new MapConfig(), new HashMap<>()));
 // Get on the JobModel version should return 2, taking into account the published version 2.
 Assert.assertEquals("3", zkUtils.getNextJobModelVersion(zkUtils.getJobModelVersion()));
}

内容来源于网络，如有侵权，请联系作者删除！

ZkKeyBuilder

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com