org.apache.hadoop.hbase.procedure.ZKProcedureUtil类的使用及代码示例

x33g5p2x  于2022-02-05 转载在 其他  
字(12.5k)|赞(0)|评价(0)|浏览(126)

本文整理了Java中org.apache.hadoop.hbase.procedure.ZKProcedureUtil类的一些代码示例，展示了ZKProcedureUtil类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。ZKProcedureUtil类的具体详情如下：
包路径：org.apache.hadoop.hbase.procedure.ZKProcedureUtil
类名称：ZKProcedureUtil

ZKProcedureUtil介绍

[英]This is a shared ZooKeeper-based znode management utils for distributed procedure. All znode operations should go through the provided methods in coordinators and members. Layout of nodes in ZK is /hbase/[op name]/acquired/ [op instance] - op data/ /[nodes that have acquired] /reached/ [op instance]/ /[nodes that have completed] /abort/ [op instance] - failure data NOTE: while acquired and completed are znode dirs, abort is actually just a znode. Assumption here that procedure names are unique
[中]这是一个基于ZooKeeper的共享znode管理UTIL，用于分布式过程。所有znode操作都应该通过协调员和成员中提供的方法进行。ZK中的节点布局为/hbase/[op name]/acquired/[op instance]-op data//[nodes have acquired]/reach/[op instance]/[nodes have completed]/abort/[op instance]-failure data注意：虽然acquired和completed是znode dirs，但abort实际上只是znode。这里假设过程名称是唯一的

代码示例

代码示例来源：origin: apache/hbase

/**
 * Verify that the prepare, commit and abort nodes for the operation are removed from zookeeper
 */
private void verifyZooKeeperClean(String operationName, ZKWatcher watcher,
  ZKProcedureUtil controller) throws Exception {
 String prepare = ZKProcedureUtil.getAcquireBarrierNode(controller, operationName);
 String commit = ZKProcedureUtil.getReachedBarrierNode(controller, operationName);
 String abort = ZKProcedureUtil.getAbortNode(controller, operationName);
 assertEquals("Didn't delete prepare node", -1, ZKUtil.checkExists(watcher, prepare));
 assertEquals("Didn't delete commit node", -1, ZKUtil.checkExists(watcher, commit));
 assertEquals("Didn't delete abort node", -1, ZKUtil.checkExists(watcher, abort));
}

代码示例来源：origin: apache/hbase

/**
 * This should be called by the member and should write a serialized root cause exception as
 * to the abort znode.
 */
@Override
public void sendMemberAborted(Subprocedure sub, ForeignException ee) {
 if (sub == null) {
  LOG.error("Failed due to null subprocedure", ee);
  return;
 }
 String procName = sub.getName();
 LOG.debug("Aborting procedure (" + procName + ") in zk");
 String procAbortZNode = zkController.getAbortZNode(procName);
 try {
  String source = (ee.getSource() == null) ? memberName: ee.getSource();
  byte[] errorInfo = ProtobufUtil.prependPBMagic(ForeignException.serialize(source, ee));
  ZKUtil.createAndFailSilent(zkController.getWatcher(), procAbortZNode, errorInfo);
  LOG.debug("Finished creating abort znode:" + procAbortZNode);
 } catch (KeeperException e) {
  // possible that we get this error for the procedure if we already reset the zk state, but in
  // that case we should still get an error for that procedure anyways
  zkController.logZKTree(zkController.getBaseZnode());
  member.controllerConnectionFailure("Failed to post zk node:" + procAbortZNode
    + " to abort procedure", e, procName);
 }
}

代码示例来源：origin: apache/hbase

zkProc.clearChildZNodes();
} catch (KeeperException e) {
 LOG.error("Unable to start the ZK-based Procedure Coordinator rpcs.", e);

代码示例来源：origin: apache/hbase

public void clearZNodes(String procedureName) throws KeeperException {
  LOG.info("Clearing all znodes for procedure " + procedureName + "including nodes "
    + acquiredZnode + " " + reachedZnode + " " + abortZnode);

  // Make sure we trigger the watches on these nodes by creating them. (HBASE-13885)
  String acquiredBarrierNode = getAcquiredBarrierNode(procedureName);
  String reachedBarrierNode = getReachedBarrierNode(procedureName);
  String abortZNode = getAbortZNode(procedureName);

  ZKUtil.createAndFailSilent(watcher, acquiredBarrierNode);
  ZKUtil.createAndFailSilent(watcher, abortZNode);

  ZKUtil.deleteNodeRecursivelyMultiOrSequential(watcher, true, acquiredBarrierNode,
   reachedBarrierNode, abortZNode);

  if (LOG.isTraceEnabled()) {
   logZKTree(this.baseZNode);
  }
 }
}

代码示例来源：origin: apache/hbase

private void watchForAbortedProcedures() {
 LOG.debug("Checking for aborted procedures on node: '" + zkController.getAbortZnode() + "'");
 try {
  // this is the list of the currently aborted procedues
  List<String> children = ZKUtil.listChildrenAndWatchForNewChildren(zkController.getWatcher(),
         zkController.getAbortZnode());
  if (children == null || children.isEmpty()) {
   return;
  }
  for (String node : children) {
   String abortNode = ZNodePaths.joinZNode(zkController.getAbortZnode(), node);
   abort(abortNode);
  }
 } catch (KeeperException e) {
  member.controllerConnectionFailure("Failed to list children for abort node:"
    + zkController.getAbortZnode(), e, null);
 }
}

代码示例来源：origin: apache/hbase

private void waitForNewProcedures() {
 // watch for new procedues that we need to start subprocedures for
 LOG.debug("Looking for new procedures under znode:'" + zkController.getAcquiredBarrier() + "'");
 List<String> runningProcedures = null;
 try {
  runningProcedures = ZKUtil.listChildrenAndWatchForNewChildren(zkController.getWatcher(),
   zkController.getAcquiredBarrier());
  if (runningProcedures == null) {
   LOG.debug("No running procedures.");
   return;
  }
 } catch (KeeperException e) {
  member.controllerConnectionFailure("General failure when watching for new procedures",
   e, null);
 }
 if (runningProcedures == null) {
  LOG.debug("No running procedures.");
  return;
 }
 for (String procName : runningProcedures) {
  // then read in the procedure information
  String path = ZNodePaths.joinZNode(zkController.getAcquiredBarrier(), procName);
  startNewSubprocedure(path);
 }
}

代码示例来源：origin: apache/hbase

/**
 * This attempts to create an acquired state znode for the procedure (snapshot name).
 *
 * It then looks for the reached znode to trigger in-barrier execution.  If not present we
 * have a watcher, if present then trigger the in-barrier action.
 */
@Override
public void sendMemberAcquired(Subprocedure sub) throws IOException {
 String procName = sub.getName();
 try {
  LOG.debug("Member: '" + memberName + "' joining acquired barrier for procedure (" + procName
    + ") in zk");
  String acquiredZNode = ZNodePaths.joinZNode(ZKProcedureUtil.getAcquireBarrierNode(
   zkController, procName), memberName);
  ZKUtil.createAndFailSilent(zkController.getWatcher(), acquiredZNode);
  // watch for the complete node for this snapshot
  String reachedBarrier = zkController.getReachedBarrierNode(procName);
  LOG.debug("Watch for global barrier reached:" + reachedBarrier);
  if (ZKUtil.watchAndCheckExists(zkController.getWatcher(), reachedBarrier)) {
   receivedReachedGlobalBarrier(reachedBarrier);
  }
 } catch (KeeperException e) {
  member.controllerConnectionFailure("Failed to acquire barrier for procedure: "
    + procName + " and member: " + memberName, e, procName);
 }
}

代码示例来源：origin: apache/hbase

@Override
public void sendGlobalBarrierReached(Procedure proc, List<String> nodeNames) throws IOException {
 String procName = proc.getName();
 String reachedNode = zkProc.getReachedBarrierNode(procName);
 LOG.debug("Creating reached barrier zk node:" + reachedNode);
 try {
  ZKUtil.createWithParents(zkProc.getWatcher(), reachedNode);
   if (ZKUtil.watchAndCheckExists(zkProc.getWatcher(), znode)) {
    byte[] dataFromMember = ZKUtil.getData(zkProc.getWatcher(), znode);

代码示例来源：origin: apache/hbase

String prepare = ZKProcedureUtil.getAcquireBarrierNode(controller.getZkController(), operationName);
ZKUtil.createSetData(watcher, prepare, ProtobufUtil.prependPBMagic(data));
String commit = ZKProcedureUtil.getReachedBarrierNode(controller.getZkController(), operationName);
LOG.debug("Found prepared, posting commit node:" + commit);
ZKUtil.createAndFailSilent(watcher, commit);
ZKUtil.deleteNodeRecursively(watcher, controller.getZkController().getBaseZnode());
assertEquals("Didn't delete prepare node", -1, ZKUtil.checkExists(watcher, prepare));
assertEquals("Didn't delete commit node", -1, ZKUtil.checkExists(watcher, commit));

代码示例来源：origin: apache/hbase

/**
 * This is the abort message being sent by the coordinator to member
 *
 * TODO this code isn't actually used but can be used to issue a cancellation from the
 * coordinator.
 */
@Override
final public void sendAbortToMembers(Procedure proc, ForeignException ee) {
 String procName = proc.getName();
 LOG.debug("Aborting procedure '" + procName + "' in zk");
 String procAbortNode = zkProc.getAbortZNode(procName);
 try {
  LOG.debug("Creating abort znode:" + procAbortNode);
  String source = (ee.getSource() == null) ? coordName : ee.getSource();
  byte[] errorInfo = ProtobufUtil.prependPBMagic(ForeignException.serialize(source, ee));
  // first create the znode for the procedure
  ZKUtil.createAndFailSilent(zkProc.getWatcher(), procAbortNode, errorInfo);
  LOG.debug("Finished creating abort node:" + procAbortNode);
 } catch (KeeperException e) {
  // possible that we get this error for the procedure if we already reset the zk state, but in
  // that case we should still get an error for that procedure anyways
  zkProc.logZKTree(zkProc.baseZNode);
  coordinator.rpcConnectionFailure("Failed to post zk node:" + procAbortNode
    + " to abort procedure '" + procName + "'", new IOException(e));
 }
}

代码示例来源：origin: apache/hbase

String procName = proc.getName();
String abortNode = zkProc.getAbortZNode(procName);
try {
 if (ZKUtil.watchAndCheckExists(zkProc.getWatcher(), abortNode)) {
  abort(abortNode);
String acquire = zkProc.getAcquiredBarrierNode(procName);
LOG.debug("Creating acquire znode:" + acquire);
try {
 ZKUtil.createWithParents(zkProc.getWatcher(), acquire, data);
  if (ZKUtil.watchAndCheckExists(zkProc.getWatcher(), znode)) {
   coordinator.memberAcquiredBarrier(procName, node);

代码示例来源：origin: co.cask.hbase/hbase

public void clearZNodes(String procedureName) throws KeeperException {
  // TODO This is potentially racy since not atomic. update when we support zk that has multi
  LOG.info("Clearing all znodes for procedure " + procedureName + "including nodes "
    + acquiredZnode + " " + reachedZnode + " " + abortZnode);
  ZKUtil.deleteNodeRecursively(watcher, getAcquiredBarrierNode(procedureName));
  ZKUtil.deleteNodeRecursively(watcher, getReachedBarrierNode(procedureName));
  ZKUtil.deleteNodeRecursively(watcher, getAbortZNode(procedureName));
 }
}

代码示例来源：origin: apache/hbase

String opName = ZKUtil.getNodeName(path);
String abortZNode = zkController.getAbortZNode(opName);
try {
 if (ZKUtil.watchAndCheckExists(zkController.getWatcher(), abortZNode)) {
  LOG.debug("Not starting:" + opName + " because we already have an abort notification.");
  return;
 byte[] data = ZKUtil.getData(zkController.getWatcher(), path);
 if (!ProtobufUtil.isPBMagicPrefix(data)) {
  String msg = "Data in for starting procedure " + opName +

代码示例来源：origin: apache/hbase

@Override
final public void close() throws IOException {
 zkProc.close();
}

代码示例来源：origin: apache/hbase

public String getReachedBarrierNode(String opInstanceName) {
 return ZKProcedureUtil.getReachedBarrierNode(this, opInstanceName);
}

代码示例来源：origin: apache/hbase

public String getAbortZNode(String opInstanceName) {
 return ZKProcedureUtil.getAbortNode(this, opInstanceName);
}

代码示例来源：origin: apache/hbase

public String getAcquiredBarrierNode(String opInstanceName) {
 return ZKProcedureUtil.getAcquireBarrierNode(this, opInstanceName);
}

代码示例来源：origin: co.cask.hbase/hbase

zkProc.clearChildZNodes();
} catch (KeeperException e) {
 LOG.error("Unable to start the ZK-based Procedure Coordinator rpcs.", e);
LOG.debug("Starting the controller for procedure member:" + zkProc.getMemberName());
return true;

代码示例来源：origin: apache/hbase

/**
 * Delete znodes that are no longer in use.
 */
@Override
final public void resetMembers(Procedure proc) throws IOException {
 String procName = proc.getName();
 boolean stillGettingNotifications = false;
 do {
  try {
   LOG.debug("Attempting to clean out zk node for op:" + procName);
   zkProc.clearZNodes(procName);
   stillGettingNotifications = false;
  } catch (KeeperException.NotEmptyException e) {
   // recursive delete isn't transactional (yet) so we need to deal with cases where we get
   // children trickling in
   stillGettingNotifications = true;
  } catch (KeeperException e) {
   String msg = "Failed to complete reset procedure " + procName;
   LOG.error(msg, e);
   throw new IOException(msg, e);
  }
 } while (stillGettingNotifications);
}

代码示例来源：origin: co.cask.hbase/hbase

/**
 * This attempts to create an acquired state znode for the procedure (snapshot name).
 *
 * It then looks for the reached znode to trigger in-barrier execution.  If not present we
 * have a watcher, if present then trigger the in-barrier action.
 */
@Override
public void sendMemberAcquired(Subprocedure sub) throws IOException {
 String procName = sub.getName();
 try {
  LOG.debug("Member: '" + memberName + "' joining acquired barrier for procedure (" + procName
    + ") in zk");
  String acquiredZNode = ZKUtil.joinZNode(ZKProcedureUtil.getAcquireBarrierNode(
   zkController, procName), memberName);
  ZKUtil.createAndFailSilent(zkController.getWatcher(), acquiredZNode);
  // watch for the complete node for this snapshot
  String reachedBarrier = zkController.getReachedBarrierNode(procName);
  LOG.debug("Watch for global barrier reached:" + reachedBarrier);
  if (ZKUtil.watchAndCheckExists(zkController.getWatcher(), reachedBarrier)) {
   receivedReachedGlobalBarrier(reachedBarrier);
  }
 } catch (KeeperException e) {
  member.controllerConnectionFailure("Failed to acquire barrier for procedure: "
    + procName + " and member: " + memberName, new IOException(e));
 }
}

内容来源于网络，如有侵权，请联系作者删除！

ZKProcedureUtil

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com