org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(9.4k)|赞(0)|评价(0)|浏览(106)

本文整理了Java中org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster()方法的一些代码示例，展示了HBaseTestingUtility.startMiniMapReduceCluster()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。HBaseTestingUtility.startMiniMapReduceCluster()方法的具体详情如下：
包路径：org.apache.hadoop.hbase.HBaseTestingUtility
类名称：HBaseTestingUtility
方法名：startMiniMapReduceCluster

HBaseTestingUtility.startMiniMapReduceCluster介绍

[英]Starts a MiniMRCluster with a default number of TaskTracker's.
[中]

代码示例

代码示例来源：origin: apache/hbase

/**
 * Starts a <code>MiniMRCluster</code> with a default number of
 * <code>TaskTracker</code>'s.
 *
 * @throws IOException When starting the cluster fails.
 */
public MiniMRCluster startMiniMapReduceCluster() throws IOException {
 // Set a very high max-disk-utilization percentage to avoid the NodeManagers from failing.
 conf.setIfUnset(
   "yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage",
   "99.0");
 startMiniMapReduceCluster(2);
 return mrCluster;
}

代码示例来源：origin: apache/hbase

hbt.startMiniMapReduceCluster();

代码示例来源：origin: apache/hbase

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: apache/hbase

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: apache/hbase

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: apache/hbase

TEST_UTIL.startMiniMapReduceCluster();
BACKUP_ROOT_DIR =
  new Path(new Path(TEST_UTIL.getConfiguration().get("fs.defaultFS")),

代码示例来源：origin: apache/hbase

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

代码示例来源：origin: apache/hbase

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

代码示例来源：origin: KylinOLAP/Kylin

testUtil.startMiniMapReduceCluster();
System.out.println("Minicluster started.");

代码示例来源：origin: KylinOLAP/Kylin

private static void startupMiniClusterAndImportData() throws Exception {
  logger.info("Going to start mini cluster.");
  if (existInClassPath(iiEndpointClassName)) {
    HBaseMiniclusterHelper.UTIL.getConfiguration().setStrings(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY, iiEndpointClassName);
  }
  //https://issues.apache.org/jira/browse/HBASE-11711
  UTIL.getConfiguration().setInt("hbase.master.info.port", -1);//avoid port clobbering
  MiniHBaseCluster hbaseCluster = UTIL.startMiniCluster();
  Configuration config = hbaseCluster.getConf();
  String host = config.get(HConstants.ZOOKEEPER_QUORUM);
  String port = config.get(HConstants.ZOOKEEPER_CLIENT_PORT);
  String parent = config.get(HConstants.ZOOKEEPER_ZNODE_PARENT);
  // see in: https://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
  config.set("zookeeper.session.timeout", "1200000");
  config.set("hbase.zookeeper.property.tickTime", "6000");
  // reduce rpc retry
  config.set(HConstants.HBASE_CLIENT_PAUSE, "3000");
  config.set(HConstants.HBASE_CLIENT_RETRIES_NUMBER, "1");
  config.set(HConstants.HBASE_CLIENT_OPERATION_TIMEOUT, "60000");
  hbaseconnectionUrl = "hbase:" + host + ":" + port + ":" + parent;
  updateKylinConfigWithMinicluster();
  UTIL.startMiniMapReduceCluster();
  // create the metadata htables;
  @SuppressWarnings("unused")
  HBaseResourceStore store = new HBaseResourceStore(KylinConfig.getInstanceFromEnv());
  // import the table content
  HbaseImporter.importHBaseData(hbaseTarLocation, UTIL.getConfiguration());
}

代码示例来源：origin: org.apache.hbase/hbase-server

/**
 * Starts a <code>MiniMRCluster</code> with a default number of
 * <code>TaskTracker</code>'s.
 *
 * @throws IOException When starting the cluster fails.
 */
public MiniMRCluster startMiniMapReduceCluster() throws IOException {
 // Set a very high max-disk-utilization percentage to avoid the NodeManagers from failing.
 conf.setIfUnset(
   "yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage",
   "99.0");
 startMiniMapReduceCluster(2);
 return mrCluster;
}

代码示例来源：origin: XiaoMi/themis

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 TransactionTestBase.setUpBeforeClass();
 if (TEST_UTIL != null) {
  TEST_UTIL.startMiniMapReduceCluster();
 }
}

代码示例来源：origin: com.aliyun.hbase/alihbase-mapreduce

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(1, 3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: org.apache.hbase/hbase-mapreduce

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(1, 3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: org.apache.hbase/hbase-mapreduce

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(1, 3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: com.aliyun.hbase/alihbase-mapreduce

@BeforeClass
public static void setUpBeforeClass() throws Exception {
 setUpBaseConf(TEST_UTIL.getConfiguration());
 TEST_UTIL.startMiniCluster(1, 3);
 TEST_UTIL.startMiniMapReduceCluster();
}

代码示例来源：origin: com.aliyun.hbase/alihbase-mapreduce

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(1, 3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

代码示例来源：origin: com.aliyun.hbase/alihbase-mapreduce

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(1, 3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

代码示例来源：origin: org.apache.hbase/hbase-mapreduce

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(1, 3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

代码示例来源：origin: org.apache.hbase/hbase-mapreduce

@BeforeClass
 public static void setUpBeforeClass() throws Exception {
  setUpBaseConf(TEST_UTIL.getConfiguration());
  // Setup separate test-data directory for MR cluster and set corresponding configurations.
  // Otherwise, different test classes running MR cluster can step on each other.
  TEST_UTIL.getDataTestDir();

  // set the always on security provider
  UserProvider.setUserProviderForTesting(TEST_UTIL.getConfiguration(),
   HadoopSecurityEnabledUserProviderForTesting.class);

  // setup configuration
  SecureTestUtil.enableSecurity(TEST_UTIL.getConfiguration());

  TEST_UTIL.startMiniCluster(1, 3);
  TEST_UTIL.startMiniMapReduceCluster();

  // Wait for the ACL table to become available
  TEST_UTIL.waitTableEnabled(AccessControlLists.ACL_TABLE_NAME);
 }
}

内容来源于网络，如有侵权，请联系作者删除！

HBaseTestingUtility

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    HBaseTestingUtility类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com