本文整理了Java中com.dremio.provision.yarn.YarnController
类的一些代码示例,展示了YarnController
类的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。YarnController
类的具体详情如下:
包路径:com.dremio.provision.yarn.YarnController
类名称:YarnController
[英]Class that allows to control Dremio YARN deployment It is a singleton to start only single controller per process and only if needed
[中]类,该类允许控制Dremio纱线部署每个进程只启动一个控制器,并且仅在需要时启动
代码示例来源:origin: dremio/dremio-oss
public YarnService(DremioConfig config, ProvisioningStateListener stateListener, NodeProvider executionNodeProvider) {
this(stateListener, new YarnController(config), executionNodeProvider);
}
代码示例来源:origin: dremio/dremio-oss
private TwillRunnerService getTwillService(Cluster cluster) {
TwillRunnerService twillService = yarnController.getTwillService(cluster.getId());
if (twillService == null) {
// possibly DremioMaster restarted, start TwillRunner
YarnConfiguration yarnConfig = new YarnConfiguration();
updateYarnConfiguration(cluster, yarnConfig);
yarnController.startTwillRunner(yarnConfig);
}
return yarnController.getTwillService(cluster.getId());
}
代码示例来源:origin: dremio/dremio-oss
public TwillController startCluster(YarnConfiguration yarnConfiguration, List<Property> propertyList) {
TwillController tmpController = createPreparer(yarnConfiguration, propertyList).start();
return tmpController;
}
代码示例来源:origin: dremio/dremio-oss
when(controller.startCluster(any(YarnConfiguration.class), eq(cluster.getClusterConfig().getSubPropertyList())))
.thenReturn(twillController);
when(controller.getTwillService(cluster.getId())).thenReturn(twillRunnerService);
when(twillController.getRunId()).thenReturn(runId);
when(resourceReport.getRunnableResources(DacDaemonYarnApplication.YARN_RUNNABLE_NAME)).thenReturn(resources);
代码示例来源:origin: dremio/dremio-oss
@Test
public void testStopCluster() throws Exception {
assumeNonMaprProfile();
Cluster myCluster = createCluster();
myCluster.setState(ClusterState.RUNNING);
YarnController controller = Mockito.mock(YarnController.class);
YarnService yarnService = new YarnService(new TestListener(), controller, Mockito.mock(NodeProvider.class));
TwillController twillController = Mockito.mock(TwillController.class);
RunId runId = RunIds.generate();
when(controller.startCluster(any(YarnConfiguration.class), eq(myCluster.getClusterConfig().getSubPropertyList())))
.thenReturn(twillController);
when(twillController.getRunId()).thenReturn(runId);
myCluster.setRunId(new com.dremio.provision.RunId(runId.getId()));
yarnService.stopCluster(myCluster);
assertEquals(ClusterState.STOPPED, myCluster.getState());
}
代码示例来源:origin: dremio/dremio-oss
new DacDaemonYarnApplication.Environment());
TwillRunnerService twillRunner = startTwillRunner(yarnConfiguration);
.withApplicationClassPaths(yarnClasspath)
.withBundlerClassAcceptor(new HadoopClassExcluder())
.setLogLevels(ImmutableMap.of(Logger.ROOT_LOGGER_NAME, yarnContainerLogLevel()))
.withEnv(envVars)
.withMaxRetries(YARN_RUNNABLE_NAME, MAX_APP_RESTART_RETRIES)
preparer.addJVMOptions(prepareCommandOptions(yarnConfiguration, propertyList));
代码示例来源:origin: dremio/dremio-oss
@Test
public void testYarnController() throws Exception {
assumeNonMaprProfile();
YarnController yarnController = new YarnController();
YarnConfiguration yarnConfiguration = createYarnConfig("resource-manager", "hdfs://name-node:8020");
List<Property> propertyList = new ArrayList<>();
propertyList.add(new Property("JAVA_HOME", "/abc/bcd").setType(PropertyType.ENV_VAR));
String jvmOptions = yarnController.prepareCommandOptions(yarnConfiguration, propertyList);
logger.info("JVMOptions: {}", jvmOptions);
代码示例来源:origin: dremio/dremio-oss
yarnController.invalidateTwillService(cluster.getId());
stateListener.stopped(cluster);
} catch (ProvisioningHandlingException ex) {
代码示例来源:origin: dremio/dremio-oss
ResourceReport resourceReport = Mockito.mock(ResourceReport.class);
when(controller.startCluster(any(YarnConfiguration.class), any(List.class)))
.thenReturn(twillController);
when(twillController.getRunId()).thenReturn(runId);
when(twillController.getResourceReport()).thenReturn(resourceReport);
when(controller.getTwillService(cluster.getId())).thenReturn(twillRunnerService);
when(twillRunnerService.lookup(DacDaemonYarnApplication.YARN_APPLICATION_NAME_DEFAULT, runId)).thenReturn(twillController);
代码示例来源:origin: dremio/dremio-oss
ResourceReport resourceReport = Mockito.mock(ResourceReport.class);
when(controller.startCluster(any(YarnConfiguration.class), any(List.class)))
.thenReturn(twillController);
when(twillController.getRunId()).thenReturn(runId);
代码示例来源:origin: dremio/dremio-oss
private ClusterEnriched startClusterAsync(Cluster cluster) throws YarnProvisioningHandlingException {
YarnConfiguration yarnConfiguration = new YarnConfiguration();
updateYarnConfiguration(cluster, yarnConfiguration);
List<Property> props = cluster.getClusterConfig().getSubPropertyList();
// only to show those props on UI/API
defaultsConfigurator.getDistroTypeDefaultsConfigurator(
cluster.getClusterConfig().getDistroType(), cluster.getClusterConfig().getIsSecure()).mergeProperties(props);
List<Property> cleansedProperties = new ArrayList<>();
for (Property prop : props) {
if (!EXCLUDED.contains(prop.getKey())) {
cleansedProperties.add(prop);
}
}
// async call - unfortunately I can not add event handlers before start of TwillController
// which means we can miss failures
TwillController twillController = yarnController.startCluster(yarnConfiguration, cleansedProperties);
String runId = twillController.getRunId().getId();
RunId dRunId = new RunId(runId);
cluster.setState(ClusterState.STARTING);
cluster.setRunId(dRunId);
OnRunningRunnable onRunning = new OnRunningRunnable(cluster);
twillController.onRunning(onRunning, Threads.SAME_THREAD_EXECUTOR);
initOnTerminatingThread(cluster, twillController);
return getClusterInfo(cluster);
}
代码示例来源:origin: dremio/dremio-oss
ResourceReport resourceReport = Mockito.mock(ResourceReport.class);
when(controller.startCluster(any(YarnConfiguration.class), any(List.class)))
.thenReturn(twillController);
when(twillController.getRunId()).thenReturn(runId);
代码示例来源:origin: dremio/dremio-oss
ResourceReport resourceReport = Mockito.mock(ResourceReport.class);
when(controller.startCluster(any(YarnConfiguration.class), any(List.class)))
.thenReturn(twillController);
when(twillController.getRunId()).thenReturn(runId);
内容来源于网络,如有侵权,请联系作者删除!