org.apache.hadoop.hive.metastore.api.Table.getSd()方法的使用及代码示例

x33g5p2x  于2022-01-29 转载在 其他  
字(9.2k)|赞(0)|评价(0)|浏览(343)

本文整理了Java中org.apache.hadoop.hive.metastore.api.Table.getSd()方法的一些代码示例，展示了Table.getSd()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Table.getSd()方法的具体详情如下：
包路径：org.apache.hadoop.hive.metastore.api.Table
类名称：Table
方法名：getSd

Table.getSd介绍

暂无

代码示例

代码示例来源：origin: apache/hive

  1. public String getTableLocation() {
  2.  return table.getSd().getLocation();
  3. }

代码示例来源：origin: apache/hive

  1. public static Path getPath(Table table) {
  2.  String location = table.getSd().getLocation();
  3.  if (location == null) {
  4.   return null;
  5.  }
  6.  return new Path(location);
  7. }

代码示例来源：origin: apache/hive

  1. final public Path getPath() {
  2.  String location = tTable.getSd().getLocation();
  3.  if (location == null) {
  4.   return null;
  5.  }
  6.  return new Path(location);
  7. }

代码示例来源：origin: apache/drill

  1. final public Path getPath() {
  2.  String location = tTable.getSd().getLocation();
  3.  if (location == null) {
  4.   return null;
  5.  }
  6.  return new Path(location);
  7. }

代码示例来源：origin: apache/hive

  1. private Path getQueryWorkingDir(Table table) {
  2.  return new Path(table.getSd().getLocation(), getQueryId());
  3. }

代码示例来源：origin: apache/drill

  1. public Path getDataLocation() {
  2.  if (table.isPartitioned()) {
  3.   return new Path(tPartition.getSd().getLocation());
  4.  } else {
  5.   return new Path(table.getTTable().getSd().getLocation());
  6.  }
  7. }

代码示例来源：origin: apache/hive

  1. public Path getDataLocation() {
  2.  if (table.isPartitioned()) {
  3.   if (tPartition.getSd() == null)
  4.    return null;
  5.   else
  6.    return new Path(tPartition.getSd().getLocation());
  7.  } else {
  8.   if (table.getTTable() == null || table.getTTable().getSd() == null)
  9.    return null;
  10.   else
  11.    return new Path(table.getTTable().getSd().getLocation());
  12.  }
  13. }

代码示例来源：origin: apache/hive

  1. private PartitionHelper newWarehousePartitionHelper() throws MetaException, WorkerException {
  2.  String location = table.getTable().getSd().getLocation();
  3.  Path tablePath = new Path(location);
  4.  List<FieldSchema> partitionFields = table.getTable().getPartitionKeys();
  5.  List<String> partitionColumns = new ArrayList<>(partitionFields.size());
  6.  for (FieldSchema field : partitionFields) {
  7.   partitionColumns.add(field.getName());
  8.  }
  9.  return new WarehousePartitionHelper(configuration, tablePath, partitionColumns);
  10. }

代码示例来源：origin: prestodb/presto

  1. private static List<String> listAllDataPaths(HiveMetastore metastore, String schemaName, String tableName)
  2. {
  3.   ImmutableList.Builder<String> locations = ImmutableList.builder();
  4.   Table table = metastore.getTable(schemaName, tableName).get();
  5.   if (table.getSd().getLocation() != null) {
  6.     // For unpartitioned table, there should be nothing directly under this directory.
  7.     // But including this location in the set makes the directory content assert more
  8.     // extensive, which is desirable.
  9.     locations.add(table.getSd().getLocation());
  10.   }
  11.   Optional<List<String>> partitionNames = metastore.getPartitionNames(schemaName, tableName);
  12.   if (partitionNames.isPresent()) {
  13.     metastore.getPartitionsByNames(schemaName, tableName, partitionNames.get()).stream()
  14.         .map(partition -> partition.getSd().getLocation())
  15.         .filter(location -> !location.startsWith(table.getSd().getLocation()))
  16.         .forEach(locations::add);
  17.   }
  18.   return locations.build();
  19. }

代码示例来源：origin: apache/hive

  1. private static String getPartitionPath(Table table, List<String> partValues) {
  2.  return partValues.get(1).equalsIgnoreCase("isLocatedOutsideTablePath")? // i.e. Is the partition outside the table-dir?
  3.      table.getSd().getLocation().replace(table.getTableName(), "location_outside_" + table.getTableName())
  4.       + "_" + partValues.get(0) + "_" + partValues.get(1)
  5.    : null ; // Use defaults... Partitions are put in the table directory.
  6. }

代码示例来源：origin: apache/hive

  1. private PartitionHelper newMetaStorePartitionHelper() throws MetaException, WorkerException {
  2.  String user = authenticatedUser == null ? System.getProperty("user.name") : authenticatedUser.getShortUserName();
  3.  boolean secureMode = authenticatedUser == null ? false : authenticatedUser.hasKerberosCredentials();
  4.  try {
  5.   IMetaStoreClient metaStoreClient = new UgiMetaStoreClientFactory(metaStoreUri, configuration, authenticatedUser,
  6.     user, secureMode).newInstance(HCatUtil.getHiveMetastoreClient(configuration));
  7.   String tableLocation = table.getTable().getSd().getLocation();
  8.   Path tablePath = new Path(tableLocation);
  9.   return new MetaStorePartitionHelper(metaStoreClient, table.getDatabaseName(), table.getTableName(), tablePath);
  10.  } catch (IOException e) {
  11.   throw new WorkerException("Could not create meta store client.", e);
  12.  }
  13. }

代码示例来源：origin: apache/hive

  1. @Test(expected = MetaException.class)
  2. public void testAddPartitionSpecChangeRootPathDiffInSd() throws Exception {
  3.  Table table = createTable();
  4.  String rootPath = table.getSd().getLocation() + "/addPartSpecRootPath/";
  5.  String rootPath1 = table.getSd().getLocation() + "/addPartSdPath/";
  6.  String rootPath2 = table.getSd().getLocation() + "/someotherpath/";
  7.  Partition partition = buildPartition(DB_NAME, TABLE_NAME, "2007", rootPath1 + "part2007/");
  8.  PartitionSpecProxy partitionSpecProxy =
  9.    buildPartitionSpec(DB_NAME, TABLE_NAME, rootPath, Lists.newArrayList(partition));
  10.  partitionSpecProxy.setRootLocation(rootPath2);
  11.  client.add_partitions_pspec(partitionSpecProxy);
  12. }

代码示例来源：origin: apache/hive

  1. private Path getPartitionLocation() throws NoSuchObjectException, MetaException, TException {
  2.  Path partitionLocacation;
  3.  if (partition.isEmpty()) {
  4.   partitionLocacation = new Path(table.getSd().getLocation());
  5.  } else {
  6.   // TODO: calculate this instead. Just because we're writing to the location doesn't mean that it'll
  7.   // always be wanted in the meta store right away.
  8.   List<Partition> partitionEntries = metaStoreClient.listPartitions(table.getDbName(), table.getTableName(),
  9.     partition, (short) 1);
  10.   partitionLocacation = new Path(partitionEntries.get(0).getSd().getLocation());
  11.  }
  12.  return partitionLocacation;
  13. }

代码示例来源：origin: apache/hive

  1. @Before
  2. public void injectMocks() throws Exception {
  3.  when(mockClient.getTable(DATABASE_NAME, TABLE_NAME)).thenReturn(mockTable);
  4.  when(mockTable.getDbName()).thenReturn(DATABASE_NAME);
  5.  when(mockTable.getTableName()).thenReturn(TABLE_NAME);
  6.  when(mockTable.getPartitionKeys()).thenReturn(PARTITION_KEYS);
  7.  when(mockTable.getSd()).thenReturn(tableStorageDescriptor);
  8.  tableStorageDescriptor.setLocation(TABLE_LOCATION);
  9.  when(mockClient.getPartition(DATABASE_NAME, TABLE_NAME, PARTITIONED_VALUES)).thenReturn(mockPartition);
  10.  when(mockPartition.getSd()).thenReturn(mockPartitionStorageDescriptor);
  11.  when(mockPartitionStorageDescriptor.getLocation()).thenReturn(PARTITION_LOCATION);
  12.  helper = new MetaStorePartitionHelper(mockClient, DATABASE_NAME, TABLE_NAME, TABLE_PATH);
  13. }

代码示例来源：origin: apache/hive

  1. @Test(expected = MetaException.class)
  2. public void testAddPartitionSpecChangeRootPathFromNull() throws Exception {
  3.  Table table = createTable();
  4.  String rootPath = table.getSd().getLocation() + "/addPartSpecRootPath/";
  5.  String rootPath1 = table.getSd().getLocation() + "/someotherpath/";
  6.  Partition partition = buildPartition(DB_NAME, TABLE_NAME, "2007", rootPath + "part2007/");
  7.  PartitionSpecProxy partitionSpecProxy =
  8.    buildPartitionSpec(DB_NAME, TABLE_NAME, null, Lists.newArrayList(partition));
  9.  partitionSpecProxy.setRootLocation(rootPath1);
  10.  client.add_partitions_pspec(partitionSpecProxy);
  11. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testAddPartitionSpecSetRootPath() throws Exception {
  3.  Table table = createTable();
  4.  String rootPath = table.getSd().getLocation() + "/addPartSpecRootPath/";
  5.  String rootPath1 = table.getSd().getLocation() + "/someotherpath/";
  6.  Partition partition = buildPartition(DB_NAME, TABLE_NAME, "2007", rootPath + "part2007/");
  7.  PartitionSpecProxy partitionSpecProxy =
  8.    buildPartitionSpec(DB_NAME, TABLE_NAME, rootPath1, Lists.newArrayList(partition));
  9.  client.add_partitions_pspec(partitionSpecProxy);
  10.  Partition resultPart = client.getPartition(DB_NAME, TABLE_NAME, Lists.newArrayList("2007"));
  11.  Assert.assertEquals(rootPath + "part2007", resultPart.getSd().getLocation());
  12. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testDropTableExternalWithoutPurge() throws Exception {
  3.  Table table = externalTable;
  4.  client.dropTable(table.getDbName(), table.getTableName(), true, true, false);
  5.  Assert.assertTrue("Table path should not be removed",
  6.    metaStore.isPathExists(new Path(table.getSd().getLocation())));
  7.  Assert.assertFalse("Table path should be in trash",
  8.    metaStore.isPathExistsInTrash(new Path(table.getSd().getLocation())));
  9. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testDropTableWithPurge() throws Exception {
  3.  Table table = testTables[0];
  4.  client.dropTable(table.getDbName(), table.getTableName(), true, true, true);
  5.  Assert.assertFalse("Table path should be removed",
  6.    metaStore.isPathExists(new Path(table.getSd().getLocation())));
  7.  Assert.assertFalse("Table path should not be in trash",
  8.    metaStore.isPathExistsInTrash(new Path(table.getSd().getLocation())));
  9. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testDropTableExternalWithPurge() throws Exception {
  3.  Table table = externalTable;
  4.  client.dropTable(table.getDbName(), table.getTableName(), true, true, true);
  5.  Assert.assertTrue("Table path should not be removed",
  6.    metaStore.isPathExists(new Path(table.getSd().getLocation())));
  7.  Assert.assertFalse("Table path should not be in trash",
  8.    metaStore.isPathExistsInTrash(new Path(table.getSd().getLocation())));
  9. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testDropTableWithoutPurge() throws Exception {
  3.  Table table = testTables[0];
  4.  client.dropTable(table.getDbName(), table.getTableName(), true, true, false);
  5.  Assert.assertFalse("Table path should be removed",
  6.    metaStore.isPathExists(new Path(table.getSd().getLocation())));
  7.  Assert.assertTrue("Table path should be in trash",
  8.    metaStore.isPathExistsInTrash(new Path(table.getSd().getLocation())));
  9. }

内容来源于网络，如有侵权，请联系作者删除！

Table

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Table类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com