org.apache.hadoop.hive.metastore.api.Table.setParameters()方法的使用及代码示例

x33g5p2x  于2022-01-29 转载在 其他  
字(12.8k)|赞(0)|评价(0)|浏览(205)

本文整理了Java中org.apache.hadoop.hive.metastore.api.Table.setParameters()方法的一些代码示例，展示了Table.setParameters()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Table.setParameters()方法的具体详情如下：
包路径：org.apache.hadoop.hive.metastore.api.Table
类名称：Table
方法名：setParameters

Table.setParameters介绍

暂无

代码示例

代码示例来源：origin: apache/hive

  1. public void setParameters(Map<String, String> params) {
  2.  tTable.setParameters(params);
  3. }

代码示例来源：origin: apache/drill

  1. public void setParameters(Map<String, String> params) {
  2.  tTable.setParameters(params);
  3. }

代码示例来源：origin: prestodb/presto

  1. @Override
  2. public synchronized void updateTableStatistics(String databaseName, String tableName, Function<PartitionStatistics, PartitionStatistics> update)
  3. {
  4.   PartitionStatistics currentStatistics = getTableStatistics(databaseName, tableName);
  5.   PartitionStatistics updatedStatistics = update.apply(currentStatistics);
  6.   Table originalTable = getTable(databaseName, tableName)
  7.       .orElseThrow(() -> new TableNotFoundException(new SchemaTableName(databaseName, tableName)));
  8.   Table modifiedTable = originalTable.deepCopy();
  9.   HiveBasicStatistics basicStatistics = updatedStatistics.getBasicStatistics();
  10.   modifiedTable.setParameters(updateStatisticsParameters(modifiedTable.getParameters(), basicStatistics));
  11.   alterTable(databaseName, tableName, modifiedTable);
  12.   com.facebook.presto.hive.metastore.Table table = fromMetastoreApiTable(modifiedTable);
  13.   OptionalLong rowCount = basicStatistics.getRowCount();
  14.   List<ColumnStatisticsObj> metastoreColumnStatistics = updatedStatistics.getColumnStatistics().entrySet().stream()
  15.       .map(entry -> createMetastoreColumnStatistics(entry.getKey(), table.getColumn(entry.getKey()).get().getType(), entry.getValue(), rowCount))
  16.       .collect(toImmutableList());
  17.   if (!metastoreColumnStatistics.isEmpty()) {
  18.     setTableColumnStatistics(databaseName, tableName, metastoreColumnStatistics);
  19.   }
  20.   Set<String> removedColumnStatistics = difference(currentStatistics.getColumnStatistics().keySet(), updatedStatistics.getColumnStatistics().keySet());
  21.   removedColumnStatistics.forEach(column -> deleteTableColumnStatistics(databaseName, tableName, column));
  22. }

代码示例来源：origin: prestodb/presto

  1. public static org.apache.hadoop.hive.metastore.api.Table toMetastoreApiTable(Table table, PrincipalPrivileges privileges)
  2. {
  3.   org.apache.hadoop.hive.metastore.api.Table result = new org.apache.hadoop.hive.metastore.api.Table();
  4.   result.setDbName(table.getDatabaseName());
  5.   result.setTableName(table.getTableName());
  6.   result.setOwner(table.getOwner());
  7.   result.setTableType(table.getTableType());
  8.   result.setParameters(table.getParameters());
  9.   result.setPartitionKeys(table.getPartitionColumns().stream().map(ThriftMetastoreUtil::toMetastoreApiFieldSchema).collect(toList()));
  10.   result.setSd(makeStorageDescriptor(table.getTableName(), table.getDataColumns(), table.getStorage()));
  11.   result.setPrivileges(toMetastoreApiPrincipalPrivilegeSet(table.getOwner(), privileges));
  12.   result.setViewOriginalText(table.getViewOriginalText().orElse(null));
  13.   result.setViewExpandedText(table.getViewExpandedText().orElse(null));
  14.   return result;
  15. }

代码示例来源：origin: apache/incubator-gobblin

  1. table.setDbName(hiveTable.getDbName());
  2. table.setTableName(hiveTable.getTableName());
  3. table.setParameters(getParameters(props));
  4. if (hiveTable.getCreateTime().isPresent()) {
  5.  table.setCreateTime(Ints.checkedCast(hiveTable.getCreateTime().get()));

代码示例来源：origin: apache/hive

  1. public TableBuilder(Database database) {
  2.  this.database = database;
  3.  partitions = new ArrayList<>();
  4.  columnNames = new ArrayList<>();
  5.  columnTypes = new ArrayList<>();
  6.  partitionKeys = Collections.emptyList();
  7.  table = new Table();
  8.  table.setDbName(database.getName());
  9.  table.setTableType(TableType.MANAGED_TABLE.toString());
  10.  Map<String, String> tableParams = new HashMap<String, String>();
  11.  tableParams.put("transactional", Boolean.TRUE.toString());
  12.  table.setParameters(tableParams);
  13.  sd = new StorageDescriptor();
  14.  sd.setInputFormat(OrcInputFormat.class.getName());
  15.  sd.setOutputFormat(OrcOutputFormat.class.getName());
  16.  sd.setNumBuckets(1);
  17.  table.setSd(sd);
  18.  serDeInfo = new SerDeInfo();
  19.  serDeInfo.setParameters(new HashMap<String, String>());
  20.  serDeInfo.getParameters().put(serdeConstants.SERIALIZATION_FORMAT, "1");
  21.  serDeInfo.setSerializationLib(OrcSerde.class.getName());
  22.  sd.setSerdeInfo(serDeInfo);
  23. }

代码示例来源：origin: apache/hive

  1. newTable.setTableName(tableName);
  2. if (tblProps != null) {
  3.  newTable.setParameters(tblProps);

代码示例来源：origin: apache/hive

  1. TransactionalValidationListener.INSERTONLY_TRANSACTIONAL_PROPERTY);
  2. table.setParameters(parameters);
  3. if (isTemporary) table.setTemporary(true);

代码示例来源：origin: apache/hive

  1. tbl.setParameters(
  2.   adjustStatsParamsForGet(tbl.getParameters(), tbl.getParameters(), tbl.getWriteId(), validWriteIds));

代码示例来源：origin: apache/hive

  1. tableParams.put("hcat.testarg", "testArgValue");
  2. tbl.setParameters(tableParams);

代码示例来源：origin: apache/hive

  1. Table build() {
  2.   StorageDescriptor sd = new StorageDescriptor();
  3.   if (columns == null) {
  4.    sd.setCols(Collections.emptyList());
  5.   } else {
  6.    sd.setCols(columns);
  7.   }
  8.   SerDeInfo serdeInfo = new SerDeInfo();
  9.   serdeInfo.setSerializationLib(serde);
  10.   serdeInfo.setName(tableName);
  11.   sd.setSerdeInfo(serdeInfo);
  12.   sd.setInputFormat(inputFormat);
  13.   sd.setOutputFormat(outputFormat);
  14.   if (location != null) {
  15.    sd.setLocation(location);
  16.   }
  17.   Table table = new Table();
  18.   table.setDbName(dbName);
  19.   table.setTableName(tableName);
  20.   table.setSd(sd);
  21.   table.setParameters(parameters);
  22.   table.setOwner(owner);
  23.   if (partitionKeys != null) {
  24.    table.setPartitionKeys(partitionKeys);
  25.   }
  26.   table.setTableType(tableType.toString());
  27.   return table;
  28.  }
  29. }

代码示例来源：origin: apache/hive

  1. private void createTable(String dbName, String tableName) throws Exception {
  2.  String databaseName = (dbName == null) ? Warehouse.DEFAULT_DATABASE_NAME
  3.    : dbName;
  4.  try {
  5.   msc.dropTable(databaseName, tableName);
  6.  } catch (Exception e) {
  7.  } // can fail with NoSuchObjectException
  8.  Table tbl = new Table();
  9.  tbl.setDbName(databaseName);
  10.  tbl.setTableName(tableName);
  11.  tbl.setTableType("MANAGED_TABLE");
  12.  StorageDescriptor sd = new StorageDescriptor();
  13.  sd.setCols(getTableColumns());
  14.  tbl.setPartitionKeys(getPartitionKeys());
  15.  tbl.setSd(sd);
  16.  sd.setBucketCols(new ArrayList<String>(2));
  17.  sd.setSerdeInfo(new SerDeInfo());
  18.  sd.getSerdeInfo().setName(tbl.getTableName());
  19.  sd.getSerdeInfo().setParameters(new HashMap<String, String>());
  20.  sd.getSerdeInfo().getParameters().put(serdeConstants.SERIALIZATION_FORMAT, "1");
  21.  sd.getSerdeInfo().setSerializationLib(ColumnarSerDe.class.getName());
  22.  sd.setInputFormat(RCFileInputFormat.class.getName());
  23.  sd.setOutputFormat(RCFileOutputFormat.class.getName());
  24.  Map<String, String> tableParams = new HashMap<String, String>();
  25.  tbl.setParameters(tableParams);
  26.  msc.createTable(tbl);
  27. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testAlterTableChangeCols() throws Exception {
  3.  Table originalTable = partitionedTable;
  4.  Table newTable = originalTable.deepCopy();
  5.  List<FieldSchema> cols = newTable.getSd().getCols();
  6.  // Change a column
  7.  cols.get(0).setName("modified_col");
  8.  // Remove a column
  9.  cols.remove(1);
  10.  // Add a new column
  11.  cols.add(new FieldSchema("new_col", "int", null));
  12.  // Store the changes
  13.  client.alter_table(originalTable.getDbName(), originalTable.getTableName(), newTable);
  14.  Table alteredTable = client.getTable(newTable.getDbName(), newTable.getTableName());
  15.  Assert.assertTrue("Original table directory should be kept",
  16.    metaStore.isPathExists(new Path(originalTable.getSd().getLocation())));
  17.  // The following data might be changed
  18.  alteredTable.setParameters(newTable.getParameters());
  19.  Assert.assertEquals("The table data should be the same", newTable, alteredTable);
  20.  // Modify partition column type, and comment
  21.  newTable.getPartitionKeys().get(0).setType("string");
  22.  newTable.getPartitionKeys().get(0).setComment("changed comment");
  23.  client.alter_table(originalTable.getDbName(), originalTable.getTableName(), newTable);
  24.  alteredTable = client.getTable(newTable.getDbName(), newTable.getTableName());
  25.  // The following data might be changed
  26.  alteredTable.setParameters(newTable.getParameters());
  27.  Assert.assertEquals("The table data should be the same", newTable, alteredTable);
  28. }

代码示例来源：origin: apache/hive

  1. StorageDescriptor sd = new StorageDescriptor(oldtbl.getSd());
  2. newTable.setSd(sd);
  3. newTable.setParameters(oldtbl.getParameters());
  4. if (location == null) {
  5.  newTable.getSd().setLocation(oldtbl.getSd().getLocation());

代码示例来源：origin: apache/storm

  1. tbl.setParameters(tableParams);
  2. client.createTable(tbl);
  3. try {

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testAlterTableExternalTable() throws Exception {
  3.  Table originalTable = externalTable;
  4.  String originalTableName = originalTable.getTableName();
  5.  String originalDatabase = originalTable.getDbName();
  6.  Table newTable = originalTable.deepCopy();
  7.  newTable.setTableName("new_external_table_for_test");
  8.  client.alter_table(originalDatabase, originalTableName, newTable);
  9.  List<String> tableNames = client.getTables(originalDatabase, originalTableName);
  10.  Assert.assertEquals("Original table should be removed", 0, tableNames.size());
  11.  Assert.assertTrue("Original table directory should be kept",
  12.    metaStore.isPathExists(new Path(originalTable.getSd().getLocation())));
  13.  Table alteredTable = client.getTable(newTable.getDbName(), newTable.getTableName());
  14.  Assert.assertEquals("New location should be the same", originalTable.getSd().getLocation(),
  15.    alteredTable.getSd().getLocation());
  16.  Path dataFile = new Path(alteredTable.getSd().getLocation() + "/dataFile");
  17.  Assert.assertTrue("The location should contain data", metaStore.isPathExists(dataFile));
  18.  // The extra parameters will be added on server side, so check that the required ones are
  19.  // present
  20.  for(String key: newTable.getParameters().keySet()) {
  21.   Assert.assertEquals("parameters are present", newTable.getParameters().get(key),
  22.     alteredTable.getParameters().get(key));
  23.  }
  24.  // The parameters are checked manually, so do not check them
  25.  newTable.setParameters(alteredTable.getParameters());
  26.  Assert.assertEquals("The table data should be the same", newTable, alteredTable);
  27. }

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testAlterTableExternalTableChangeLocation() throws Exception {
  3.  Table originalTable = externalTable;
  4.  // Change the location, and see the results
  5.  Table newTable = originalTable.deepCopy();
  6.  newTable.getSd().setLocation(newTable.getSd().getLocation() + "_modified");
  7.  client.alter_table(originalTable.getDbName(), originalTable.getTableName(), newTable);
  8.  Table alteredTable = client.getTable(newTable.getDbName(), newTable.getTableName());
  9.  Assert.assertTrue("Original table directory should be kept",
  10.    metaStore.isPathExists(new Path(originalTable.getSd().getLocation())));
  11.  Assert.assertEquals("New location should be the new one", newTable.getSd().getLocation(),
  12.    alteredTable.getSd().getLocation());
  13.  Path dataFile = new Path(alteredTable.getSd().getLocation() + "/dataFile");
  14.  Assert.assertFalse("The location should not contain data", metaStore.isPathExists(dataFile));
  15.  // The extra parameters will be added on server side, so check that the required ones are
  16.  // present
  17.  for(String key: newTable.getParameters().keySet()) {
  18.   Assert.assertEquals("parameters are present", newTable.getParameters().get(key),
  19.     alteredTable.getParameters().get(key));
  20.  }
  21.  // The parameters are checked manually, so do not check them
  22.  newTable.setParameters(alteredTable.getParameters());
  23.  // The following data should be changed, other data should be the same
  24.  newTable.getSd().setLocation(alteredTable.getSd().getLocation());
  25.  Assert.assertEquals("The table data should be the same", newTable, alteredTable);
  26. }

代码示例来源：origin: apache/hive

  1. table.setParameters(newParams);
  2. sharedCache.alterTableInCache(catName, dbName, tblName, table);
  3. sharedCache.updateTableColStatsInCache(catName, dbName, tblName, colStats.getStatsObj());

代码示例来源：origin: apache/hive

  1. @Test
  2. public void testAlterTable() throws Exception {
  3.  Table originalTable = testTables[2];
  4.  String originalTableName = originalTable.getTableName();
  5.  String originalDatabase = originalTable.getDbName();
  6.  Table newTable = getTableWithAllParametersSet();
  7.  newTable.setTableName(originalTableName);
  8.  newTable.setDbName(originalDatabase);
  9.  // Partition keys can not be set, but getTableWithAllParametersSet is added one, so remove for
  10.  // this test
  11.  newTable.setPartitionKeys(originalTable.getPartitionKeys());
  12.  client.alter_table(originalDatabase, originalTableName, newTable);
  13.  Table alteredTable = client.getTable(originalDatabase, originalTableName);
  14.  // The extra parameters will be added on server side, so check that the required ones are
  15.  // present
  16.  for(String key: newTable.getParameters().keySet()) {
  17.   Assert.assertEquals("parameters are present", newTable.getParameters().get(key),
  18.     alteredTable.getParameters().get(key));
  19.  }
  20.  // The parameters are checked manually, so do not check them
  21.  newTable.setParameters(alteredTable.getParameters());
  22.  // Some of the data is set on the server side, so reset those
  23.  newTable.setCreateTime(alteredTable.getCreateTime());
  24.  newTable.setCreationMetadata(alteredTable.getCreationMetadata());
  25.  newTable.setWriteId(alteredTable.getWriteId());
  26.  Assert.assertTrue(alteredTable.isSetId());
  27.  alteredTable.unsetId();
  28.  Assert.assertEquals("The table data should be the same", newTable, alteredTable);
  29. }

代码示例来源：origin: apache/hive

  1. table.setParameters(createdTable.getParameters());
  2. table.setCreationMetadata(createdTable.getCreationMetadata());
  3. table.setWriteId(createdTable.getWriteId());

内容来源于网络，如有侵权，请联系作者删除！

Table

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Table类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com