org.apache.hadoop.hive.ql.metadata.Hive.getPartitionNames()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(11.3k)|赞(0)|评价(0)|浏览(257)

本文整理了Java中org.apache.hadoop.hive.ql.metadata.Hive.getPartitionNames()方法的一些代码示例，展示了Hive.getPartitionNames()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Hive.getPartitionNames()方法的具体详情如下：
包路径：org.apache.hadoop.hive.ql.metadata.Hive
类名称：Hive
方法名：getPartitionNames

Hive.getPartitionNames介绍

暂无

代码示例

代码示例来源：origin: apache/hive

public List<String> getPartitionNames(String tblName, short max) throws HiveException {
 String[] names = Utilities.getDbTableName(tblName);
 return getPartitionNames(names[0], names[1], max);
}

代码示例来源：origin: apache/drill

public List<String> getPartitionNames(String tblName, short max) throws HiveException {
 String[] names = Utilities.getDbTableName(tblName);
 return getPartitionNames(names[0], names[1], max);
}

代码示例来源：origin: apache/hive

/**
 * Primary constructor that fetches all partitions in a given table, given
 * a Hive object and a table object, and a partial partition spec.
 */
public PartitionIterable(Hive db, Table table, Map<String, String> partialPartitionSpec,
             int batch_size, boolean getColStats) throws HiveException {
 this.currType = Type.LAZY_FETCH_PARTITIONS;
 this.db = db;
 this.table = table;
 this.partialPartitionSpec = partialPartitionSpec;
 this.batch_size = batch_size;
 this.getColStats = getColStats;
 if (this.partialPartitionSpec == null){
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(), (short) -1);
 } else {
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(),partialPartitionSpec,(short)-1);
 }
}

代码示例来源：origin: apache/drill

/**
 * Primary constructor that fetches all partitions in a given table, given
 * a Hive object and a table object, and a partial partition spec.
 */
public PartitionIterable(Hive db, Table table, Map<String, String> partialPartitionSpec,
  int batch_size) throws HiveException {
 this.currType = Type.LAZY_FETCH_PARTITIONS;
 this.db = db;
 this.table = table;
 this.partialPartitionSpec = partialPartitionSpec;
 this.batch_size = batch_size;
 if (this.partialPartitionSpec == null){
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(), (short) -1);
 } else {
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(),partialPartitionSpec,(short)-1);
 }
}

代码示例来源：origin: apache/hive

/**
 * get all the partitions of the table that matches the given partial
 * specification. partition columns whose value is can be anything should be
 * an empty string.
 *
 * @param tbl
 *          object for which partition is needed. Must be partitioned.
 * @param partialPartSpec
 *          partial partition specification (some subpartitions can be empty).
 * @return list of partition objects
 * @throws HiveException
 */
public List<Partition> getPartitionsByNames(Table tbl,
  Map<String, String> partialPartSpec)
  throws HiveException {
 if (!tbl.isPartitioned()) {
  throw new HiveException(ErrorMsg.TABLE_NOT_PARTITIONED, tbl.getTableName());
 }
 List<String> names = getPartitionNames(tbl.getDbName(), tbl.getTableName(),
   partialPartSpec, (short)-1);
 List<Partition> partitions = getPartitionsByNames(tbl, names);
 return partitions;
}

代码示例来源：origin: apache/drill

/**
 * get all the partitions of the table that matches the given partial
 * specification. partition columns whose value is can be anything should be
 * an empty string.
 *
 * @param tbl
 *          object for which partition is needed. Must be partitioned.
 * @param partialPartSpec
 *          partial partition specification (some subpartitions can be empty).
 * @return list of partition objects
 * @throws HiveException
 */
public List<Partition> getPartitionsByNames(Table tbl,
  Map<String, String> partialPartSpec)
  throws HiveException {
 if (!tbl.isPartitioned()) {
  throw new HiveException(ErrorMsg.TABLE_NOT_PARTITIONED, tbl.getTableName());
 }
 List<String> names = getPartitionNames(tbl.getDbName(), tbl.getTableName(),
   partialPartSpec, (short)-1);
 List<Partition> partitions = getPartitionsByNames(tbl, names);
 return partitions;
}

代码示例来源：origin: apache/hive

/**
 * Pruning partition by getting the partition names first and pruning using Hive expression
 * evaluator on client.
 * @param tab the table containing the partitions.
 * @param partitions the resulting partitions.
 * @param prunerExpr the SQL predicate that involves partition columns.
 * @param conf Hive Configuration object, can not be NULL.
 * @return true iff the partition pruning expression contains non-partition columns.
 */
static private boolean pruneBySequentialScan(Table tab, List<Partition> partitions,
  ExprNodeGenericFuncDesc prunerExpr, HiveConf conf) throws HiveException, MetaException {
 PerfLogger perfLogger = SessionState.getPerfLogger();
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 List<String> partNames = Hive.get().getPartitionNames(
   tab.getDbName(), tab.getTableName(), (short) -1);
 String defaultPartitionName = conf.getVar(HiveConf.ConfVars.DEFAULTPARTITIONNAME);
 List<String> partCols = extractPartColNames(tab);
 List<PrimitiveTypeInfo> partColTypeInfos = extractPartColTypes(tab);
 boolean hasUnknownPartitions = prunePartitionNames(
   partCols, partColTypeInfos, prunerExpr, defaultPartitionName, partNames);
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 if (!partNames.isEmpty()) {
  partitions.addAll(Hive.get().getPartitionsByNames(tab, partNames));
 }
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 return hasUnknownPartitions;
}

代码示例来源：origin: apache/drill

} else {
 cols = Hive.getFieldsFromDeserializer(colPath, deserializer);
 List<String> parts = db.getPartitionNames(dbTab[0].toLowerCase(), dbTab[1].toLowerCase(), (short) -1);
 AggrStats aggrStats = db.getAggrColStatsFor(dbTab[0].toLowerCase(), dbTab[1].toLowerCase(), colNames, parts);
 colStats = aggrStats.getColStats();

代码示例来源：origin: apache/drill

/**
 * Pruning partition by getting the partition names first and pruning using Hive expression
 * evaluator on client.
 * @param tab the table containing the partitions.
 * @param partitions the resulting partitions.
 * @param prunerExpr the SQL predicate that involves partition columns.
 * @param conf Hive Configuration object, can not be NULL.
 * @return true iff the partition pruning expression contains non-partition columns.
 */
static private boolean pruneBySequentialScan(Table tab, List<Partition> partitions,
  ExprNodeGenericFuncDesc prunerExpr, HiveConf conf) throws HiveException, MetaException {
 PerfLogger perfLogger = SessionState.getPerfLogger();
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 List<String> partNames = Hive.get().getPartitionNames(
   tab.getDbName(), tab.getTableName(), (short) -1);
 String defaultPartitionName = conf.getVar(HiveConf.ConfVars.DEFAULTPARTITIONNAME);
 List<String> partCols = extractPartColNames(tab);
 List<PrimitiveTypeInfo> partColTypeInfos = extractPartColTypes(tab);
 boolean hasUnknownPartitions = prunePartitionNames(
   partCols, partColTypeInfos, prunerExpr, defaultPartitionName, partNames);
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 if (!partNames.isEmpty()) {
  partitions.addAll(Hive.get().getPartitionsByNames(tab, partNames));
 }
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 return hasUnknownPartitions;
}

代码示例来源：origin: apache/hive

parts = db.getPartitionNames(tbl.getDbName(),
   tbl.getTableName(), showParts.getPartSpec(), (short) -1);
} else {
 parts = db.getPartitionNames(tbl.getDbName(), tbl.getTableName(), (short) -1);

代码示例来源：origin: apache/hive

} else {
 cols = Hive.getFieldsFromDeserializer(colPath, deserializer);
 List<String> parts = db.getPartitionNames(dbTab[0].toLowerCase(), dbTab[1].toLowerCase(), (short) -1);
 AggrStats aggrStats = db.getAggrColStatsFor(
   dbTab[0].toLowerCase(), dbTab[1].toLowerCase(), colNames, parts, false);

代码示例来源：origin: apache/drill

parts = db.getPartitionNames(tbl.getDbName(),
   tbl.getTableName(), showParts.getPartSpec(), (short) -1);
} else {
 parts = db.getPartitionNames(tbl.getDbName(), tbl.getTableName(), (short) -1);

代码示例来源：origin: apache/hive

? null : getPartitionNames(table.getDbName(), table.getTableName(), partSpec, (short) -1));
if (snapshot == null) {
 getMSC().truncateTable(table.getDbName(), table.getTableName(), partNames);

代码示例来源：origin: com.facebook.presto.hive/hive-apache

public List<String> getPartitionNames(String tblName, short max) throws HiveException {
 String[] names = Utilities.getDbTableName(tblName);
 return getPartitionNames(names[0], names[1], max);
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

/**
 * Primary constructor that fetches all partitions in a given table, given
 * a Hive object and a table object, and a partial partition spec.
 */
public PartitionIterable(Hive db, Table table, Map<String, String> partialPartitionSpec,
  int batch_size) throws HiveException {
 this.currType = Type.LAZY_FETCH_PARTITIONS;
 this.db = db;
 this.table = table;
 this.partialPartitionSpec = partialPartitionSpec;
 this.batch_size = batch_size;
 if (this.partialPartitionSpec == null){
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(), (short) -1);
 } else {
  partitionNames = db.getPartitionNames(
    table.getDbName(),table.getTableName(),partialPartitionSpec,(short)-1);
 }
}

代码示例来源：origin: org.apache.hadoop.hive/hive-exec

public List<String> getPartitionNames(String tblName, short max) throws HiveException {
 Table t = newTable(tblName);
 return getPartitionNames(t.getDbName(), t.getTableName(), max);
}

代码示例来源：origin: org.apache.hadoop.hive/hive-exec

parts = db.getPartitionNames(tbl.getDbName(),
   tbl.getTableName(), showParts.getPartSpec(), (short) -1);
} else {
 parts = db.getPartitionNames(tbl.getDbName(), tbl.getTableName(), (short) -1);

代码示例来源：origin: com.facebook.presto.hive/hive-apache

/**
 * get all the partitions of the table that matches the given partial
 * specification. partition columns whose value is can be anything should be
 * an empty string.
 *
 * @param tbl
 *          object for which partition is needed. Must be partitioned.
 * @param partialPartSpec
 *          partial partition specification (some subpartitions can be empty).
 * @return list of partition objects
 * @throws HiveException
 */
public List<Partition> getPartitionsByNames(Table tbl,
  Map<String, String> partialPartSpec)
  throws HiveException {
 if (!tbl.isPartitioned()) {
  throw new HiveException(ErrorMsg.TABLE_NOT_PARTITIONED, tbl.getTableName());
 }
 List<String> names = getPartitionNames(tbl.getDbName(), tbl.getTableName(),
   partialPartSpec, (short)-1);
 List<Partition> partitions = getPartitionsByNames(tbl, names);
 return partitions;
}

代码示例来源：origin: org.apache.hadoop.hive/hive-exec

List<String> names = getPartitionNames(tbl.getDbName(), tbl.getTableName(),
  partialPartSpec, (short)-1);

代码示例来源：origin: com.facebook.presto.hive/hive-apache

/**
 * Pruning partition by getting the partition names first and pruning using Hive expression
 * evaluator on client.
 * @param tab the table containing the partitions.
 * @param partitions the resulting partitions.
 * @param prunerExpr the SQL predicate that involves partition columns.
 * @param conf Hive Configuration object, can not be NULL.
 * @return true iff the partition pruning expression contains non-partition columns.
 */
static private boolean pruneBySequentialScan(Table tab, List<Partition> partitions,
  ExprNodeGenericFuncDesc prunerExpr, HiveConf conf) throws HiveException, MetaException {
 PerfLogger perfLogger = PerfLogger.getPerfLogger();
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 List<String> partNames = Hive.get().getPartitionNames(
   tab.getDbName(), tab.getTableName(), (short) -1);
 String defaultPartitionName = conf.getVar(HiveConf.ConfVars.DEFAULTPARTITIONNAME);
 List<String> partCols = extractPartColNames(tab);
 List<PrimitiveTypeInfo> partColTypeInfos = extractPartColTypes(tab);
 boolean hasUnknownPartitions = prunePartitionNames(
   partCols, partColTypeInfos, prunerExpr, defaultPartitionName, partNames);
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PRUNE_LISTING);
 perfLogger.PerfLogBegin(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 if (!partNames.isEmpty()) {
  partitions.addAll(Hive.get().getPartitionsByNames(tab, partNames));
 }
 perfLogger.PerfLogEnd(CLASS_NAME, PerfLogger.PARTITION_RETRIEVING);
 return hasUnknownPartitions;
}

内容来源于网络，如有侵权，请联系作者删除！

Hive

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Hive类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com