如何使用hcatalogjavaapi获取行和字段分隔符

fcipmucu  于 2021-06-28  发布在  Hive
关注(0)|答案(1)|浏览(521)

pom相关性:

<dependency>
        <groupId>org.apache.hive.hcatalog</groupId>
        <artifactId>hive-webhcat-java-client</artifactId>
        <version>1.2.1</version>
    </dependency>

我能够获得列、分区列、输入文件格式等。
有用代码:

HiveConf hcatConf = new HiveConf();

    hcatConf.setVar(HiveConf.ConfVars.METASTOREURIS, connectionUri);
    hcatConf.set("hive.metastore.local", "false");
    hcatConf.setIntVar(HiveConf.ConfVars.METASTORETHRIFTCONNECTIONRETRIES, THRIFT_CONNECTION_RETRY);
    hcatConf.set(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
    hcatConf.set(HiveConf.ConfVars.SEMANTIC_ANALYZER_HOOK.varname, HCatSemanticAnalyzer.class.getName());
    hcatConf.set(HiveConf.ConfVars.PREEXECHOOKS.varname, "");
    hcatConf.set(HiveConf.ConfVars.POSTEXECHOOKS.varname, "");

    hcatConf.setTimeVar(HiveConf.ConfVars.METASTORE_CLIENT_SOCKET_TIMEOUT, TIME_OUT, TimeUnit.MILLISECONDS);

    HCatClient client = null;
    HCatTable hTable = null;

    try {
        client = HCatClient.create(hcatConf);
        hTable = client.getTable(databaseName, tableName);
        System.out.println(hTable.getInputFileFormat());
        System.out.println(hTable.getOutputFileFormat());
        System.out.println(hTable.getSerdeLib());

    } catch (HCatException hCatEx) {
        LOG.error("Not able to connect to hive. Caused By;", hCatEx);
    }

如何获取文本表的行和字段分隔符?
根据getserdeparams()的javadoc, public Map<String,String> getSerdeParams() -返回字段分隔符等参数。
但就我而言,这张Map上只有一个条目

{serialization.format=1}
flvlnr44

flvlnr441#

如果我创建一个表:

create table tbl1 (c1 int)  stored as textfile

当我跑的时候 show create table tbl1 :

CREATE TABLE `tbl1`(
  `c1` int)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://localhost:8020/apps/hive/warehouse/dev.db/tbl1'
TBLPROPERTIES (
  'transient_lastDdlTime'='1477067078')

不显示默认分隔符。
创建带分隔符的表时:

create table tbl2 (c1 int)  ROW FORMAT DELIMITED FIELDS TERMINATED BY "\," LINES TERMINATED BY "\n" stored as textfile;

当我跑的时候 show create table tbl2 :

CREATE TABLE `tbl2`(
  `c1` int)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY ',' 
  LINES TERMINATED BY '\n' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://localhost:8020/apps/hive/warehouse/dev.db/tbl2'
TBLPROPERTIES (
  'transient_lastDdlTime'='1477067160')

在第二种情况下,我明确提到了分隔符。所以, getSerdeParams() 返回所需的值。

相关问题