spark,无法获得带有分区的create表,但是没有分区可以工作

z9zf31ra  于 2021-06-26  发布在  Hive
关注(0)|答案(1)|浏览(351)

我的数据结构是这样的

/mnt/path/db/table/keya=01/keyb=123
/mnt/path/db/table/keya=01/keyb=124
/mnt/path/db/table/keya=02/keyb=123

此表创建成功

CREATE EXTERNAL TABLE `test_table_a`(
..irrelevant schema..
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  '/mnt/path/db/table/keya=0101/keyb=123'

然后

select count(*) from test_table_a;
//returns
1876 
//correct

我可以很好地查询数据,但是我想要一个分区表。
我试过这个

CREATE EXTERNAL TABLE `test_table_a`(
..irrelevant schema..
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
PARTITIONED BY (
  `keya` string,
  `keyb` string)
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  '/mnt/path/db/table'

还有这个

CREATE EXTERNAL TABLE `test_table_a`(
..irrelevant schema..
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
PARTITIONED BY (
  `keya` string,
  `keyb` string)
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  '/mnt/path/db/table/keya=*/keyb=*'

然而,在这两方面我都得到了这个结果

select count(*) from test_table_a;
//returns
0 
show partitions test_table_a;
//returns
//nothing
zfciruhq

zfciruhq1#

当您在位置的顶部创建一个外部分区表(数据已经存在于这个位置)时,请在您的配置单元shell中执行下面的命令

hive> msck repair table <db.name>.<table_name>;

检查您是否能够在test\u table\u a table中看到分区信息和数据。
创建表语句:

CREATE EXTERNAL TABLE `test_table_a`(
..irrelevant schema..
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
PARTITIONED BY (
  `keya` string,
  `keyb` string)
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  '/mnt/path/db/table';

相关问题