从s3 bucket创建avro配置单元表

mtb9vblg  于 2021-06-29  发布在  Hive
关注(0)|答案(0)|浏览(253)
I have a S3 bucket which contains data in day wise partitions. Each of the partition contains a days worth of data in Avro format .I need to load this data into a hive table.
    How can I create a partitioned Avro hive table to store data for last 90 days.

    I tried the below create table command
   table creation command

    hive> CREATE EXTERNAL TABLE fxgm_avro
        >   PARTITIONED BY (dt string)
        >   ROW FORMAT SERDE
        >   'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
        >   STORED AS INPUTFORMAT
        >   'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
        >   OUTPUTFORMAT
        >   'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
        >   LOCATION 's3://glbl_mkts/fxgm/bkt%3Dkinesis_fxgm_logs/format%3Davro/dt%3D2016-01-12/part-r-00000.avro'
        >   TBLPROPERTIES (
        >     'avro.schema.url'='file://fxgm_avr/fxgm.avsc');
    OK
    Time taken: 3.535 seconds

从表中选择时,不选择任何记录hive>select*from fxgm\u avro limit 10;正常时间:0.473秒

When I tried to select from the table, no records is selected.
    I wonder this is the correct way or something I am missing.

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题