如何在avro模式中指定时间戳

nwsw7zdq  于 2021-06-26  发布在  Hive
关注(0)|答案(2)|浏览(337)

我正在尝试转换此值 1520200800049Avro . 这是我的模式中字段的规范,它是一个 union 顺便说一下:

{
 "name" : "TimestampField"
 "type" : ["null",{
    "type":"long",
    "logicalType":"timestamp-millis"
 }]
}

这是我得到的错误:

Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["null",{"type":"long","logicalType":"timestamp-millis"}]: 1520200800049 
    at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:740) 
    at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:205) 
    at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:123) 
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75) 
    at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:166) 
    at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:156) 
    at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118) 
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75) 
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62) 
    at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)

当我将模式更改为:

{
 "name" : "TimestampField"
 "type" : ["null" , "long"]
}

它起作用了。但这不是重点。我想把它转换成 timestamp 为了 Hive 目的。

hof1towb

hof1towb1#

检查配置单元服务器类路径中的avrojar。它可能是不支持逻辑类型的1.7版本(https://avro.apache.org/docs/1.7.4/spec.html)

edqdpe6u

edqdpe6u2#

它受Hive1.1支持。此处显示详细信息

相关问题