我正在尝试使用amazon athena读取geojson文件。
我的输入数据的头部如下所示:
{
"type": "FeatureCollection",
"name": "sql_statement",
"features": [
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22482 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981739.267883020918816, 30855.609566356935829 ], [ -981739.267883020918816, 31355.354737498135364 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -981241.022986860014498, 30855.609566356935829 ], [ -981739.267883020918816, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22483 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -981241.022986860014498, 30855.609566356935829 ], [ -981241.022986860014498, 31355.354737498135364 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980742.778090699226595, 30855.609566356935829 ], [ -981241.022986860014498, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22484 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980742.778090699226595, 30855.609566356935829 ], [ -980742.778090699226595, 31355.354737498135364 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -980244.533194538322277, 30855.609566356935829 ], [ -980742.778090699226595, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22485 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -980244.533194538322277, 30855.609566356935829 ], [ -980244.533194538322277, 31355.354737498135364 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979746.288298377417959, 30855.609566356935829 ], [ -980244.533194538322277, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22486 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979746.288298377417959, 30855.609566356935829 ], [ -979746.288298377417959, 31355.354737498135364 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -979248.043402216513641, 30855.609566356935829 ], [ -979746.288298377417959, 30855.609566356935829 ] ] ] } },
{ "type": "Feature", "properties": { "gridsize": 500.0, "big_gid": 353, "little_gid": 22487 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -979248.043402216513641, 30855.609566356935829 ], [ -979248.043402216513641, 31355.354737498135364 ], [ -978749.798506055609323, 31355.354737498135364 ], [ -978749.798506055609323, 30855.609566356935829 ], [ -979248.043402216513641, 30855.609566356935829 ] ] ] } },
我使用地理空间文档作为模板定义了表:
CREATE external TABLE IF NOT EXISTS testdb.grid_500
(
gridsize double,
big_gid int,
little_gid int,
geometry binary
)
ROW FORMAT SERDE 'com.esri.hadoop.hive.serde.JsonSerde'
STORED AS INPUTFORMAT 'com.esri.json.hadoop.EnclosedJsonInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 's3://jdl-athena/grid/'
;
但是,我对该表运行的任何查询都会返回相同的、但毫无帮助的错误:
Your query has the following error(s):
HIVE_CURSOR_ERROR: HIVE_CURSOR_ERROR
This query ran against the "testdb" database, unless qualified by the query.
Please post the error message on our forum or contact customer support with
Query Id: 25d3da93-5cfd-46bb-9f77-4eb014679ba6.
你知道我为什么会收到这个错误吗?或者我该如何调试它?我甚至不知道如何获得足够的信息来诊断可能发生的事情。
1条答案
按热度按时间6ioyuze21#
所以在googlefu上花了很多时间之后,我发现(当然)esri已经为地理数据创建了自己的json格式。我使用的是开放标准
GeoJson
esri格式称为ESRI Enclosed JSON
. 对于geometric json,athena只支持esri格式,不支持geojson。我在这里发现了这种区别:https://github.com/esri/gis-tools-for-hadoop/issues/29
所以看起来我需要不同格式的输入数据。