我正在尝试启动一个druid管理器来接收存储在hadoop中的parqurt数据。然而,我得到以下错误,我找不到任何有关它的信息:
“error”:“无法将类型id'index\u hadoop'解析为[simple type,class io.druid.indexing.overlord.supervisor.supervisorspec]:已知类型id=[noopsuperservisorspec,kafka]\n位于[source:(org.eclipse.jetty.server.httpinputoverhttp)
我尝试在扩展加载列表中加载hadoop deep storage、parquet和avro扩展来修复它,但是没有成功。
这是我的主管json配置:
{
"type" : "index_hadoop",
"spec" : {
"dataSchema" : {
"dataSource" : "hadoop-batch-timeseries",
"parser" : {
"type": "parquet",
"parseSpec" : {
"format" : "parquet",
"flattenSpec": {
"useFieldDiscovery": true,
"fields": [
]
},
"timestampSpec" : {
"column" : "timestamp",
"format" : "auto"
},
"dimensionsSpec" : {
"dimensions": [ "installation", "var_id", "value" ],
"dimensionExclusions" : [],
"spatialDimensions" : []
}
}
},
"metricsSpec" : [
{
"type" : "count",
"name" : "count"
}
],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "DAY",
"queryGranularity" : "NONE",
"intervals" : [ "2018-10-01/2018-11-30" ]
}
},
"ioConfig": {
"type": "hadoop",
"inputSpec": {
"type": "granularity",
"dataGranularity": "day",
"inputFormat": "org.apache.druid.data.input.parquet.DruidParquetInputFormat",
"inputPath": "/warehouse/tablespace/external/hive/demo.db/integers",
"filePattern": "*.parquet",
"pathFormat": "'year'=yyy/'month'=MM/'day'=dd"
},
},
"tuningConfig" : {
"type": "hadoop"
}
},
"hadoopDependencyCoordinates": "3.1.0"
}
1条答案
按热度按时间disho6za1#
我也遇到了同样的问题。通过将其作为任务提交而不是作为主管提交来解决: