我尝试在启用了tez的azurehdinsight上的配置单元上创建索引。我可以成功创建索引,但无法重建它们:作业失败,输出如下:
Map 1: -/- Reducer 2: 0/1
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1421234198072_0091_1_01, diagnostics=[Vertex Input: measures initializer failed.]
Vertex killed, vertexName=Reducer 2, vertexId=vertex_1421234198072_0091_1_00, diagnostics=[Vertex > received Kill in INITED state.]
DAG failed due to vertex failure. failedVertices:1 killedVertices:1
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask
我已使用以下作业创建了表和索引:
DROP TABLE IF EXISTS Measures;
CREATE TABLE Measures(
topology string,
val double,
date timestamp,
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE LOCATION 'wasb://<mycontainer>@<mystorage>.blob.core.windows.net/';
CREATE INDEX measures_index_topology ON TABLE Measures (topology) AS 'COMPACT' WITH DEFERRED REBUILD;
CREATE INDEX measures_index_date ON TABLE Measures (date) AS 'COMPACT' WITH DEFERRED REBUILD;
ALTER INDEX measures_index_topology ON Measures REBUILD;
ALTER INDEX measures_index_date ON Measures REBUILD;
我错在哪里?为什么我的重建索引失败了?致以最诚挚的问候
1条答案
按热度按时间2q5ifsrm1#
看起来tez在空表上生成索引时可能有问题。我可以得到与您相同的错误(不使用json serde),如果您查看失败的dag的应用程序日志,您可能会看到如下内容:
如果用一个伪记录填充表,它似乎可以正常工作。我用过:
之后,索引重建就可以正常运行了。