我已经在emr上创建了Hive表
create external table tests3(
transaction_id String,
order_id String,
user_id String,
amount String,
subscriber_number String,
product_type String,
provider String,
region String,
status String,
created_time String,
last_updated_time BIGINT,
)
row format
serde 'org.openx.data.jsonserde.JsonSerDe'
location 's3://locationtobucket';
当我从tests3中选择*时,我会得到记录。但是运行任何tez任务都会给我以下错误。
Query : select count(*) from tests3;
Vertex failed, vertexName=Map 1, vertexId=vertex_1513582536692_0022_1_00
, diagnostics=[Vertex vertex_1513582536692_0022_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: tests3 initializer fail
ed, vertex=vertex_1513582536692_0022_1_00 [Map 1], com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.s3.model.AmazonS3Exception: Bad Req
uest (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: DEBA2E241B9DE8C3),
2条答案
按热度按时间monwx1rj1#
只要将配置单元执行引擎更改为mr,它就会正常启动。
zbwhf8kr2#
请检查您的帐户是否具有访问s3和您尝试访问的bucket的适当权限。另外,检查s3 bucket url是否正确,例如s3://locationtobucket/missingfile
标准的第一步是:通过以下命令尝试使用具有相同凭据的aws命令行工具:
hdfs fs-ls s3a://locationtobucket/