将数据从hdfs加载到hbase

krcsximq  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(340)

我使用的是ApacheHadoop1.1.1和ApacheHBase 0.94.3。我想将数据从hdfs加载到hbase。我写Pig剧本就是为了达到这个目的。首先在habse中创建hbase表,然后编写pig脚本将数据从hdfs加载到hbase。但它没有将数据加载到hbase表中。不知道这是怎么回事。下面是用于创建hbase表的命令:
创建表“mydata”,“mycf”
下面是将数据从hdfs加载到hbase的pig脚本:

A = LOAD '/user/hduser/Dataparse/goodrec1.txt' USING PigStorage(',') as (c1:int, c2:chararray,c3:chararray,c4:int,c5:chararray);   
    STORE A INTO 'hbase://mydata'
    USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
          'mycf:c1,mycf:c2,mycf:c3,mycf:c4,mycf:c5');

在完成剧本后,它说

2014-04-29 16:01:06,367 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete

2014-04-29 16:01:06,376 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2014-04-29 16:01:06,382 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
1.1.1   0.12.0  hduser  2014-04-29 15:58:07     2014-04-29 16:01:06     UNKNOWN

Failed!

Failed Jobs:
JobId   Alias   Feature Message Outputs
job_201403142119_0084   A       MAP_ONLY        Message: Job failed! Error -           JobCleanup Task Failure, Task: task_201403142119_0084_m_000001     hbase://mydata,

Input(s):
Failed to read data from "/user/hduser/Dataparse/goodrec1.txt"

Output(s):
Failed to produce result in "hbase://mydata"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_201403142119_0084

2014-04-29 16:01:06,382 [main] INFO   org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

请帮我把哪里弄错了?

9rnv2umw

9rnv2umw1#

在hbase的输出中指定的列太多。您有5个输入列和5个输出列,但是hbastorage要求第一列是行键,因此输出中应该只有4个

相关问题