将数据从hdfs加载到hbase

krcsximq  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(380)

我使用的是ApacheHadoop1.1.1和ApacheHBase 0.94.3。我想将数据从hdfs加载到hbase。我写Pig剧本就是为了达到这个目的。首先在habse中创建hbase表,然后编写pig脚本将数据从hdfs加载到hbase。但它没有将数据加载到hbase表中。不知道这是怎么回事。下面是用于创建hbase表的命令:
创建表“mydata”,“mycf”
下面是将数据从hdfs加载到hbase的pig脚本:

  1. A = LOAD '/user/hduser/Dataparse/goodrec1.txt' USING PigStorage(',') as (c1:int, c2:chararray,c3:chararray,c4:int,c5:chararray);
  2. STORE A INTO 'hbase://mydata'
  3. USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
  4. 'mycf:c1,mycf:c2,mycf:c3,mycf:c4,mycf:c5');

在完成剧本后,它说

  1. 2014-04-29 16:01:06,367 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
  2. 2014-04-29 16:01:06,376 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
  3. 2014-04-29 16:01:06,382 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
  4. HadoopVersion PigVersion UserId StartedAt FinishedAt Features
  5. 1.1.1 0.12.0 hduser 2014-04-29 15:58:07 2014-04-29 16:01:06 UNKNOWN
  6. Failed!
  7. Failed Jobs:
  8. JobId Alias Feature Message Outputs
  9. job_201403142119_0084 A MAP_ONLY Message: Job failed! Error - JobCleanup Task Failure, Task: task_201403142119_0084_m_000001 hbase://mydata,
  10. Input(s):
  11. Failed to read data from "/user/hduser/Dataparse/goodrec1.txt"
  12. Output(s):
  13. Failed to produce result in "hbase://mydata"
  14. Counters:
  15. Total records written : 0
  16. Total bytes written : 0
  17. Spillable Memory Manager spill count : 0
  18. Total bags proactively spilled: 0
  19. Total records proactively spilled: 0
  20. Job DAG:
  21. job_201403142119_0084
  22. 2014-04-29 16:01:06,382 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

请帮我把哪里弄错了?

9rnv2umw

9rnv2umw1#

在hbase的输出中指定的列太多。您有5个输入列和5个输出列,但是hbastorage要求第一列是行键,因此输出中应该只有4个

相关问题