pig将hdfs数据导入hbase时出错

dw1jzc5e  于 2021-06-02  发布在  Hadoop
关注(0)|答案(2)|浏览(352)

我的集群版本是{hadoop2.7.1,hbase1.1.2,pig0.15}我尝试通过pig将hdfs数据导入hbase,但发现了问题,错误日志如下所示:

ERROR 1200: Pig script failed to parse:
<file 3hbase.pig, line 4, column 4> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[gprsinfo:Direction gprsinfo:Latitude gprsinfo:Longitude gprsinfo:Returntime gprsinfo:Speed gprsinfo:Id]'

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Pig script failed to parse:
<file 3hbase.pig, line 4, column 4> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[gprsinfo:Direction gprsinfo:Latitude gprsinfo:Longitude gprsinfo:Returntime gprsinfo:Speed gprsinfo:Id]'
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1748)
        at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
        at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
        at org.apache.pig.Main.run(Main.java:631)
        at org.apache.pig.Main.main(Main.java:177)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: Failed to parse: Pig script failed to parse:
<file 3hbase.pig, line 4, column 4> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[gprsinfo:Direction gprsinfo:Latitude gprsinfo:Longitude gprsinfo:Returntime gprsinfo:Speed gprsinfo:Id]'
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
        ... 16 more
Caused by:
<file 3hbase.pig, line 4, column 4> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[gprsinfo:Direction gprsinfo:Latitude gprsinfo:Longitude gprsinfo:Returntime gprsinfo:Speed gprsinfo:Id]'
        at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:1009)
        at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7806)
        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
        ... 17 more
Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[gprsinfo:Direction gprsinfo:Latitude gprsinfo:Longitude gprsinfo:Returntime gprsinfo:Speed gprsinfo:Id]'
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772)
        at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:988)
        ... 23 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:740)
        ... 24 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
        at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:427)
        at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:368)
        at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:239)
        ... 29 more
niwlg2el

niwlg2el1#

pig只包括hbase版本<1.0的hbase连接器(请检查pig的lib dir以获得特定的pig版本),以保持hadoop1的兼容性。
您必须降级到该版本的hbase,或者根据hbase 1.1.2从源代码重新编译pig。
编辑:此处详细讨论:https://issues.apache.org/jira/browse/pig-4728

daolsyd0

daolsyd02#

你可以去其他版本,这个集成是为我工作。
hadoop 2.5.2、hbase 0.98、pig 0.15
我希望你的问题能得到解决

相关问题