未能在配置单元上创建inputinitializermanager错误-tez

6rvt4ljy  于 2021-06-28  发布在  Hive
关注(0)|答案(1)|浏览(659)

我已经安装了apache tez 0.8.1、hadoop 2.7.0版和hive 2.01版。我能够成功地运行map reduce作业。但是当我配置hive并尝试运行简单的计数查询时，它返回了以下错误。从尝试查找jar的错误中，我已经将jar放置在类路径中，但错误仍然没有解决。
请帮我解决这个问题。提前谢谢！！。

  1. hive> select count(*) from sample1;
  2.     Query ID = root_20160728215555_a58e91a6-8913-4a57-8715-bc1739a2cb02
  3.     Total jobs = 1
  4.     Launching Job 1 out of 1
  6.     ----------------------------------------------------------------------------------------------
  7.         VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED  
  8.     ----------------------------------------------------------------------------------------------
  9.     Map 1            container        FAILED     -1          0        0       -1       0       0  
  10.     Reducer 2        container        KILLED      1          0        0        1       0       0  
  11.     ----------------------------------------------------------------------------------------------
  12.     VERTICES: 00/02  [>>--------------------------] 0%    ELAPSED TIME: 0.17 s     
  13.     ----------------------------------------------------------------------------------------------
  14.     Status: Failed
  15.     Vertex failed, vertexName=Map 1, vertexId=vertex_1469720608711_0011_1_00, diagnostics=[Vertex vertex_1469720608711_0011_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
  16.         at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
  17.         at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
  18.         at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:138)
  19.         at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:115)
  20.         at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4676)
  21.         at org.apache.tez.dag.app.dag.impl.VertexImpl.access$4300(VertexImpl.java:204)
  22.         at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:3445)
  23.         at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3394)
  24.         at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3375)
  25.         at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
  26.         at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
  27.         at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
  28.         at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
  29.         at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
  30.         at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1975)
  31.         at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:203)
  32.         at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2090)
  33.         at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2076)
  34.         at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
  35.         at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
  36.         at java.lang.Thread.run(Thread.java:745)
  37.     Caused by: java.lang.reflect.InvocationTargetException
  38.         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  39.         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  40.         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  41.         at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  42.         at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
  43.         ... 20 more
  44.     Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/split/SplitLocationProvider
  45.         at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:96)
  46.         ... 25 more
  47.     Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.split.SplitLocationProvider
  48.         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  49.         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  50.         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
  51.         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  52.         ... 26 more
  53.     ]
  54.     Vertex killed, vertexName=Reducer 2, vertexId=vertex_1469720608711_0011_1_01, diagnostics=[Vertex received Kill in NEW state., Vertex vertex_1469720608711_0011_1_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
  55.     DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1

更新：
在我面对上述问题之后，我复制了hive lib文件夹中的hadoop-core-1.2.1.jar。之后，我在启动hive时面临另一个问题。从跟踪中，我可以发现传递了一些非法参数。

  1. Exception in thread "main" java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  2.         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1550)
  3.         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
  4.         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
  5.         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
  6.         at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3080)
  7.         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3108)
  8.         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:543)
  9.         at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:516)
  10.         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:712)
  11.         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:648)
  12.         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  13.         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  14.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  15.         at java.lang.reflect.Method.invoke(Method.java:497)
  16.         at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
  17.     Caused by: java.lang.reflect.InvocationTargetException
  18.         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  19.         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  20.         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  21.         at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
  22.         at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1548)
  23.         ... 14 more
  24.     Caused by: java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 1.2.1
  25.         at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion(ShimLoader.java:165)
  26.         at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:132)
  27.         at org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:93)
  28.         at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:376)
  29.         at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:268)
  30.         at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
  31.         at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
  32.         at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58)
  33.         at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
  34.         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:517)
  35.         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:482)
  36.         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:544)
  37.         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:370)
  38.         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
  39.         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
  40.         at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
  41.         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:219)
  42.         at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:67)
Hivehadoop2bigdataapache-tez

1条答案

按热度按时间
6ljaweal

6ljaweal1#

问题在于apache-tez版本。apache-tez 0.8.1与hadoop 2.7.0和hive 2.0.1不兼容。
已经下载了0.8.4 src并构建了它，解决了这个问题。
谢谢！！。

赞(0）分享  回复(0）举报  2021-06-28
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com