我有以下工具:hadoop-2.6.0、hive-0.14.0、hbase-0.94.8、sqoop-1.4.5、pig-0.14.0安装在ubuntu 14.0.4的psuedo分布式环境中。
我的目标是使用hcatalog作为接口来处理hive、pig和mapreduce应用程序。
我做的步骤:1。我将mysql配置为远程元存储,mysql-connector-java-5.1.37jar复制到hive\u home/lib中。我已经在hive\u home/conf中为远程元存储创建了hive-site.xml,但运行在同一台机器上。2我有一个hive-env.sh文件,其中hadoop\u home指向hadoop-2.6.0 home。三。我在端口90834上运行远程元存储。在bashrc文件中,我设置了以下env变量:
# Hadoop variables start
export JAVA_HOME=/usr/lib/jvm/java-7-oracle
export HADOOP_HOME=/home/user/hadoop-2.6.0
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
# Hadoop variables end
export HADOOP_USER_CLASSPATH_FIRST=true
export PIG_USER_CLASSPATH_FIRST=true
# PIG ENV VARIABLE
export PIG_HOME=/home/user/pig-0.14.0
export PATH=$PATH:$PIG_HOME/bin
# Hive Env Variable
export HIVE_HOME=/home/user/hive-0.14.0/apache-hive-0.14.0-bin
export PATH=$PATH:$HIVE_HOME/bin
# HCatalog env
export HCAT_HOME=$HIVE_HOME/hcatalog
export HCAT_HOME
export PATH=$PATH:$HCAT_HOME/bin
HCATJAR=$HCAT_HOME/share/hacatalog/hive-hcatalog-core-0.14.0.jar
export HCATJAR
HCATPIGJAR=$HCAT_HOME/share/hcatalog/hive-hcatalog-pig-adapter-0.14.0.jar
export HCATPIGJAR
export HADOOP_CLASSPATH=$HCATJAR:$HCATPIGJAR:$HIVE_HOME/lib/hive-exec-0.14.0.jar\
:$HIVE_HOME/lib/hive-metastore-0.14.0.jar:$HIVE_HOME/lib/jdo-api-*.jar:$HIVE_HOME/lib/libfb303-*.jar\
:$HIVE_HOME/lib/libthrift-*.jar:$HIVE_HOME/conf:$HADOOP_HOME/etc/hadoop
# Pig hcatalog integration
export PIG_OPTS=-Dhive.metastore.uris=thrift://localhost:9083
export PIG_CLASSPATH=$HCAT_HOME/share/hcatalog/*:$HIVE_HOME/lib/*:$HCATPIGJAR:$HIVE_HOME/conf:$HADOOP_HOME/etc/hadoop
我试图在hive\u home/hcatalog/bin路径上调用“hcat”命令。下面是正在生成的错误:
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
**Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4**
at org.apache.hadoop.ipc.Client.call(Client.java:1070)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:427)
... 6 more
观察:在谷歌搜索了很多,如果我理解正确的“服务器ipc版本9无法与客户端版本4通信”是hadoop版本不匹配。所以我在hive-env.sh中添加了hadoop\u home来引用hadoop-2.6.0。错误仍然存在。我不确定我错过了什么。在此方面的任何帮助都将不胜感激。
暂无答案!
目前还没有任何答案,快来回答吧!