我正在使用hcatalogue流api编写一个示例程序
我有一个正在运行的hadoop、hiveserver和hivemetastore服务器。
我编写了一个java程序来连接hivemetastore
public class HCatalogueStreamingclient {
public static void main(String[] args) {
System.setProperty("hadoop.home.dir", "E:\\midhun\\hadoop\\hive\\winutils");
String dbName = "hive_streaming";
String tblName = "alerts";
ArrayList<String> partitionVals = new ArrayList<String>(2);
partitionVals.add("Asia");
partitionVals.add("India");
HiveEndPoint hiveEP = new HiveEndPoint("thrift://192.168.10.149:8000", dbName, tblName, partitionVals);
HiveConf conf = new HiveConf();
conf.set("hive.exec.scratchdir", "/tmp/hivetmp");
try {
StreamingConnection connection = hiveEP.newConnection(true,conf);
} catch (ConnectionError e) {
e.printStackTrace();
} catch (InvalidPartition e) {
e.printStackTrace();
} catch (InvalidTable e) {
e.printStackTrace();
} catch (PartitionCreationFailed e) {
e.printStackTrace();
} catch (ImpersonationFailed e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
在运行程序时,我遇到以下异常
Exception in thread "main" java.lang.RuntimeException: The root scratch dir: /tmp/hivetmp on HDFS should be writable. Current permissions are: rw-rw-rw-
at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:690)
at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.createPartitionIfNotExists(HiveEndPoint.java:445)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:314)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:278)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoint.java:215)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:192)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:113)
at com.mj.HCatalogueStreamingclient.main(HCatalogueStreamingclient.java:27)
有人知道如何对hdfs文件夹/tmp/hivetmp授予写权限吗
1条答案
按热度按时间zf9nrax11#
问题已确定。
我们需要为hadoop目录/tmp/hivetmp授予所有权限。授予权限的命令是$hadoop\u home/bin/hadoop fs-chmod 777-r/tmp/hivetmp
我在Windows7 64位操作系统上运行这个程序。我们需要下载适合操作系统和版本的winutils。
在做了以上几点之后,我能够使用hivehcataloguestreamingapi建立连接