apache配置单元无法连接到derby:metastore_db 虽然德比的ij可以

nc1teljy  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(263)

我正在三台机器上的hadoop集群上设置一个配置单元。hadoop(2.7.1)和derby(10.11)运行良好:

hduser@master:~$ ij
ij version 10.11
ij> connect 'jdbc:derby://localhost:1527/metastore_db;create=true';
ij> select * from a;
ID
-----------

0 rows selected
ij>

但hive抱怨:

...
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
        ... 8 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
        ... 14 more
Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby://localhost:1527/metastore_db;create=true , username = APP. Terminating connectio pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
java.sql.SQLException: No suitable driver found for jdbc:derby://localhost:1527/metastore_db;create=true
...

环境变量:


# Java

export JAVA_HOME=/usr/lib/jvm/jdk

# derby

export DERBY_HOME=/usr/local/derby
export PATH=$PATH:$DERBY_HOME/bin
export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar

# Hadoop

export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

# hive

export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin
export CLASSPATH=$CLASSPATH:/usr/local/hive/lib/*:.

hive-site.xml在配置中只有此属性:

<property>
     <name>javax.jdo.option.ConnectionURL</name>
     <value>jdbc:derby://localhost:1527/metastore_db;create=true </value>
     <description>JDBC connect string for a JDBC metastore </description>
  </property>

发生什么事?

biswetbf

biswetbf1#

当我在另一个终端选项卡中打开sparkshell会话时,我开始看到这个错误,当时在不同的终端选项卡中已经有一个打开的会话。关闭这个旧的终端选项卡,然后在当前选项卡中再次启动sparkshell会话,解决了这个问题。

kmpatx3s

kmpatx3s2#

你说:

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar

但你也说过:

jdbc:derby://localhost:1527/metastore_db;create=true

因此,您为derby的客户机/服务器配置提供了一个jdbc连接url,但您提供的类路径是为derby的嵌入式配置提供的。
因此您收到了错误:

No suitable driver found

您可以更改类路径,也可以更改jdbc连接url。
如果要使用derby的客户机/服务器配置,请将类路径切换为:

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derbyclient.jar

因为这个配置需要jdbc客户机驱动程序。
如果要使用derby的嵌入式配置,请不要使用类路径,并将jdbc连接url更改为:

jdbc:derby:metastore_db;create=true

要了解有关不同derby配置的更多信息,请阅读以下内容:http://db.apache.org/derby/docs/10.11/getstart/cgsquck70629.html

相关问题