无法启动spark thriftserver(spark的hive-site.xml未覆盖默认值)

gcxthw6b  于 2021-06-28  发布在  Hive
关注(0)|答案(1)|浏览(330)

背景:
世纪7
hadoop-2.7.3版本
spark-2.0.1-bin-hadoop2.7版本
apache-hive-2.1.0-bin(仅用于通过配置单元启动metastore服务——服务metastore)
配置了hadoop\u home、spark\u home和hive\u home等
MySQL5.7.16版本
已经将mysql-connector-java-5.1.40-bin.jar放入hive/lib/和spark/jar/
hive-site.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://www.test.com:3306/metastore</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>hive</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>password</value>
    </property>
    <property>
        <name>hive.metastore.uris</name>
        <value>thrift://www.test.com:9083</value>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>true</value>
    </property>
</configuration>

我将hive-site.xml放入hive/conf/和spark/conf/
但是运行start-thriftserver.sh,我得到了错误日志(在spark\u home/logs/spark--hivethriftserver2.out中):

......
INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
INFO metastore: Trying to connect to metastore with URI thrift://www.test.com:9083
INFO metastore: Connected to metastore.
......
DEBUG ObjectStore: Overriding javax.jdo.option.ConnectionURL value null from  jpox.properties with jdbc:derby:memory:;databaseName=/tmp/spark-37dcab7f-655a-4506-abd7-492a8620a33e/metastore;create=true
......
INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
......
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

在日志中我们可以看到属性“hive.metastore.uris”确实起作用。
但是其他mysql相关属性似乎不起作用,metastore仍然使用默认的数据库derby。
感谢您的帮助!谢谢。
断然的:
我删除了hive.metastore.uris,因为我只是在本地使用了hive。

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://www.test.com:9083</value>
</property>

将hive.metasotre.schema.verification设置为false。谢谢尼玛的帮助。

vsnjm48y

vsnjm48y1#

找到此错误日志

Caused by: MetaException(message:Version information not found in metastore

集合,值 hive.metastore.schema.verification 在hive和spark conf的hive-site.xml中设置为false,然后重新启动服务并重试

相关问题