hive介绍

Hive是一个数据仓库基础工具在Hadoop中用来处理结构化数据。它架构在Hadoop之上，总归为大数据，并使得查询和分析方便。并提供简单的sql查询功能，可以将sql语句转换为MapReduce任务进行运行。
如果没有Hive:使用者->mapreduce->hadoop数据（使用者需要会复杂的mapreduce）
有hive：使用者->HQL（SQL）->hive->mapreduce->hadoop数据（只需要会SQL语句）
下面介绍hive的安装及配置过程

hive安装和配置

1、下载hive安装包

hive安装包下载地址：http://mirror.bit.edu.cn/apache/hive/

选择hive-2.3.7版本进行下载，执行命令：

wget http://mirror.bit.edu.cn/apache/hive/hive-2.3.7/apache-hive-2.3.7-bin.tar.gz

2、解压

将hive解压到/opt/software/hive(我的目录)下

tar -zxvf apache-hive-2.3.7-bin.tar.gz

3、修改环境变量/etc/profile

vim /etc/profile

1 #hive
2 export HIVE_HOME=/opt/software/hive/apache-hive-2.3.7-bin
3 export PATH=$PATH:$HIVE_HOME/bin

执行source /etc.profile,是环境变量生效。

4、执行hive --version

 hive --version

有hive的版本显示则表示安装成功！

5、修改配置

先进入配置目录

cd /opt/software/hive/apache-hive-2.3.7-bin/conf

然后复制hive默认配置模板，命名为hive-site.xml,并修改配置内容

cp hive-default.xml.template hive-site.xml
vim hive-site.xml

主要配置mysql的连接信息。

<!-- 插入一下代码 -->
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>123456</value>
    </property>
   <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://localhost:3306/hive</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <!-- 到此结束代码 -->

配置如下：

注意：hive-site.xml默认有这四个属性的配置，需要删除原有的配置内容。由于配置内容较多，不好定位到配置项，可以将文件下载到windows系统后进行操作。

6、复制mysql的驱动程序到lib目录下面

mysql驱动程序mysql-connector-java下载地址：https://mvnrepository.com/artifact/mysql/mysql-connector-java

找到相应的文件进行下载。

7、在mysql中hive的schema

注意：在此之前需要创建mysql下的hive数据库

schematool -dbType mysql -initSchema

出现如上信息表上初始化schema成功。

8、执行hive命令

hive

如果出现如上界面则表示hive配置成功。

9、容易出现的几个错误

报错一：

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /tmp/hive. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1327)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3895)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:984)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

**解决方案：**退出hadoop安全模式

hadoop dfsadmin -safemode leave

报错二：

Logging initialized using configuration in jar:file:/opt/software/hive/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
	at org.apache.hadoop.fs.Path.<init>(Path.java:171)
	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:663)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:553)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at java.net.URI.checkPath(URI.java:1823)
	at java.net.URI.<init>(URI.java:745)
	at org.apache.hadoop.fs.Path.initialize(Path.java:202)

**解决方案：**配置文件修改如下属性（主要是设置目录）

  <property>
    <name>hive.exec.scratchdir</name>
    <value>/tmp/hive</value>
    <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description>
  </property>

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/tmp/hive/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>

  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/tmp/hive/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

Hive安装与配置详解（图解）