这可以在ScalaSpark中使用HiveWare连接器完成（我使用的是hortonworks）。使用的jar是“hive-warehouse-connector_2.11-1.0.0.3.1.0.0-78.jar”。
示例代码：

import com.hortonworks.hwc.HiveWarehouseSession
import com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder
val conf = new SparkConf()
val spark = SparkSession.builder().appName(AppName).enableHiveSupport().getOrCreate()
spark.conf.set("spark.sql.hive.hiveserver2.jdbc.url","//your hive url")
val hive = HiveWarehouseBuilder.session(spark).build()
val res = hive.table("db.tablename")
res.show(20,False)

这也可以通过使用pyspark来实现，但是您必须向它添加一些配置，下面是在pyspark中执行相同操作的代码示例

from pyspark.sql import  *
from pyspark_llap import HiveWarehouseSession
from pyspark.sql import SparkSession
from pyspark.sql.rdd.RDD
from pyspark.sql.types import *
from pyspark.sql import Window
from pyspark.sql.functions import *
from pyspark.sql.functions import *

spark = SparkSession.builder \
                   .appName("appname") \
                   .enableHiveSupport() \
                   .getOrCreate()

hive = HiveWarehouseSession.session(spark).build()
sample = hive.executeQuery("""select * from <schema.tablename>""")
sample.show()`

要使用spark submit执行上述操作，请添加以下配置
spark submit--jars/usr/hdp/3.1.0.0-78/hive\u warehouse\u connector/hive-warehouse-connector-assembly-1.0.0.3.1.0.0-78.jar--py files/usr/hdp/current/hive\u warehouse\u connector/pyspark\u hwc-1.0.0.0-78.zip--conf spark.sql.hive.hiveserver2.jdbc.url=“jdbc url；servicediscoverymode=zookeeperha；zookeepernamespace=hs2activePasseHA”--conf spark.hadoop.hive.llap.daemon.service.hosts=“@llap0”--conf spark.datasource.hive.warehouse.load.staging.dir=“/tmp”--conf spark.hadoop.hive.zookeer.quorum=“all zookeer url”--conf spark.sql.hive.hiveserver2.jdbc.url.principal=“jdbc连接的url”--confspark.security.credentials.hiveserver2.enabled=“true”testpysparkjob.py