spark sql无法输出Dataframe

voase2hg  于 2021-06-24  发布在  Hive
关注(0)|答案(1)|浏览(326)

我尝试运行以下代码,但无法获得结果,错误消息如下所示:

from pyspark.sql import SparkSession
spark = SparkSession.builder.appName('hive').enableHiveSupport().getOrCreate()

list = spark.read.format("csv").option("header", "true").load(r"mypath/mydata.csv")
list.createOrReplaceTempView("mydata")
df = spark.sql("""select * from mydata""")

错误信息:

Traceback (most recent call last):

  File "<ipython-input-31-61851d7298cc>", line 1, in <module>
    df = spark.sql("""select * from mydata""")

  File "C:\ProgramData\Anaconda3\lib\site-packages\pyspark\sql\session.py", line 767, in sql
    return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)

  File "C:\ProgramData\Anaconda3\lib\site-packages\py4j\java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)

  File "C:\ProgramData\Anaconda3\lib\site-packages\pyspark\sql\utils.py", line 69, in deco
    raise AnalysisException(s.split(': ', 1)[1], stackTrace)

AnalysisException: 'java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;'

如果有人能帮我解决这个问题,我将使用spyder和Python3.7。
谢谢您!

x7yiwoj4

x7yiwoj41#

如果不使用enablehivesuppprt,请将其删除

spark = SparkSession.builder.appName('hive').getOrCreate()

相关问题