检查spark中metastore的表可用性

lmvvr0a8  于 2021-05-27  发布在  Spark
关注(0)|答案(2)|浏览(411)

在使用下面的select查询之前,我想检查该表是否在特定的配置单元数据库中可用。如何从metastore获取信息。

sparkSession.sql("select * from db_name.table_name")
chhkpiq4

chhkpiq41#

试试这个


# This will give the list of tables in the database as a dataframe.

tables = sparkSession.sql('SHOW TABLES IN '+db_name)

# You can collect them to bring in a list of row items

tables_list = tables.select('tableName').collect() 

# convert them into a python array

tables_list_arr=[]
for table_name in tables:
    tables_list_arr.append(str(table_name['tableName']))

# Then execute your command

if(table_name in tables_list_arr):
    sparkSession.sql("select * from db_name.table_name")
axr492tv

axr492tv2#

在表上运行操作之前,可以运行下面的命令

sparkSession.sql("use databaseName");
 val df = sparkSession.sql("show tables like 'tableName'")
 if(df.head(1).isEmpty == false){
//write the code
}

相关问题