我使用的是spark 2.3.1和连接器/j 5.1.47。
我编写了一个简单的程序来检查metastore的连通性：

from pyspark.context import SparkContext
from pyspark.sql import SparkSession
from pyspark.conf import SparkConf
conf = SparkConf()
conf.set("javax.jdo.option.ConnectionURL", "jdbc:mysql://localhost/my_metastore?createDatabaseIfNotExist=true&useSSL=false")
conf.set("javax.jdo.option.ConnectionDriverName", "com.mysql.jdbc.Driver")
conf.set("javax.jdo.option.ConnectionUserName", "root")
conf.set("javax.jdo.option.ConnectionPassword", "****")
spark = SparkSession.builder \
                .config(conf=conf) \
                .enableHiveSupport() \
                .getOrCreate()
spark.sql("SELECT NOW()").collect()
spark.stop()

令我惊讶的是，我发现在我停止spark会话之后，metastore连接仍然是活动的！

mysql> show processlist;
+------+------+-----------------+--------------+---------+------+----------+------------------+
| Id   | User | Host            | db           | Command | Time | State    | Info             |
+------+------+-----------------+--------------+---------+------+----------+------------------+
|    3 | lc   | localhost       | NULL         | Query   |    0 | starting | show processlist |
| 4342 | root | localhost:54368 | my_metastore | Sleep   |    5 |          | NULL             |
| 4343 | root | localhost:54369 | my_metastore | Sleep   |    5 |          | NULL             |
| 4346 | root | localhost:54372 | my_metastore | Sleep   |    5 |          | NULL             |
| 4347 | root | localhost:54373 | my_metastore | Sleep   |    5 |          | NULL             |
+------+------+-----------------+--------------+---------+------+----------+------------------+
5 rows in set (0.00 sec)

你知道是Spark还是接头/j的问题吗？