最后，使用shc，我可以使用pyspark代码连接到hbase-1.2.1和spark-2.3.1。以下是我的工作：
我所有的hadoop[namenode、datanode、nodemanager、resourcemanager]&hbase[hmaster、hregionserver、hquorumpeer]deamons都在我的ec2示例上启动并运行。
我将emp.csv文件放在hdfs location/test/emp.csv，其中包含以下数据：

key,empId,empName,empWeight
1,"E007","Bhupesh",115.10
2,"E008","Chauhan",110.23
3,"E009","Prithvi",90.0
4,"E0010","Raj",80.0
5,"E0011","Chauhan",100.0

我用以下代码创建了readwritehbase.py文件[用于从hdfs读取emp.csv文件，然后在hbase中首先创建tblemployee，将数据推入tblemployee，然后再次从同一个表中读取一些数据并在控制台上显示]：

from pyspark.sql import SparkSession

def main():
    spark = SparkSession.builder.master("yarn-client").appName("HelloSpark").getOrCreate()

    dataSourceFormat = "org.apache.spark.sql.execution.datasources.hbase"
    writeCatalog = ''.join("""{
                "table":{"namespace":"default", "name":"tblEmployee", "tableCoder":"PrimitiveType"},
                "rowkey":"key",
                "columns":{
                  "key":{"cf":"rowkey", "col":"key", "type":"int"},
                  "empId":{"cf":"personal","col":"empId","type":"string"},
                  "empName":{"cf":"personal", "col":"empName", "type":"string"},
                  "empWeight":{"cf":"personal", "col":"empWeight", "type":"double"}
                }
              }""".split())

    writeDF = spark.read.format("csv").option("header", "true").option("inferSchema", "true").load("/test/emp.csv")
    print("csv file read", writeDF.show())
    writeDF.write.options(catalog=writeCatalog, newtable=5).format(dataSourceFormat).save()
    print("csv file written to HBase")

    readCatalog = ''.join("""{
                "table":{"namespace":"default", "name":"tblEmployee"},
                "rowkey":"key",
                "columns":{
                  "key":{"cf":"rowkey", "col":"key", "type":"int"},
                  "empId":{"cf":"personal","col":"empId","type":"string"},
                  "empName":{"cf":"personal", "col":"empName", "type":"string"}
                }
              }""".split())

    print("going to read data from Hbase table")
    readDF = spark.read.options(catalog=readCatalog).format(dataSourceFormat).load()
    print("data read from HBase table")
    readDF.select("empId", "empName").show()
    readDF.show()

# entry point for PySpark application

if __name__ == '__main__':
    main()

在vm控制台上使用以下命令运行此脚本：

spark-submit --master yarn-client --packages com.hortonworks:shc-core:1.1.1-2.1-s_2.11 --repositories http://nexus-private.hortonworks.com/nexus/content/repositories/IN-QA/ readwriteHBase.py

中间结果：读取csv文件后：

+---+-----+-------+---------+
|key|empId|empName|empWeight|
+---+-----+-------+---------+
|  1| E007|Bhupesh|    115.1|
|  2| E008|Chauhan|   110.23|
|  3| E009|Prithvi|     90.0|
|  4|E0010|    Raj|     80.0|
|  5|E0011|Chauhan|    100.0|
+---+-----+-------+---------+

最终输出：从hbase表读取数据后：

+-----+-------+
|empId|empName|
+-----+-------+
| E007|Bhupesh|
| E008|Chauhan|
| E009|Prithvi|
|E0010|    Raj|
|E0011|Chauhan|
+-----+-------+

注意：在创建hbase表并将数据插入hbase表时，它希望numberofregions应该大于3，因此我添加了 options(catalog=writeCatalog, newtable=5) 向hbase添加数据时