你能告诉我newapihadooprdd的maven依赖性吗

u4vypkhs  于 2021-06-09  发布在  Hbase
关注(0)|答案(1)|浏览(463)

你能告诉我newapihadooprdd对maven的依赖性吗。我的代码是:

JavaPairRDD<ImmutableBytesWritable, Result> hBaseRDD = ctx.newAPIHadoopRDD(
                        conf,
                        TextInputFormat.class,,
                        org.apache.hadoop.hbase.io.ImmutableBytesWritable.class,
                        org.apache.hadoop.hbase.client.Result.class);

事先谢谢你的帮助。

dphi5xsq

dphi5xsq1#

您正在使用textinputformat。必须是tableinputformat。您使用的conf是sparkconf还是org.apache.hadoop.conf.configuration。?
maven依赖项。

<dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-common</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-server</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-hadoop-compat</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-protocol</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-hadoop2-compat</artifactId>
        <version>${hbase.version}</version>
    </dependency>
    <!-- HBase dependencies Start -->

代码:

Configuration hbaseConf = HBaseConfiguration.create();
hbaseConf.set("hbase.zookeeper.quorum", "localhost");
hbaseConf.set(TableInputFormat.INPUT_TABLE, "table1");

JavaPairRDD<ImmutableBytesWritable, Result> routerRDD = sc.newAPIHadoopRDD(hbaseConf, TableInputFormat.class, ImmutableBytesWritable.class, Result.class);

相关问题