为什么hbase api在扫描带有前缀过滤器的表之后返回空集?

mlmc2os5  于 2021-06-08  发布在  Hbase
关注(0)|答案(1)|浏览(385)

我正在做一个实时管道,我把spark流和hbase连接起来。为了实现这个过程,我必须在hbase表中执行一个filter,确切地说是一个前缀filter,因为我希望匹配键以某个字符串开头的记录。
我正在筛选的表称为“hm\u notificaciones”。我可以成功地连接到hbase shell并从命令行扫描表。运行以下命令:

scan "hm_notificaciones"

我得到以下记录:
行-列+单元格

46948854-20180307              column=info_oferta:id_oferta, timestamp=1520459448795, value=123456

 46948854-20180312170423        column=info_oferta:id_establecimiento, timestamp=1520892403770, value=9999

 46948854-20180312170423        column=info_oferta:id_oferta, timestamp=1520892390858, value=123445

 46948854-20180312170536        column=info_oferta:id_establecimiento, timestamp=1520892422044, value=9239

 46948854-20180312170536        column=info_oferta:id_oferta, timestamp=1520892435173, value=4432

 46948854-20180313110824        column=info_oferta:id_establecimiento, timestamp=1520957374921, value=9990

 46948854-20180313110824        column=info_oferta:id_oferta, timestamp=1520957362458, value=12313

我一直在使用hbase api运行前缀过滤器。我正在编写一些scala代码来连接api并生成过滤器。以下代码编译并执行,但返回空结果:

def scanTable( table_name:String, family: String, search_key: String )= {
  val conf: Configuration = HBaseConfiguration.create()
  val connection: Connection = ConnectionFactory.createConnection(conf)

  // This is a test to verify if I can connect to HBase API. 
  //This statements work and print all the table names in HBase
  val admin = connection.getAdmin

  println("Listing all tablenames")
  val list_table_names = admin.listTableNames()
  list_table_names.foreach(println)

  val table: Table = connection.getTable( TableName.valueOf(table_name) )
  //val htable = new HTable(conf, tableName)

  var colValueMap: Map[String, String] = Map()
  var keyColValueMap: Map[String, Map[String, String]] = Map()
  val prefix = Bytes.toBytes(search_key)
  val scan = new Scan(prefix)
  scan.addFamily(Bytes.toBytes(family))
  val prefix_filter = new PrefixFilter(prefix)
  scan.setFilter(prefix_filter)
  val scanner = table.getScanner(scan)

  for( row <- scanner){
    val content = row.getNoVersionMap
    for( entry <- content.entrySet ){
      for( sub_entry <- entry.getValue.entrySet){
        colValueMap += (Bytes.toString( sub_entry.getKey) -> Bytes.toString(sub_entry.getValue) )
      }
      keyColValueMap += (Bytes.toString(row.getRow) -> colValueMap )
    }
  }

  //this doesn't execute 
  for( ( k, v) <- colValueMap) {
    printf( "key: %s", "value: %s\n", k, v )
  }

  //this never executes since scanner is null (or empty)
  for (result <- scanner) {
    for (cell <- result.rawCells) {
      println("Cell: " + cell + ", Value: " + Bytes.toString(cell.getValueArray, cell.getValueOffset, cell.getValueLength))
    }
  }

  scanner.close
  table.close
  connection.close

}

我尝试了两种方法来打印/获取数据:组合一个Map和遍历resultscanner。但是,我的过滤器似乎不起作用,因为它返回的是null/空集。
您知道有没有其他方法可以在hbase上执行前缀筛选器?
我用来测试上述代码的代码如下:

user_key = "46948854-20181303144609"
scanTable("hm_notificaciones", "info_oferta", user_key)
uoifb46i

uoifb46i1#

第二个循环将不会进入,因为您已经在上一步中迭代了扫描仪。

for (result <- scanner) {
    for (cell <- result.rawCells) {
      println("Cell: " + cell + ", Value: " + Bytes.toString(cell.getValueArray, cell.getValueOffset, cell.getValueLength))
    }
  }

并使用keycolvaluemap打印。对我有用,再检查一遍。

for( ( k, v) <- colValueMap) {
    printf( "key: %s", "value: %s\n", k, v )
  }

相关问题