cloudera impala jdbc查询没有看到array< string>hive列

y4ekin9u  于 2021-06-29  发布在  Hive
关注(0)|答案(1)|浏览(766)

我在hive中有一个表,其结构如下:

  1. > describe volatility2;
  2. Query: describe volatility2
  3. +------------------+---------------+---------+
  4. | name | type | comment |
  5. +------------------+---------------+---------+
  6. | version | int | |
  7. | unmappedmkfindex | int | |
  8. | mfvol | array<string> | |
  9. +------------------+---------------+---------+

它是由spark hivecontext代码使用如下dataframe api创建的:

  1. val volDF = hc.createDataFrame(volRDD)
  2. volDF.saveAsTable(volName)

它继承了模式中定义的rdd结构:

  1. def schemaVolatility: StructType = StructType(
  2. StructField("Version", IntegerType, false) ::
  3. StructField("UnMappedMKFIndex", IntegerType, false) ::
  4. StructField("MFVol", DataTypes.createArrayType(StringType), true) :: Nil)

但是,当我试图使用最新的jdbc impala驱动程序从这个表中进行选择时,最后一列对它不可见。我的查询非常简单—尝试将数据打印到控制台—与驱动程序下载提供的示例代码完全相同:

  1. String sqlStatement = "select * from default.volatility2";
  2. Class.forName(jdbcDriverName);
  3. con = DriverManager.getConnection(connectionUrl);
  4. Statement stmt = con.createStatement();
  5. ResultSet rs = stmt.executeQuery(sqlStatement);
  6. System.out.println("\n== Begin Query Results ======================");
  7. ResultSetMetaData metadata = rs.getMetaData();
  8. for (int i=1; i<=metadata.getColumnCount(); i++) {
  9. System.out.println(rs.getMetaData().getColumnName(i)+":"+rs.getMetaData().getColumnTypeName(i));
  10. }
  11. System.out.println("== End Query Results =======================\n\n");

控制台输出如下:

  1. == Begin Query Results ======================
  2. version:version
  3. unmappedmkfindex:unmappedmkfindex
  4. == End Query Results =======================

是司机的错误还是我遗漏了什么?

eimct9ow

eimct9ow1#

我找到了自己问题的答案。把它贴在这里,这样可以帮助别人,节省搜索时间。显然,impala最近在sql中引入了所谓的“复杂类型”支持,其中包括array。该文档的链接如下:
http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_complex_types.html#complex_types_using
根据这一点,我要做的是将查询更改为如下所示:

  1. select version, unmappedmkfindex, mfvol.ITEM from volatility2, volatility2.mfvol

我得到了正确的预期结果

相关问题