将数据从Hive导入GeoMesa Accumulo

56lgkhnf  于 2023-10-18  发布在  Hive
关注(0)|答案(1)|浏览(184)

我正在尝试使用JDBC转换器将数据从Hive摄取到GeoMesa Accumulo中。下面是我对JDBC转换器的配置:

  1. geomesa.converters.hive = {
  2. type = "jdbc",
  3. connection = "jdbc:hive2://host:10000/default",
  4. id-field = "$1",
  5. fields = [
  6. { name = "my_id", type = "string", transform = "$1" },
  7. { name = "shape", type = "geometry", transform = "geometry($2)" }
  8. ]
  9. }

要摄取数据,我使用以下命令:

  1. echo "SELECT * FROM table" | geomesa-accumulo ingest -u user -p pass -c catalog -s hive.sft -f feature -C hive.conf

尽管已经将Hive JDBC驱动程序复制到GeoMesa库,但在运行上述命令时仍遇到以下错误:

  1. 2023-09-13 10:51:59,640 WARN [org.locationtech.geomesa.convert2.package$ErrorHandlingIterator] Failed parsing input:
  2. java.sql.SQLFeatureNotSupportedException: Method not supported
  3. at org.apache.hive.jdbc.HiveConnection.prepareCall(HiveConnection.java:1215)
  4. at org.locationtech.geomesa.convert.jdbc.JdbcConverter$StatementIterator.hasNext(JdbcConverter.scala:86)
  5. at org.locationtech.geomesa.convert2.package$ErrorHandlingIterator.liftedTree1$1(package.scala:84)
  6. at org.locationtech.geomesa.convert2.package$ErrorHandlingIterator.hasNext(package.scala:83)
  7. at org.locationtech.geomesa.convert.jdbc.JdbcConverter$ResultSetIterator.hasNext(JdbcConverter.scala:119)
  8. at org.locationtech.geomesa.utils.collection.CloseableIterator$FlatMapCloseableIterator.hasNext(CloseableIterator.scala:140)
  9. at scala.collection.Iterator.foreach(Iterator.scala:943)
  10. at scala.collection.Iterator.foreach$(Iterator.scala:943)
  11. at org.locationtech.geomesa.utils.collection.CloseableIterator$FlatMapCloseableIterator.foreach(CloseableIterator.scala:132)
  12. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$7(LocalConverterIngest.scala:179)
  13. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$7$adapted(LocalConverterIngest.scala:173)
  14. at org.locationtech.geomesa.utils.io.CloseablePool$CommonsPoolPool.borrow(CloseablePool.scala:68)
  15. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$6(LocalConverterIngest.scala:173)
  16. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$6$adapted(LocalConverterIngest.scala:172)
  17. at org.locationtech.geomesa.utils.io.package$WithClose$.apply(package.scala:64)
  18. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$4(LocalConverterIngest.scala:172)
  19. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$4$adapted(LocalConverterIngest.scala:168)
  20. at scala.collection.Iterator.foreach(Iterator.scala:943)
  21. at scala.collection.Iterator.foreach$(Iterator.scala:943)
  22. at org.locationtech.geomesa.utils.collection.CloseableIterator$CloseableSingleIterator.foreach(CloseableIterator.scala:85)
  23. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$3(LocalConverterIngest.scala:168)
  24. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$3$adapted(LocalConverterIngest.scala:167)
  25. at org.locationtech.geomesa.utils.io.package$WithClose$.apply(package.scala:64)
  26. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$2(LocalConverterIngest.scala:167)
  27. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$2$adapted(LocalConverterIngest.scala:166)
  28. at org.locationtech.geomesa.utils.io.CloseablePool$CommonsPoolPool.borrow(CloseablePool.scala:68)
  29. at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.run(LocalConverterIngest.scala:166)
  30. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  31. at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  32. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  33. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  34. at java.lang.Thread.run(Thread.java:750)

有没有人对解决这个问题有什么见解?

q43xntqr

q43xntqr1#

这似乎是GeoMesa中的一个bug。它应该调用prepareStatement而不是prepareCall,尽管两者似乎都可以与其他数据库一起工作(例如,Postgresql)。我已经打开了一个工单来跟踪问题here
如果可能的话,您可以尝试将这一行改为使用prepareStatement,从源代码构建,然后看看是否可以解决问题。

相关问题