从presto hive中的表中获取数据时出错\u cursor\u error

epggiuax  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(1003)

我们在单节点服务器上使用prestodb(0.69)和client。在这里,我们使用的是配置单元目录,表采用orc格式,由350000000行组成。
当运行查询“selectcolumn1fromcorc\utable1 wherecolumn2=123456789”时,我们得到一个配置单元光标错误。column2的数据类型是“int”,下面是错误堆栈:-

"failures" : [ { 
      "type" : "com.facebook.presto.spi.PrestoException", 
      "message" : "Read past end of RLE integer from compressed stream Stream for column 2 kind DATA position: 477741 length: 477741 range: 0 offset: 478409 limit: 478409 range 0 = 0 to 477741 uncompressed: 212681 to 212681", 
      "cause" : { 
        "type" : "java.io.EOFException", 
        "message" : "Read past end of RLE integer from compressed stream Stream for column 2 kind DATA position: 477741 length: 477741 range: 0 offset: 478409 limit: 478409 range 0 = 0 to 477741 uncompressed: 212681 to 212681", 
        "suppressed" : [ ], 
        "stack" : [ "org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.readValues(RunLengthIntegerReaderV2.java:46)", "org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.next(RunLengthIntegerReaderV2.java:287)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$LongTreeReader.next(RecordReaderImpl.java:473)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:1157)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:2196)", "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:106)", "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:57)", "com.facebook.presto.hive.GenericHiveRecordCursor.advanceNextPosition(GenericHiveRecordCursor.java:241)", "ScanFilterAndProjectOperator_11.filterAndProjectRowOriented(Unknown Source)", "com.facebook.presto.operator.AbstractScanFilterAndProjectOperator.getOutput(AbstractScanFilterAndProjectOperator.java:177)", "com.facebook.presto.operator.Driver.process(Driver.java:329)", "com.facebook.presto.operator.Driver.processFor(Driver.java:271)", "com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:674)", "com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:443)", "com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:577)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)", "java.lang.Thread.run(Thread.java:745)" ] 
      }, 
      "suppressed" : [ ], 
      "stack" : [ "com.facebook.presto.hive.GenericHiveRecordCursor.advanceNextPosition(GenericHiveRecordCursor.java:257)", "ScanFilterAndProjectOperator_11.filterAndProjectRowOriented(Unknown Source)", "com.facebook.presto.operator.AbstractScanFilterAndProjectOperator.getOutput(AbstractScanFilterAndProjectOperator.java:177)", "com.facebook.presto.operator.Driver.process(Driver.java:329)", "com.facebook.presto.operator.Driver.processFor(Driver.java:271)", "com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:674)", "com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:443)", "com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:577)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)", "java.lang.Thread.run(Thread.java:745)" ], 
      "errorCode" : { 
        "code" : 16777217, 
        "name" : "HIVE_CURSOR_ERROR" 
      }

查询在由几行组成的表上运行良好。谁能帮我解决这个问题。
下面是config.properties:

coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=8080
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://172.168.1.99:8080
xfyts7mz

xfyts7mz1#

Hive能看懂这张table吗?如果可以的话,这很可能是一个在比presto使用的更新版本的配置单元库中修复的bug,您需要等到presto升级到最新的配置单元版本。如果配置单元无法读取该表,则说明文件已损坏,或者orc读取器中仍存在错误。

相关问题