我们在单节点服务器上使用prestodb(0.69)和client。在这里,我们使用的是配置单元目录,表采用orc格式,由350000000行组成。
当运行查询“selectcolumn1fromcorc\utable1 wherecolumn2=123456789”时,我们得到一个配置单元光标错误。column2的数据类型是“int”,下面是错误堆栈:-
"failures" : [ {
"type" : "com.facebook.presto.spi.PrestoException",
"message" : "Read past end of RLE integer from compressed stream Stream for column 2 kind DATA position: 477741 length: 477741 range: 0 offset: 478409 limit: 478409 range 0 = 0 to 477741 uncompressed: 212681 to 212681",
"cause" : {
"type" : "java.io.EOFException",
"message" : "Read past end of RLE integer from compressed stream Stream for column 2 kind DATA position: 477741 length: 477741 range: 0 offset: 478409 limit: 478409 range 0 = 0 to 477741 uncompressed: 212681 to 212681",
"suppressed" : [ ],
"stack" : [ "org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.readValues(RunLengthIntegerReaderV2.java:46)", "org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.next(RunLengthIntegerReaderV2.java:287)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$LongTreeReader.next(RecordReaderImpl.java:473)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StructTreeReader.next(RecordReaderImpl.java:1157)", "org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:2196)", "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:106)", "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.next(OrcInputFormat.java:57)", "com.facebook.presto.hive.GenericHiveRecordCursor.advanceNextPosition(GenericHiveRecordCursor.java:241)", "ScanFilterAndProjectOperator_11.filterAndProjectRowOriented(Unknown Source)", "com.facebook.presto.operator.AbstractScanFilterAndProjectOperator.getOutput(AbstractScanFilterAndProjectOperator.java:177)", "com.facebook.presto.operator.Driver.process(Driver.java:329)", "com.facebook.presto.operator.Driver.processFor(Driver.java:271)", "com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:674)", "com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:443)", "com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:577)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)", "java.lang.Thread.run(Thread.java:745)" ]
},
"suppressed" : [ ],
"stack" : [ "com.facebook.presto.hive.GenericHiveRecordCursor.advanceNextPosition(GenericHiveRecordCursor.java:257)", "ScanFilterAndProjectOperator_11.filterAndProjectRowOriented(Unknown Source)", "com.facebook.presto.operator.AbstractScanFilterAndProjectOperator.getOutput(AbstractScanFilterAndProjectOperator.java:177)", "com.facebook.presto.operator.Driver.process(Driver.java:329)", "com.facebook.presto.operator.Driver.processFor(Driver.java:271)", "com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:674)", "com.facebook.presto.execution.TaskExecutor$PrioritizedSplitRunner.process(TaskExecutor.java:443)", "com.facebook.presto.execution.TaskExecutor$Runner.run(TaskExecutor.java:577)", "java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)", "java.lang.Thread.run(Thread.java:745)" ],
"errorCode" : {
"code" : 16777217,
"name" : "HIVE_CURSOR_ERROR"
}
查询在由几行组成的表上运行良好。谁能帮我解决这个问题。
下面是config.properties:
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=8080
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://172.168.1.99:8080
1条答案
按热度按时间xfyts7mz1#
Hive能看懂这张table吗?如果可以的话,这很可能是一个在比presto使用的更新版本的配置单元库中修复的bug,您需要等到presto升级到最新的配置单元版本。如果配置单元无法读取该表,则说明文件已损坏,或者orc读取器中仍存在错误。