列计数大于1996099046,无法计算百分位数

ca1c2owp  于 2021-06-14  发布在  Cassandra
关注(0)|答案(1)|浏览(393)

当我运行表格时,直方图显示以下信息:
nodetool tablehistograms键空间tablename

Column counts are larger than 1996099046, unable to calculate percentiles

Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                              (micros)          (micros)           (bytes)
50%             0.00              0.00              0.00         268650950               NaN
75%             0.00              0.00              0.00        3449259151               NaN
95%             0.00              0.00              0.00       25628284214               NaN
98%             0.00              0.00              0.00       44285675122               NaN
99%             0.00              0.00              0.00       44285675122               NaN
Min             0.00              0.00              0.00            105779                 0
Max             0.00              0.00              0.00       442856751229223372036854776000

Cassandra版本:

[cqlsh 5.0.1 | Cassandra 3.11.2 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.

Replication factor 3
4 node cluster
Getting the above message in one node only 
Tried repairing the table but failed with streaming error :

40328:ERROR [StreamReceiveTask:53] 2019-06-10 13:54:33,684 StreamSession.java:593 - [Stream #c9214180-8b82-11e9-90ce-399bac480141] Streaming error occurred on session with peer <IP ADDRESS>
40329-java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Unable to compute ceiling for max when histogram overflowed
40330-  at org.apache.cassandra.utils.Throwables.maybeFail(Throwables.java:51) ~[apache-cassandra-3.11.2.jar:3.11.2]
40331-  at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:373) ~[apache-cassandra-3.11.2.jar:3.11.2]
40332-  at org.apache.cassandra.index.SecondaryIndexManager.buildIndexesBlocking(SecondaryIndexManager.java:383) ~[apache-cassandra-3.11.2.jar:3.11.2]
40333-  at org.apache.cassandra.index.SecondaryIndexManager.buildAllIndexesBlocking(SecondaryIndexManager.java:270) ~[apache-cassandra-3.11.2.jar:3.11.2]
40334-  at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:216) ~[apache-cassandra-3.11.2.jar:3.11.2]
40335-  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_144]
40336-  at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
40337-  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_144]
40338-  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144]
--
0354:ERROR [Reference-Reaper:1] 2019-06-10 13:54:33,907 Ref.java:224 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@7bd8303d) to class org.apache.cassandra.io.util.ChannelProxy$Cleanup@1084465868:PATH/talename-5b621cd0c53311e7a612ffada4e45177/mc-26405-big-Index.db was not released before the reference was garbage collected

表说明包括:

AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

知道为什么会这样吗?欢迎任何帮助或建议。

ddarikpa

ddarikpa1#

分区中不能有20亿个单元格。同时,在一个有44gb分区的表上有一个二级索引也会有多方面的问题。要解决这个问题,除了删除索引和构建新的数据模型以进行迁移之外,实在没有什么办法。您可以构建一个自定义版本的cassandra来忽略这个异常,但是当您处于理论上可能的极限时,其他的东西会很快出现。你已经过了一个点,我很惊讶你在跑。
如果流错误来自修复,则可以在修复数据模型时忽略它。如果它是从引导,我想你需要一个自定义版本的cassandra保持运行在这段时间(或者可以忽略掉你正在更换的down节点)。请记住,节点故障现在对您是一个严重的威胁,因为引导可能无法工作。当你在一个分区里放了这么多的东西时,它就不能被扩展,所以选择是有限的。

相关问题