java—cassandra对压缩oops问题免疫吗？

j5fpnvbx 于 2021-06-14 发布在 Cassandra

关注(0)|答案(1)|浏览(390)

我在读java压缩oops。我看到java所做的是将压缩指针用于所有的对象引用。
我能理解的是，对于大量的物体 -XX:+UseCompressedOops 至 -XX:-UseCompressedOops ，可能会占用更多的内存。
假设你有一百万个对象，在第一种情况下，你需要4mb的数据作为参考，在第二种情况下，你需要8mb。
现在，我有一台64gb内存的电脑，我想用50gb的内存给Cassandra。但现在我想知道，额外的18gb是否真的有用，还是会被许多无用的指针数据填满。
所以我的问题是：
Cassandra是管理大量的小对象，还是只管理少数几个大对象(i、 e.它是否为每个单元格、每行、每个表分配一个单独的缓冲区？）
在第一种情况下，当超过32gbcassandra缓存与50gbcassandra缓存时，我们可能得不到多少好处。

Java cassandra pointers

来源：https://stackoverflow.com/questions/54033180/is-cassandra-mostly-immune-to-the-compressed-oops-problem

1条答案

按热度按时间

oo7oh9g91#

从堆的Angular 来看，我的一个集群节点是这样的：


# /usr/lib/jvm/jdk1.8.0_121/bin/jmap -histo -F 2480

Attaching to process ID 2480, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.121-b13
Iterating over heap. This may take a while...
Object Histogram:

num       #instances    #bytes  Class description
--------------------------------------------------------------------------
1:              1300610 1008416856      byte[]
2:              8451955 405693840       java.nio.HeapByteBuffer
3:              2349359 75179488        org.apache.cassandra.db.BufferCell
4:              2723073 73455464        java.nio.ByteBuffer[]
5:              1804559 57745888        org.apache.cassandra.db.composites.CompoundSparseCellName
6:              510867  46169960        java.lang.Object[]
7:              989235  23741640        java.util.concurrent.ConcurrentSkipListMap$Node
8:              25068   20455288        double[]
9:              851046  20425104        org.apache.cassandra.db.composites.CompoundDenseCellName
10:             580063  18562016        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$Node
11:             580056  18561792        com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node
12:             42317   15771032        long[]
13:             600999  14423976        com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$WeightedValue
14:             590310  14167440        org.apache.cassandra.cache.KeyCacheKey
15:             554649  13311576        org.apache.cassandra.db.RowIndexEntry
16:             35239   12545512        int[]
17:             434971  10439304        java.lang.Long
18:             143327  9172928 java.nio.DirectByteBufferR
19:             354204  8500896 java.lang.String
20:             98616   8241104 char[]
21:             37512   8102592 com.codahale.metrics.WeightedSnapshot$WeightedSample[]
22:             181096  7243840 org.apache.cassandra.db.BufferExpiringCell
23:             284486  6827664 org.apache.cassandra.db.AtomicBTreeColumns$Holder
24:             264728  6353472 org.apache.cassandra.cql3.ColumnIdentifier
25:             264405  6345720 java.util.concurrent.ConcurrentSkipListMap$Index
26:             122067  4882680 org.apache.cassandra.io.sstable.IndexHelper$IndexInfo
27:             1       4194320 com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$Node[]
28:             99838   2396112 org.apache.cassandra.dht.Murmur3Partitioner$LongToken
29:             89034   2136816 org.apache.cassandra.db.BufferDecoratedKey
30:             88888   2133312 org.apache.cassandra.db.AtomicBTreeColumns
31:             64001   2048032 org.apache.cassandra.db.composites.CompoundSparseCellName$WithCollection
32:             58297   1399128 java.util.ArrayList
33:             56381   1353144 java.util.EnumMap$EntryIterator$Entry
34:             26794   1286112 java.util.HashMap
35:             37450   1198400 java.util.HashMap$Node
36:             47639   1143336 java.lang.Double
37:             35632   1140224 com.codahale.metrics.WeightedSnapshot$WeightedSample
38:             24311   972440  java.util.TreeMap$Entry
39:             18219   874512  org.apache.cassandra.io.sstable.format.big.SSTableNamesIterator
40:             35651   855624  org.apache.cassandra.db.composites.BoundedComposite
41:             35610   854640  org.apache.cassandra.db.composites.CompoundComposite
42:             26165   837280  java.util.TreeMap$KeyIterator
43:             1248    818688  io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
44:             25419   813408  org.apache.cassandra.db.RowIndexEntry$IndexedEntry
45:             50021   800336  java.lang.Integer
46:             6585    736576  java.lang.Class
47:             4182    736032  org.apache.cassandra.net.MessagingService$Verb[]
48:             7718    652400  java.util.HashMap$Node[]
49:             19391   620512  java.util.concurrent.ConcurrentHashMap$Node
50:             9349    600624  java.util.Hashtable$Entry[]
51:             12504   600192  java.util.TimSort
52:             18661   597152  java.util.Vector
53:             4100    557600  com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap$PaddedAtomicReference
54:             4953    554736  java.net.SocksSocketImpl
55:             13215   528656  java.util.concurrent.ConcurrentSkipListMap$Index[]
56:             15888   508416  java.util.concurrent.ConcurrentSkipListMap$HeadIndex
57:             19379   465096  org.apache.cassandra.db.ArrayBackedSortedColumns$4
58:             9339    448272  java.util.Hashtable
59:             4520    419824  short[]
60:             25970   415520  java.lang.Object
61:             16148   387552  javax.management.ObjectName$Property
62:             9358    374320  java.security.ProtectionDomain
63:             12504   300096  com.codahale.metrics.WeightedSnapshot
64:             9356    299392  java.security.CodeSource
65:             12367   296808  java.util.concurrent.ConcurrentLinkedDeque$Node
66:             5891    282768  org.apache.cassandra.net.MessageOut
67:             5676    272448  java.util.TreeMap
68:             8542    271256  javax.management.ObjectName$Property[]
69:             8034    257088  java.util.EnumMap$EntryIterator
70:             7077    226464  java.net.InetAddress$InetAddressHolder
71:             7072    226304  java.util.ArrayList$Itr
72:             5644    225760  org.apache.cassandra.db.PreHashedDecoratedKey
73:             9288    222912  java.util.concurrent.ConcurrentLinkedQueue$Node
74:             3085    222120  com.sun.jmx.remote.util.OrderClassLoaders
75:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader
76:             3085    222120  javax.management.remote.rmi.RMIConnectionImpl$CombinedClassLoader$ClassLoaderWrapper
77:             6730    215360  java.util.concurrent.locks.AbstractQueuedSynchronizer$Node
78:             8413    201912  org.apache.cassandra.db.DeletionInfo
79:             12504   200064  com.codahale.metrics.WeightedSnapshot$1
80:             4096    196608  java.util.concurrent.ConcurrentHashMap$TreeNode
81:             250     196544  java.util.concurrent.ConcurrentHashMap$Node[]
82:             4881    195240  org.apache.cassandra.db.ArrayBackedSortedColumns
83:             4800    192000  java.util.ArrayList$SubList
84:             150     191840  io.netty.buffer.PoolSubpage[]
85:             3952    189696  org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask
86:             7635    183240  org.apache.thrift.protocol.TField
87:             3716    178368  org.apache.cassandra.db.SliceByNamesReadCommand
88:             2778    177792  io.netty.buffer.PoolSubpage
89:             1979    174152  java.lang.reflect.Method
90:             4271    170840  javax.management.ObjectName
91:             3522    169056  org.apache.cassandra.io.sstable.Descriptor
92:             2340    168480  org.apache.thrift.protocol.TBinaryProtocol
93:             2990    167240  org.apache.cassandra.db.Cell[]
94:             5076    162432  org.apache.cassandra.db.ArrayBackedSortedColumns$ForwardsCellIterator
95:             425     159800  java.lang.Thread
96:             3988    159520  com.google.common.collect.SingletonImmutableBiMap
97:             4949    158368  java.net.Socket
98:             6548    157152  java.net.Inet4Address
99:             4870    155840  sun.misc.FDBigInteger
100:            9730    155680  java.util.HashSet
[...]
2865:           1       16      org.apache.cassandra.service.PendingRangeCalculatorService$1
2866:           1       16      sun.reflect.generics.tree.VoidDescriptor
2867:           1       16      org.apache.cassandra.thrift.Cassandra$Processor$insert
Total :         25759325        -Xms7967M -Xmx7967M -Xmn800M
Heap traversal took 1130.769 seconds.

堆上有25.759.325个示例，此节点以 -Xms7967M -Xmx7967M -Xmn800M -所以这里有很多例子。
有关调整资源，请参阅：https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opstunejvm.html
在我的一个测试中，我在一台主机上运行了4个cassandra示例，每个主机有8gb的堆（在一台内存为64gbram的机器上，为os页面缓存留有32gb的空间，每个主机都有机架感知，以确保在主机停机时不会丢失多个副本）。

赞(0）回复(0）举报 2021-06-14

我来回答

java—cassandra对压缩oops问题免疫吗？

1条答案

相关问题

热门标签

最新问答