在elassandra中,cassandra的数据大小是8gb,而elasticsearch.data的大小是83gb。我们的数据流为5毫秒/秒,下面是用于创建表和索引的查询:
表创建:
CREATE TABLE IF NOT EXISTS x.abc (
internal_tag text,
generated_at timestamp,
collected_at timestamp,
data_type text,
metadata text,
recorded_at timestamp,
value text,
PRIMARY KEY(internal_tag, generated_at)
)
WITH CLUSTERING ORDER BY(generated_at ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = { 'keys': 'ALL', 'rows_per_partition': 'NONE' }
AND comment = ''
AND compaction = { 'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4' }
AND compression = { 'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor' }
AND crc_check_chance = 1.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
索引创建:
curl -XPUT -H 'Content-Type: application/json' 'http://10.0.0.01:9200/x_abc_index' -d '{
"settings": {
"keyspace": "x"
},
"mappings":{
"abc" : {
"discover":".*"
}
}
}'
请提出任何解决方案,以克服数据大小的问题。谢谢
1条答案
按热度按时间w8ntj3qf1#
lebigcat建议,可以通过减少Map中索引字段的数量或选择正确的Map来减小elasticsearch索引的大小。