我有一个情况,我有一个表在cloudera Impala (Parquet格式),
统计表包括:
大小:23gb行:67m行大小:约5kb列:308
my cloudera是总共6个节点的cloudera群集(每个磁盘84tb,每个ram 251gb)
kudu主服务器和tablet服务器2个主节点,5个tablet服务器(一个节点充当tablet服务器和主节点)
这是我的表模式(结构)
CREATE TABLE SRV_REQ_X
PRIMARY KEY (row_id)
PARTITION BY HASH(row_id) PARTITIONS 5
STORED AS KUDU
TBLPROPERTIES ('kudu.table_name'='IMPALA_DATABASE.KUDU TABLE NAME','kudu.master_addresses'='host1:7051,host2:7051','kudu.num_tablet_replicas' = '3')
AS
Select columns* from table*
不同性能测试
The properties I have checked and played with are
memory_limit_hard_bytes = Checked with 0 and 1 and 250GB (Same result Tablet
Server Crashes)
maintenance_manager_num = Checked with 1 as well as 4
记录被插入,但在某个时候会出现此错误
报告了kudu错误,第一个错误:超时:在329次尝试后未能将批94个操作写入tablet 842e935e768f4a419b193e1fb18e3155:未能写入服务器:2d35eb2445e747bea574a5e1af6e0b2a(bda-ptcl1node02.ptcl.net.pk:7050):将rpc写入192.168.228.2:7050 179.996s后超时(已发送)
我需要插入其他表,这些表大约有102m条记录,我无法理解如何根据集群调整kudu属性。
p、 s进入kudu表的记录最多为13m,具有以下属性,然后发生超时。
memory_limit_hard_bytes = 250GB
maintenance_manager_num = 4
block_cache_capacity_mb = 130GB
Partitions: 4
请帮忙!!
暂无答案!
目前还没有任何答案,快来回答吧!