聚合上的聚合

wkyowqbh  于 2021-07-15  发布在  ClickHouse
关注(0)|答案(2)|浏览(540)

我有时间序列数据(证券交易所交易),我需要按时间间隔进行聚合:1分钟、5分钟、15分钟等。高级时间框架可以从次要时间框架计算,即5 x 1分钟->5分钟。
我做的 MATERIALIZED VIEW, AggregatingMergeTree ,它成功地计算了m1,比如 maxState(price) as price_high, countState(item_id) as trades_count 但我不知道如何制定下一个时间表。如果我使用 maxMerge 在下一个视图中,我返回了一个不正确的结果,正如文档所说,我必须使用这个结果 -stateAggregatingMergeTree ,当我使用 -State 在m5中,它也抱怨错误。
我想构建一系列物化视图,其中次要视图在管道中为高级视图提供来自交易的更新
更新(sql):

CREATE MATERIALIZED VIEW IF NOT EXISTS candle_m1_state
ENGINE = AggregatingMergeTree() PARTITION BY toYYYYMM(toDateTime(timestamp_close_m1/1000)) 
ORDER BY (platform_id, symbol, timestamp_close_m1)
POPULATE AS
select
 platform_id as platform_id,
 symbol as symbol,
 '1m' as `candle_interval`,
 1000*toUnixTimestamp(toStartOfMinute(toDateTime(timestamp/1000))) as timestamp_m1,
 1000*toUnixTimestamp(addMinutes(toStartOfMinute(toDateTime(timestamp/1000)), 1)) as timestamp_close_m1,
...
 minState(price) as price_low,
 countState(item_id) as trades_count
from trade
group by platform_id, symbol, timestamp_m1, timestamp_close_m1, `candle_interval`
order by timestamp_close_m1;

/*The one below definitely wrong due to -State suffix*/
CREATE MATERIALIZED VIEW IF NOT EXISTS candle_m5_test
ENGINE = AggregatingMergeTree() PARTITION BY toYYYYMM(toDateTime(timestamp_close_m5 / 1000)) 
ORDER BY (platform_id, symbol, timestamp_close_m5) SETTINGS index_granularity = 8192 
POPULATE AS 
SELECT platform_id, symbol, '5m' AS candle_interval,
 1000 * toUnixTimestamp(toStartOfFiveMinute(toDateTime(timestamp_m1 / 1000))) AS timestamp_m5,
 1000 * toUnixTimestamp(addMinutes(toStartOfFiveMinute(toDateTime(timestamp_m1 / 1000)), 5)) AS timestamp_close_m5, 
 ...
 minState(price_low) AS price_low, 
 countState(trades_count) AS trades_count 
FROM candle_m1_state 
GROUP BY platform_id, symbol, timestamp_m5, timestamp_close_m5 
ORDER BY platform_id ASC, symbol ASC, timestamp_close_m5 ASC;
qhhrdooz

qhhrdooz1#

我不想把观点串起来。我会为每个聚合做一个视图。
也要记住 MATERIALIZED VIEW 是触发器而不是视图。
我建议:

CREATE MATERIALIZED VIEW
    stream__source__target_5m TO target_5m
AS
SELECT ...
CREATE MATERIALIZED VIEW
    stream__source__target_1m TO target_1m
AS
SELECT ...

等。
哪里 target_xm 是你的目标table。

ibrsph3r

ibrsph3r2#

很明显,为物化视图链选择查询时间我希望坚持使用该解决方案,而不是从原始数据中为每个时间帧(tf)聚合创建视图。
所以解决办法是:
原始数据->tf1物化视图(aggregatingmergetree,-state后缀)->tf2(来自tf1)(aggregatingmergetree,-mergestate后缀)
然后查询来自任何tf1、tf2。。带-合并后缀

相关问题