聚合上的聚合

wkyowqbh  于 2021-07-15  发布在  ClickHouse
关注(0)|答案(2)|浏览(605)

我有时间序列数据(证券交易所交易),我需要按时间间隔进行聚合:1分钟、5分钟、15分钟等。高级时间框架可以从次要时间框架计算,即5 x 1分钟->5分钟。
我做的 MATERIALIZED VIEW, AggregatingMergeTree ,它成功地计算了m1,比如 maxState(price) as price_high, countState(item_id) as trades_count 但我不知道如何制定下一个时间表。如果我使用 maxMerge 在下一个视图中,我返回了一个不正确的结果,正如文档所说,我必须使用这个结果 -stateAggregatingMergeTree ,当我使用 -State 在m5中,它也抱怨错误。
我想构建一系列物化视图,其中次要视图在管道中为高级视图提供来自交易的更新
更新(sql):

  1. CREATE MATERIALIZED VIEW IF NOT EXISTS candle_m1_state
  2. ENGINE = AggregatingMergeTree() PARTITION BY toYYYYMM(toDateTime(timestamp_close_m1/1000))
  3. ORDER BY (platform_id, symbol, timestamp_close_m1)
  4. POPULATE AS
  5. select
  6. platform_id as platform_id,
  7. symbol as symbol,
  8. '1m' as `candle_interval`,
  9. 1000*toUnixTimestamp(toStartOfMinute(toDateTime(timestamp/1000))) as timestamp_m1,
  10. 1000*toUnixTimestamp(addMinutes(toStartOfMinute(toDateTime(timestamp/1000)), 1)) as timestamp_close_m1,
  11. ...
  12. minState(price) as price_low,
  13. countState(item_id) as trades_count
  14. from trade
  15. group by platform_id, symbol, timestamp_m1, timestamp_close_m1, `candle_interval`
  16. order by timestamp_close_m1;
  17. /*The one below definitely wrong due to -State suffix*/
  18. CREATE MATERIALIZED VIEW IF NOT EXISTS candle_m5_test
  19. ENGINE = AggregatingMergeTree() PARTITION BY toYYYYMM(toDateTime(timestamp_close_m5 / 1000))
  20. ORDER BY (platform_id, symbol, timestamp_close_m5) SETTINGS index_granularity = 8192
  21. POPULATE AS
  22. SELECT platform_id, symbol, '5m' AS candle_interval,
  23. 1000 * toUnixTimestamp(toStartOfFiveMinute(toDateTime(timestamp_m1 / 1000))) AS timestamp_m5,
  24. 1000 * toUnixTimestamp(addMinutes(toStartOfFiveMinute(toDateTime(timestamp_m1 / 1000)), 5)) AS timestamp_close_m5,
  25. ...
  26. minState(price_low) AS price_low,
  27. countState(trades_count) AS trades_count
  28. FROM candle_m1_state
  29. GROUP BY platform_id, symbol, timestamp_m5, timestamp_close_m5
  30. ORDER BY platform_id ASC, symbol ASC, timestamp_close_m5 ASC;
qhhrdooz

qhhrdooz1#

我不想把观点串起来。我会为每个聚合做一个视图。
也要记住 MATERIALIZED VIEW 是触发器而不是视图。
我建议:

  1. CREATE MATERIALIZED VIEW
  2. stream__source__target_5m TO target_5m
  3. AS
  4. SELECT ...
  1. CREATE MATERIALIZED VIEW
  2. stream__source__target_1m TO target_1m
  3. AS
  4. SELECT ...

等。
哪里 target_xm 是你的目标table。

ibrsph3r

ibrsph3r2#

很明显,为物化视图链选择查询时间我希望坚持使用该解决方案,而不是从原始数据中为每个时间帧(tf)聚合创建视图。
所以解决办法是:
原始数据->tf1物化视图(aggregatingmergetree,-state后缀)->tf2(来自tf1)(aggregatingmergetree,-mergestate后缀)
然后查询来自任何tf1、tf2。。带-合并后缀

相关问题