Skip to content

Pass 2: sliding-window and Exponential Histogram composition — after #511 #522

Description

@zzylol

Goal

The rest of #509's window-composition rule, after tumbling windows (#518):

  • Sliding window: one summary per active window, with each sample inserted into every active window. Works for non-mergeable summaries; costs more ingestion work and memory.
  • Exponential Histogram: EH buckets that grow with age cover a long history. A query window merges the buckets it covers, with approximate boundaries for old windows. Example 3 Pattern A and Example 4 Pattern A in docs: propose workload-wide planning, summary sharing, and materialization #509.

Blocked on

Implementation of #511 (unified operator graph: OperatorNode with common metadata, no KeepPreAsap wrappers). Do not start before #511 is implemented.
Also depends on #518 (tumbling), and on two #509 TODOs: the design of summary subtract/delete nodes, and query-time materialization in physical planning.

Current state

  • SummaryWindowFramework { Tumbling, Sliding, ExponentialHistogram } exists (types/src/post_asap/summary_window.rs).
  • EH accuracy evidence exists in summary_maintenance_cost/window.rs.
  • The framework is chosen only from provider evidence registered in tests, so it is always None in production.

To decide

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions