Finding
At the series cap, every new scalar/keyed series under OnOverflowEvictOldest scans the entire w.series map to find the minimum LastSeenMs while holding the window's exclusive mutex.
Code: admitSeriesLocked
For a 100k-series plan with cardinality churn, each new series performs ~100k map iterations. The mutex also serializes all other observations and rotation for that precompute instance, so a burst of unseen labels creates quadratic CPU work and long-tail stalls. The deleted entry's sketch is not returned to the pool on this path, increasing allocation and GC pressure.
This is separate from #591, which covers the envelope path bypassing the cap entirely.
Recommended design
Maintain an eviction index with bounded update cost (for example an intrusive LRU/clock, or a heap with safe stale-entry handling). Admission should be amortized O(1) or O(log N), and eviction should recycle the sketch and delete associated cached snapshots.
The index must preserve the documented least-recently-seen semantics, including deterministic tie behavior if that matters to reproducibility.
Acceptance criteria
- No full series-map scan during observation admission.
- Eviction cost is amortized O(1) or O(log MaxSeries).
- Evicted sketches are recycled and all per-series auxiliary state is released.
- Add a benchmark that holds the runtime at cap and feeds high-churn unique keys at 1k, 10k, and 100k
MaxSeries.
- Report
ns/op, allocations/op, mutex wait/profile data, and p95/p99 admission latency; set a regression gate based on the measured target environment.
Finding
At the series cap, every new scalar/keyed series under
OnOverflowEvictOldestscans the entirew.seriesmap to find the minimumLastSeenMswhile holding the window's exclusive mutex.Code:
admitSeriesLockedFor a 100k-series plan with cardinality churn, each new series performs ~100k map iterations. The mutex also serializes all other observations and rotation for that precompute instance, so a burst of unseen labels creates quadratic CPU work and long-tail stalls. The deleted entry's sketch is not returned to the pool on this path, increasing allocation and GC pressure.
This is separate from #591, which covers the envelope path bypassing the cap entirely.
Recommended design
Maintain an eviction index with bounded update cost (for example an intrusive LRU/clock, or a heap with safe stale-entry handling). Admission should be amortized O(1) or O(log N), and eviction should recycle the sketch and delete associated cached snapshots.
The index must preserve the documented least-recently-seen semantics, including deterministic tie behavior if that matters to reproducibility.
Acceptance criteria
MaxSeries.ns/op, allocations/op, mutex wait/profile data, and p95/p99 admission latency; set a regression gate based on the measured target environment.