feat(univmon)!: port Rust's UnivMon and add its ASAPv1 codec; drop the gob and protobuf formats - #94
Open
GordonYuanyc wants to merge 2 commits into
Open
GordonYuanyc wants to merge 2 commits into
GordonYuanyc wants to merge 2 commits into
Conversation
UnivMon[K] is a pyramid of layerSize layers, each a sketchRow x sketchCol CountL2HH hashing the key with one 128-bit hash at its layer index, beside a heap of at most heapSize keys of type K. The bottom layer comes from the hash at seed index 19; Insert updates layers 0..bottom and FastInsert the bottom layer only, and the first update fixes the mode. Merge adds the counters and rebuilds each heap from both heaps' keys at merged estimates. CalcL1 / CalcL2 / CalcEntropy / CalcCard follow the standard or terminal recurrence, with candidate_complete choosing each layer's threshold. MarshalASAPv1 / UnmarshalASAPv1 encode it as kind 0x10 0x00 and match the univmon_str, univmon_i64 and univmon_empty fixtures; EncodeASAPv1Payload / DecodeASAPv1Payload carry the payload for a Hydra cell. Removes CountSketchUniv, util.go, portable.go, the gob SerializeToBytes / DeserializeUnivSketchFromBytes pair, Hydra's legacy UnivMon grid fallback and the univmon.pb cross-language producer. Hydra's UnivMon cells and the ExponentialHistogram hybrid sketch run on UnivMon[string] and UnivMon[uint64]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ield HeapEntries returns deep copies of byte-slice keys, so a caller cannot change a stored key. Insert and FastInsert reject a zero-value UnivMon. newCountL2HH is unexported. The CalcGSum and UnmarshalASAPv1 docs state that terminal mode always applies thresholds and that "isize" / "usize" keys do not decode. SketchEnvelope drops the univmon field and its import, and reserves field 15 and the name "univmon"; proto/univmon stays for proto/hydra. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
asapv1-only/integration(#85 infra plus the merged #86–#93). Part of the ASAPv1-only series: Go's UnivMon is replaced by a port of Rust's and gets its ASAPv1 codec.Changed (algorithm and API — breaking)
Go's UnivMon differed from Rust almost everywhere (fixed layer shapes, one shared hash, l2 on layer 0 only, string-only heaps, summed-count merge, no update modes or completeness flags). It is now a port of Rust's
univmon.rs/CountL2HH/HHHeap:UnivMon[K asapv1.HeapKey](generic key type via the shared heap helpers), standard and terminal update modes, candidate completeness, Rust's query recurrences (g-sum, L1, L2, entropy, cardinality) and merge. Two independent scratch comparisons (12 scenarios: string/i64/u32/u64/f64 keys, terminal mode, merges, non-power-of-two and single-column widths, 16 layers) gave byte-identical envelopes to Rust; L1, L2 and cardinality match exactly, entropy within ~2 ulps (Gomath.Log2vs Rustf64::log2).UnivSketchand its methods,CountSketchUniv,util.go.Added
MarshalASAPv1/UnmarshalASAPv1(kind0x10 0x00) and the nested-payload hooks used by Hydra's UnivMon counter. All three UnivMon fixtures passCheckGolden(the empty one as both string and int64 keys).Removed
SerializePortable/portableHashSpec, the gob pairs, Hydra's gob grid fallback, the envelope'sunivmon = 15(reserved 15; reserved "univmon";;proto/univmonstays forproto/hydrauntil the Hydra PR), and theunivmon.pbproducer intests/cross_language.Notes
isize/usizecan't be decoded in Go (no matching Go key type).apply_layered_delta, ...); nothing in Go uses it.Verification (Go 1.24.9)
go build,go vet,gofmt,go test ./...,-raceon UnivMon/Hydra/EH. Known failures also onmain:TestKLL_Reset_SubsequentInserts(flaky),tests/cross_languagewithoutXTEST_DIR. Mutation-checked. Implemented and reviewed by separate agents; review findings are addressed in the second commit (the Hydra hash-only key regression the review found is fixed in the Hydra PR on top of this one).🤖 Generated with Claude Code