Repository navigation
feat(coco)!: ASAPv1 codec with Rust-matching placement; drop the protobuf and gob formats - #92
Open
GordonYuanyc wants to merge 2 commits into
Open
GordonYuanyc wants to merge 2 commits into
GordonYuanyc wants to merge 2 commits into
Conversation
CocoSketch encodes as ASAPv1 kind 0x0c 0x00: metadata carries the standard hash spec (no seed index), rows and cols; the payload is the row-major keys (nil for a free bucket) and values. Encode and decode both reject zero or over-20-row geometry, mass in a free bucket, a key outside the column its row hashes it to, and a key held twice. Buckets now hold the full string key. Row r places a key at xxh3(seed_list[r], key) % cols, and an insert scans all rows for the key before claiming a free bucket. InsertWithHash, EstimateHash and QueryWithHash count the key strconv.FormatUint(hash, 16), and EstimateWithUDF passes the stored key. Removed: SerializePortable, SerializeToBytes, DeserializeCocoSketchFromBytes, the proto/cocosketch package, the coco field of SketchEnvelope (number 17 reserved), and the coco.pb cross-language producer. The golden submodule moves to 43b6cd2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Insert scans all d buckets for the key, otherwise adds v to the smallest (a free bucket counts as 0, ties drawn uniformly) and the key takes that bucket if it was free, else with probability v/val. The empty key and a zero value are recorded. EstimateSubstring sums the buckets whose key contains a partial key. The envelope reserves the name "coco". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #85 (
asapv1-only/infra). Part of the ASAPv1-only series: Go's CocoSketch gets an ASAPv1 codec. Decoding the spec's placement check required changing Go's Coco algorithm to match Rust's, not just adding a codec.Changed (algorithm)
HashIt(r, key) % cols(xxh3 seeded withseed_list[r]over the UTF-8 bytes, 64-bit modulo, no power-of-two constraint) — byte-identical to Rust'shash64_seededoverDataInput::Stracross 1,958 key × row cases.Insertfollows Rust'sCoco::insert: one pass over the d buckets; on no key match, the smallest bucket (free counts as 0) with a uniform reservoir tie-break takes the value, and the key wins it if it was free, else with probability v/val.""and zero values are recorded (they were dropped).EstimateSubstring(like Rust'sestimate_substring);Estimatestays exact-key. The package's previously failing substring tests now pass.Added
(*CocoSketch).MarshalASAPv1/UnmarshalASAPv1(kind0x0c 0x00); spec §3.12 decode rules incl. the placement check, duplicates, mass in free buckets and UTF-8.coco_3x7passesCheckGolden, and each known cell'sEstimateis checked.Removed
proto/cocosketchandcocoin the envelope (reserved 17; reserved "coco";);SerializePortable(portable.go); Coco's gob pair; thecoco.pbproducer intests/cross_language.Notes
InsertWithHash/QueryWithHash/EstimateHash(needed bycommon.Sketchand the ExponentialHistogram wrapper) count under the hex string of the hash, so Coco hashes internally despitecommon.Sketch's "must not hash internally" note. Follow-up: move that adapter into the EH package soCocoSketchis key-only like Rust.Insert).Verification (Go 1.24.9)
go build,go vet,gofmt,go test ./...;go test -count=20 ./sketches/CocoSketch/passes. Known failures also onmain:TestKLL_Reset_SubsequentInserts(flaky),tests/cross_languagewithoutXTEST_DIR. Implemented and reviewed by separate agents; review findings are addressed in the second commit.🤖 Generated with Claude Code