Skip to content

fix(index): accept "RQ" when parsing QuantizationType - #8214

Merged
Xuanwo merged 1 commit into
lance-format:mainfrom
jackylee-ch:fix/quantization-type-rq-parse
Aug 4, 2026
Merged

fix(index): accept "RQ" when parsing QuantizationType#8214
Xuanwo merged 1 commit into
lance-format:mainfrom
jackylee-ch:fix/quantization-type-rq-parse

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

Display writes "RQ" for QuantizationType::Rabit but FromStr only accepted "RABIT", so the two never round-tripped.

IvfIndexState::serialize stores the quantization type with to_string(); deserialize reads it back with parse::<QuantizationType>(). For an IVF_RQ index that parse always failed, and since CacheCodec::deserialize turns a body error into CacheDecode::Miss, the failure was silent — the index just never reused its serialized state.

Display has to keep emitting "RQ": index_type_string builds IVF_{quantization_type} and IndexType::try_from expects "IVF_RQ". So the fix belongs on the FromStr side. "RABIT" stays accepted for headers already on disk.

The existing round-trip coverage missed this because test_ivf_index_state_roundtrip hardcodes QuantizationType::Flat and the test_prewarm_and_query_with_serializing_backend cases are PQ and HNSW_SQ only. The new #[rstest] covers every variant.

Testing

  • cargo test -p lance-index --lib vector::quantizer — 6 pass; case_5_rabit fails without the one-line fix
  • cargo test -p lance-index --lib vector:: — 331 pass
  • cargo fmt --all -- --check, cargo clippy -p lance-index --tests --benches -- -D warnings

`Display` writes "RQ" for `QuantizationType::Rabit` but `FromStr` only
accepted "RABIT", so the two never round-tripped.

`IvfIndexState::serialize` stores the quantization type with
`to_string()` and `deserialize` reads it back with
`parse::<QuantizationType>()`. For an IVF_RQ index the parse always
failed, and because `CacheCodec::deserialize` turns a body error into
`CacheDecode::Miss`, the failure was silent: the index simply never
reused its serialized state.

`Display` has to keep emitting "RQ" — `index_type_string` builds
`IVF_{quantization_type}` and `IndexType::try_from` expects "IVF_RQ" —
so the fix is on the `FromStr` side. "RABIT" stays accepted for headers
already on disk.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Aug 4, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The parser now accepts the canonical RQ spelling emitted by Display while preserving existing RABIT inputs. This restores IVF_RQ cache-state round trips without changing the persisted index identifier.

@Xuanwo Xuanwo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this change makes sense

@Xuanwo
Xuanwo merged commit 7db9573 into lance-format:main Aug 4, 2026
5 checks passed
@jackylee-ch

Copy link
Copy Markdown
Contributor Author

Thanks @Xuanwo

@Xuanwo Xuanwo added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants