fix(models/kimi-vl): parse real Kimi-VL text config (nullable field) - #608
Merged
Conversation
Follow-up hardening for #525: the real kimi-vl-a3b-thinking checkpoint failed to load with "Failed to parse Kimi-VL text config: invalid type: null, expected usize". Its DeepSeek-V3-style text_config sets q_lora_rank to null (the Moonlight-16B backbone projects the query directly rather than using LoRA-compressed queries), but DeepSeekV3Config declared q_lora_rank as a bare usize, so serde rejected the null. Make q_lora_rank an Option<usize> with #[serde(default)] so both a null value and an absent key parse. Thread the None case through DeepSeekV3Attention: when q_lora_rank is null the attention loads a direct self_attn.q_proj weight, otherwise the LoRA-style q_a_proj -> q_a_layernorm -> q_b_proj chain, mirroring the existing DeepSeek-V2 MLA path. The real checkpoint indeed ships self_attn.q_proj (no q_a_proj/q_b_proj), confirmed from its safetensors index. The shared DeepSeekV3Config is reused by the standalone DeepSeek-V3 loader, the pipeline stage executor, and Youtu-VL. Youtu-VL always wraps a concrete rank in Some so it keeps taking the LoRA branch, and genuine DeepSeek-V3 (numeric q_lora_rank) is unchanged. DeepSeek-V3.2 and GLM-MoE-DSA keep their own config structs and are unaffected. Add regression tests that parse the real config shape (null q_lora_rank) both directly into DeepSeekV3Config and through the Kimi-VL loader's parse_required_vlm_subconfig, plus numeric and absent-key cases.
This was referenced Jul 2, 2026
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up hardening for #525: the merged Kimi-VL port fails to load its real checkpoint.
Problem
mlxcel generate -m /path/to/kimi-vl-a3b-thinking-4bit ...aborts during config parsing:The real
text_config(a DeepSeek-V3-style block) setsq_lora_ranktonull. The Moonlight-16B backbone that Kimi-VL uses projects the query directly rather than using a LoRA-compressed query, so upstream storesq_lora_rank: null.DeepSeekV3Configdeclared the field as a bareusize, and serde rejectsnullforusize.Change
DeepSeekV3Config::q_lora_rankbecomesOption<usize>with#[serde(default)], so both an explicitnulland an absent key parse.DeepSeekV3Attentionthreads theNonecase through the model build: a nullq_lora_rankloads a directself_attn.q_projweight; a numeric value keeps the LoRA-styleq_a_proj -> q_a_layernorm -> q_b_projchain. This mirrors the existing DeepSeek-V2 MLA path. The real checkpoint's safetensors index confirms it shipsself_attn.q_projand noq_a_proj/q_b_proj.Shared-path consistency
DeepSeekV3ConfigandDeepSeekV3Attentionare reused by the standalone DeepSeek-V3 loader, the pipeline stage executor, the Kimi-VL loader, and Youtu-VL:q_lora_rank: null) now load via the directq_projpath.Some, so it keeps taking the LoRA branch (no behavior change).q_lora_rank) is unchanged.Tests
src/models/deepseek_v3.rs:parses_null_q_lora_rank_config(real config shape withq_lora_rank: null) andparses_numeric_and_absent_q_lora_rank.src/loading/vlm_kimi_vl.rs:parses_real_kimi_vl_text_config_with_null_q_lora_rankexercises the loader'sparse_required_vlm_subconfigcall with the real config shape.Verification
cargo test --lib kimi_vl(16 passed)cargo test --lib deepseek_v3(15 passed)cargo check --lib --testscargo clippy --lib --tests -- -D warningscargo fmtReal-model GPU load is deferred to the orchestrator.