Skip to content

fix(models/kimi-vl): parse real Kimi-VL text config (nullable field) - #608

Merged
inureyes merged 2 commits into
mainfrom
fix/issue-525-kimi-vl-text-config
Jul 2, 2026
Merged

fix(models/kimi-vl): parse real Kimi-VL text config (nullable field)#608
inureyes merged 2 commits into
mainfrom
fix/issue-525-kimi-vl-text-config

Conversation

@inureyes

@inureyes inureyes commented Jul 2, 2026

Copy link
Copy Markdown
Member

Follow-up hardening for #525: the merged Kimi-VL port fails to load its real checkpoint.

Problem

mlxcel generate -m /path/to/kimi-vl-a3b-thinking-4bit ... aborts during config parsing:

Failed to parse Kimi-VL text config: invalid type: null, expected usize

The real text_config (a DeepSeek-V3-style block) sets q_lora_rank to null. The Moonlight-16B backbone that Kimi-VL uses projects the query directly rather than using a LoRA-compressed query, so upstream stores q_lora_rank: null. DeepSeekV3Config declared the field as a bare usize, and serde rejects null for usize.

Change

  • DeepSeekV3Config::q_lora_rank becomes Option<usize> with #[serde(default)], so both an explicit null and an absent key parse.
  • DeepSeekV3Attention threads the None case through the model build: a null q_lora_rank loads a direct self_attn.q_proj weight; a numeric value keeps the LoRA-style q_a_proj -> q_a_layernorm -> q_b_proj chain. This mirrors the existing DeepSeek-V2 MLA path. The real checkpoint's safetensors index confirms it ships self_attn.q_proj and no q_a_proj / q_b_proj.

Shared-path consistency

DeepSeekV3Config and DeepSeekV3Attention are reused by the standalone DeepSeek-V3 loader, the pipeline stage executor, the Kimi-VL loader, and Youtu-VL:

  • Kimi-VL and any Moonlight-style backbone (q_lora_rank: null) now load via the direct q_proj path.
  • Youtu-VL wraps its required numeric rank in Some, so it keeps taking the LoRA branch (no behavior change).
  • Genuine DeepSeek-V3 (numeric q_lora_rank) is unchanged.
  • DeepSeek-V3.2 and GLM-MoE-DSA keep their own separate config structs and are not affected.

Tests

  • src/models/deepseek_v3.rs: parses_null_q_lora_rank_config (real config shape with q_lora_rank: null) and parses_numeric_and_absent_q_lora_rank.
  • src/loading/vlm_kimi_vl.rs: parses_real_kimi_vl_text_config_with_null_q_lora_rank exercises the loader's parse_required_vlm_subconfig call with the real config shape.

Verification

  • cargo test --lib kimi_vl (16 passed)
  • cargo test --lib deepseek_v3 (15 passed)
  • cargo check --lib --tests
  • cargo clippy --lib --tests -- -D warnings
  • cargo fmt

Real-model GPU load is deferred to the orchestrator.

Follow-up hardening for #525: the real kimi-vl-a3b-thinking checkpoint failed to load with "Failed to parse Kimi-VL text config: invalid type: null, expected usize". Its DeepSeek-V3-style text_config sets q_lora_rank to null (the Moonlight-16B backbone projects the query directly rather than using LoRA-compressed queries), but DeepSeekV3Config declared q_lora_rank as a bare usize, so serde rejected the null.

Make q_lora_rank an Option<usize> with #[serde(default)] so both a null value and an absent key parse. Thread the None case through DeepSeekV3Attention: when q_lora_rank is null the attention loads a direct self_attn.q_proj weight, otherwise the LoRA-style q_a_proj -> q_a_layernorm -> q_b_proj chain, mirroring the existing DeepSeek-V2 MLA path. The real checkpoint indeed ships self_attn.q_proj (no q_a_proj/q_b_proj), confirmed from its safetensors index.

The shared DeepSeekV3Config is reused by the standalone DeepSeek-V3 loader, the pipeline stage executor, and Youtu-VL. Youtu-VL always wraps a concrete rank in Some so it keeps taking the LoRA branch, and genuine DeepSeek-V3 (numeric q_lora_rank) is unchanged. DeepSeek-V3.2 and GLM-MoE-DSA keep their own config structs and are unaffected.

Add regression tests that parse the real config shape (null q_lora_rank) both directly into DeepSeekV3Config and through the Kimi-VL loader's parse_required_vlm_subconfig, plus numeric and absent-key cases.
@inureyes inureyes added type:bug Bug fixes, error corrections, or issue resolutions priority:high High priority area:models Model architectures, weights, loading, metadata status:review Under review labels Jul 2, 2026
@inureyes
inureyes merged commit 0e36a86 into main Jul 2, 2026
@inureyes
inureyes deleted the fix/issue-525-kimi-vl-text-config branch July 2, 2026 13:49
@inureyes inureyes self-assigned this Aug 31, 2026
@inureyes inureyes added status:done Completed and removed status:review Under review labels Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:models Model architectures, weights, loading, metadata priority:high High priority status:done Completed type:bug Bug fixes, error corrections, or issue resolutions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant