fix: disable timestamps when CTC weights are missing - #3211
Conversation
Fail closed for native, remote-code, and vLLM Fun-ASR-Nano inference when a checkpoint omits or partially initializes CTC tensors. Document the current Fun-ASR-MLT-Nano-2512 limitation and cover missing, mismatched, partial, and complete checkpoint paths. Fixes #3208.
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to gracefully disable character-level timestamps when a checkpoint lacks complete CTC weights, updating the documentation, native model, VLLM inference, and adding corresponding unit tests. The review feedback highlights critical edge cases where CTC timestamps could be silently disabled, specifically when dealing with DistributedDataParallel (DDP) wrapped models or checkpoints (which introduce a module. prefix) and when loading wrapped state dicts from init_param_path without unwrapping them first.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Filter vLLM timestamp weights by destination key and shape before loading so incompatible checkpoints disable CTC instead of aborting text transcription.
Handle DDP-prefixed and wrapped checkpoint state for loader hooks, native and remote-code decoder initialization, and vLLM timestamp weights.
Summary
ctc_decoder.*orctc.*Fun-ASR-MLT-Nano-2512timestamp limitationRoot cause
The generic pretrained loader starts from the model's initialized state dict and replaces only compatible checkpoint tensors. Missing CTC tensors therefore remained randomly initialized while the final strict load still succeeded, so timestamp inference could run with untrained layers. The vLLM path had the same fail-open behavior.
This change disables all CTC timestamp components unless every required CTC tensor was loaded. Text transcription remains available.
Fixes #3208.
Validation
PYTHONPATH="$PWD" python -m pytest -q tests/test_fun_asr_nano*.py: 26 passedpython -m py_compileon all changed Python filesgit diff --cached --checkFun-ASR-MLT-Nano-2512checkpoint: 1,261 tensors, no CTC tensors; transcription succeeds and timestamp fields are absentFun-ASR-Nano-2512checkpoint: 1,347 tensors with 84 decoder and 2 CTC tensors; CTC remains enabled and timestamps are returnedThe vLLM checkpoint-loading behavior is covered by focused unit tests; no CUDA vLLM end-to-end run was performed in this CPU validation environment.