chore(scripts): add ftw_hotfix.py to repair FTW checkpoints from older builds - #424
Merged
Merged
Conversation
danielamadori
pushed a commit
to danielamadori/DFlash-FreeToken
that referenced
this pull request
Sep 15, 2026
…r builds (FlashML-org#424) * chore(scripts): add ftw_hotfix.py to repair FTW checkpoints from older builds * fix(scripts): make ftw_hotfix in-place repair crash-safe and validate the FTW * fix(scripts): ftw_hotfix keeps --out read-only, fails on dir fsync errors, checks PLE shapes (cherry picked from commit 505477a)
nomanoma121
pushed a commit
to nomanoma121/My-FreeToken
that referenced
this pull request
Sep 16, 2026
… the readers Upstream FlashML-org#426 stops qwen4_exp guessing what a checkpoint quantized. It had been matching the modelopt ignore list against probe names by hand, and the new nvidia/Qwen3.8-Flash-Next-NVFP4 build does not fit that guess. The expert kind now comes from QuantConfig.scheme_for_name, and attn_quant / dense_quant / lm_head_quant are no longer parsed for this family -- every module takes its own scheme from the QuantConfig when it is built. Nothing in the fork read those three for Flash-Next. FlashML-org#427 is the groundwork for it: checkpoint_quant_config moved to models/register.py, and EngineConfig installs the result in a module global (set_quant_config), because the weight readers are handed the model path and nothing else. Upstream main has no caller yet -- the consumer is its qwen4_exp block-fp8 dense reader, which is not merged here. --dense-quant is what makes the order matter. It wraps the checkpoint's config in LoadTimeFp8Config, so the install has to come after the wrap, or a reader would quantize against schemes the layers were not built from. Nothing calls get_quant_config() yet, so no test would have noticed the difference: test_engine_config_dense_quant_override now asserts the identity, and that assert is what will catch it when the fp8-dense reader lands. FlashML-org#424 adds scripts/ftw_hotfix.py, which repairs FTW checkpoints written by older builds. It does not apply to this machine or the 3060 -- both serve the HF checkpoint dirs, not FTW. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MT-z
added a commit
to MT-z/FreeToken
that referenced
this pull request
Sep 16, 2026
…off and FlashML-org#424's FTW script fb7f732 (FlashML-org#427) refactor(quant): hand the checkpoint QuantConfig to the weight readers 505477a (FlashML-org#424) chore(scripts): ftw_hotfix.py FlashML-org#427 is what makes this worth merging now rather than later. It moves checkpoint_quant_config from engine/config.py to models/register.py and installs the resolved QuantConfig in a module-level global from EngineConfig.model_config. Nothing in upstream main reads that global yet, so today it changes no behaviour -- but it is groundwork on the same path FlashML-org#418 and FlashML-org#426 already moved, and this box's model (Ornith, qwen3_5_moe, MIXED_PRECISION) loads through exactly that code. Divergence here costs more at the next commit than the merge costs now. FlashML-org#424 comes along because it sits between us and FlashML-org#427 on main. It is a standalone repair script plus docs, no engine change, and there are no FTW checkpoints on this box, so it does nothing here either way. Verified on this box (results/20260910-merge-427.txt in freetoken-systest): same 8 failures by node ID, 1,910 passed; greedy output bit-identical over 399 tokens at temperature 0.0; same startup warnings, same expert path. Assisted-by: Claude Opus 5
trcwebdesign
pushed a commit
to trcwebdesign/FreeToken
that referenced
this pull request
Sep 21, 2026
…r builds (FlashML-org#424) * chore(scripts): add ftw_hotfix.py to repair FTW checkpoints from older builds * fix(scripts): make ftw_hotfix in-place repair crash-safe and validate the FTW * fix(scripts): ftw_hotfix keeps --out read-only, fails on dir fsync errors, checks PLE shapes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up to #418 and #420. Four kinds of FTW converted by older builds fail to load on the
current build. Reconverting fixes them, but needs the source checkpoint on disk.
scripts/ftw_hotfix.pyrepairs the FTW instead, in place or into a new dir with--out:input_scale(KeyError: '...input_scale'). ModelOpt NVFP4 checkpoints with FP8attention (
nvidia/Qwen3.6-27B-NVFP4,RadixArk/Qwen3.8-27B-NVFP4,nvidia/Qwen3.6-35B-A3B-NVFP4): old converters dropped the per-tensorinput_scalescalarsthat the model now declares. Fix: fetch them from the HF repo by byte range (a few KiB per
model) or from a local copy, and append them to the FTW.
KeyError: 'model.embed.weight').deepseek-ai/DeepSeek-V4-Flash-0731:refactor(quant): config, scheme and method layers for quantization #418 changed the tensor names the model declares. Fix: rename the index entries; no data moves.
RuntimeError: Unexpected keys ... .weight_scale).nvidia/GLM-5.2-NVFP4: old FTWs store 547 dense weights as fp8 plus a per-row scale, the modelnow wants them in bf16. Fix: dequantize the weights, drop the scales, rewrite the live entries
into fresh shards.
PLE shard indices are not contiguous 0..N-1: [], refs Can't load Qwen3.8-Flash-Next #328).Qwen3.8-Flash-Next FTWs converted before fix(checkpoint): write the Qwen3.8-Flash-Next PLE table next to the FTW #420. Fix: extract the PLE tensors from the source
checkpoint into
ple-table-*.safetensorsnext to the FTW, as fix(checkpoint): write the Qwen3.8-Flash-Next PLE table next to the FTW #420 does at conversion.The script builds the current model on the meta device from the FTW's
config.json, diffs theindex against the tensors the model declares, and applies only the repairs that are needed. FTWs
that load as is are left untouched.
Not covered: FTWs converted from GGUF, and checkpoints outside
docs/models.md.docs/ftw-hotfix.mddocuments the tool. Nothing underpython/changes.Tested
H100 80GB, driver 580.95.05, torch 2.11.0+cu130, on main 3d919e9. AIME25 via
tests/e2e/test_aime.py.nvidia/Qwen3.6-27B-NVFP4FTW: fails before, passes after the fix.deepseek-ai/DeepSeek-V4-Flash-0731FTW from af71ba4: fails before, passes after the rename.RadixArk/Qwen3.8-Flash-Next-NVFP4FTW without the table: output byte-identical to the raw checkpoint after the fix.