Repository navigation
Issue #117 Arm-C SWA allocation remediation (InferSwarm #166): lifecycle-owned full->swa mapping in the standalone R6 stage runtime - #34
Merged
Conversation
…): lifecycle-owned full->swa mapping in the standalone R6 stage runtime The standalone stage path built the same HybridSWAKVCache pool as the scheduler but never reached the scheduler's alloc_swa lifecycle (CacheManager.allocate_paged): full_to_swa_index_mapping stayed at the all-zero sentinel, so every SWA-layer KV store raced on swa slot 0 and every prefix read consumed slot-0 bytes (FlashML-org#157 chunk-2 nondeterminism, earliest varying boundary L0_kv_slice_post_write). The stage runtime now owns the session SWA allocation lifecycle at the same points the scheduler uses: - prefill()/decode() (all four role branches) call _ensure_swa_session_mapping BEFORE _prepare (prepare_metadata translates the full page-table range through the mapping -- the first prefix-consuming SWA operation of the chunk); - ownership is an incremental frontier ([0, allocated)) matching CacheManager.allocate_paged's per-chunk granularity; each position's slot is allocated exactly once, mapping persists across chunks/decode; - reset_session_state rebuilds the allocator AFTER the zero loop (the zero loop wipes _swa_free to all-zeros, which would hand out sentinel slot 0 -- the FlashML-org#157-observed trap), releasing all ownership wholesale: reuse cannot inherit stale mapping; - exhaustion and position gaps fail CLOSED (RuntimeError); execution never continues on the sentinel; - non-SWA pools are untouched no-ops; pool sized max_seq_len+1 so a full-capacity session fits (sentinel convention, scheduler parity). No parallel allocator: existing pool primitives reused verbatim. No case/regime/model special cases. 64-row boundary contract, geometry, planner, tokenizer, thresholds unchanged. 35 CPU lifecycle tests with negative controls (skipped allocation, delayed allocation, slot-0 aliasing, stale-mapping-survives-reset, leak, ignored exhaustion, case-gating) in tests/research/test_issue166_swa_remediation.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes InferSwarm FlashML-org#166 (implementation remediation only; NO Arm-C requalification).
Starting heads (verified exact at session start)
55e8baaebabe67aeb967d4bd407ef26696933104(accepted research head)df0365ea606a54222415e34f6cfb127e9f938f0c(main, Merge PR fix(kernel): GGUF JIT extension fails to build with gcc under C++17 - compile as C++20 FlashML-org/FreeToken#159)The defect (FlashML-org#157-localized)
The standalone R6 stage path builds the same HybridSWAKVCache pool as the scheduler but never reaches the scheduler's alloc_swa lifecycle (CacheManager.allocate_paged): full_to_swa_index_mapping stayed at the all-zero sentinel, every SWA-layer KV store raced on swa slot 0, and every prefix read consumed slot-0 bytes (chunk-2 nondeterminism; earliest varying boundary L0_kv_slice_post_write).
The remediation
GemmaDenseStage owns the session SWA allocation lifecycle at the same lifecycle points the scheduler uses:
No parallel allocator (existing pool primitives reused verbatim); no case/regime/model special cases; 64-row boundary contract, geometry, planner, tokenizer, thresholds unchanged.
Validation (CPU-only; no GPU/model execution)
Boundary
Terminal: ISSUE117_ARM_C_SWA_REMEDIATION_READY (InferSwarm record PR). This producer is NOT promoted to qualification authority; fresh Arm-C requalification remains separately blocked pending maintainer acceptance/merge. InferSwarm-side record: Zutfen-LLC/inferswarm branch issue-166-swa-remediation @ e13cf58.