[WIP]feat(npu): add Ascend FP8/MXFP8 RLVR support and Qwen3.5 MoE reload

[UsernameFull]

Summary

This PR adds end-to-end Ascend NPU FP8 support for ROLL RLVR pipelines, including Megatron FP8 training, vLLM-Ascend online MXFP8 quantization, ModelSlim checkpoint loading, and Qwen3.5 MoE weight reload.

The changes target the following dependency stack:

• CANN 9.1
• Ascend 950
• PyTorch 2.10
• torch-npu 2.10
• Megatron-LM core_r0.17.0
• MegatronAdaptor core_r0.17.0
• vLLM v0.23.0
• vLLM-Ascend v0.23.0rc1

Motivation

ROLL previously lacked a complete Ascend FP8 training and rollout path.

This PR enables the following workflows:

1. Megatron training with BF16/FP16 master parameters and FP8 computation.
2. Online conversion of synchronized actor weights to Ascend MXFP8 for vLLM rollout.
3. Training from ModelSlim pre-quantized MXFP8 checkpoints.
4. Correct Qwen3.5 MoE weight reload with TP and EP enabled.

Main changes

Ascend FP8 runtime

• Add centralized NPU runtime initialization.
• Integrate MegatronAdaptor and TransformerEngineNPU initialization.
• Propagate FP8 format and recipe settings into Megatron.
• Add explicit validation for unsupported or incomplete FP8 configurations.
• Keep FP8 execution enabled instead of falling back to BF16.

vLLM online MXFP8 quantization

• Add online_quantization: ascend_mxfp8.
• Generate ModelSlim-compatible quantization descriptions.
• Quantize synchronized floating-point weight shards with Ascend dynamic MXFP8 quantization.
• Preserve parameter identity across repeated weight updates.
• Support dense Qwen and Qwen MoE models.

Qwen3.5 MoE reload

• Handle Ascend-specific transposed parameter layouts.
• Apply TP sharding before loading weights.
• Map global expert IDs to local experts under EP.
• Avoid duplicate transposition during repeated reloads.
• Keep router and non-quantized parameters in floating-point format.

ModelSlim checkpoint loading

• Detect Ascend MXFP8 metadata from checkpoint files.
• Match quantized weights with their scale tensors.
• Reject missing or unsupported quantization metadata.
• Require a trainable MindSpeed/TransformerEngineNPU loader instead of silently dequantizing to BF16.

Runtime stability

• Initialize and offload reference workers before vLLM reserves its KV cache on colocated devices.
• Skip unnecessary tensor-parallel collectives when TP size is 1.
• Keep the TP>1 entropy reduction path unchanged.
• Avoid repeatedly compiling the entropy reduction helper.
• Limit repeated fused-kernel fallback warnings.

Docker and examples

• Add docker/Dockerfile.A5 for the CANN 9.1 / Ascend 950 environment.
• Use the local repository as the Docker build source.
• Add BF16 and FP8/MXFP8 four-NPU RLVR example configurations.

Tests

Added focused tests for:

• Ascend MXFP8 checkpoint metadata and weight/scale matching.
• FP8 configuration validation.
• Online quantization description generation.
• Repeated vLLM weight updates.
• Qwen3.5 MoE TP/EP weight mapping.
• TP=1 entropy without distributed collectives.
• TP>1 entropy collective behavior.

Validation performed

• git diff --check
• Qwen3-30B-A3B FP8/MXFP8 pipeline started successfully with TP=1 and EP=8
• Training reached step 2
• Rollout output was inspected and contained no garbled text
• Branch remains a single commit on top of alibaba/main
• Newly added unit tests were not executed locally because no usable Python environment is installed
• docker build was not executed locally because the Docker Desktop Linux engine was unavailable

Known limitations

• TP=2 with optimizer CPU offload can exceed available pinned host memory in the tested environment.
• Switching optimizer offload to pageable memory avoids the pinned allocation failure but can trigger Ray host-memory pressure.
• The currently validated eight-NPU configuration uses TP=1 and EP=8.
• This implementation intentionally targets the dependency versions listed above and does not preserve compatibility with older vLLM/vLLM-Ascend releases.

Compatibility

Existing non-NPU and non-FP8 paths are intended to remain unchanged. Ascend-specific patches are installed only when the NPU runtime and corresponding quantization mode are active.