Qwen3.6-27B MTP is unsafe with prefix caching enabled

[j316chuck]

Summary

SkyRL currently permits the following combination without a warning or guard:

• Qwen3.6-27B (hybrid GDN model)
• vLLM MTP speculative decoding
• prefix caching enabled

This combination has an open upstream vLLM correctness issue:

• [Bug]: Accuracy drops ~20% when --enable-prefix-caching is used together with MTP speculative decoding (Qwen3.6 35B-A3B) vllm-project/vllm#43559

Prefix caching defaults to true in InferenceEngineConfig, while _apply_mtp_config() independently installs the vLLM MTP speculative config. There is no model/version compatibility guard. The existing Qwen3.5 MTP recipe also enables MTP without explicitly disabling prefix caching.

Observed Qwen3.6-27B regression

On a fixed-weight Tau Retail comparison (20 tasks × 8 rollouts, BF16 model weights, FP8 KV cache), only the MTP + prefix-cache arms collapsed:

Configuration	Accuracy	Reward	Mean steps
MTP-0 baseline	25.63%	33.13%	7.76
MTP-1 + prefix cache	8.75%	12.19%	4.89
MTP-1 + prefix cache, synchronous scheduling	6.88%	7.19%	3.94
MTP-1, prefix cache disabled	26.88%	34.69%	7.65
MTP-2, prefix cache disabled	31.25%	37.81%	7.68

Disabling async scheduling did not recover accuracy. Disabling prefix caching did.

This points to the hybrid GDN recurrent-state/cache-boundary path, rather than speculative acceptance generally.

Why this matters

For hybrid GDN models, recurrent state must correspond exactly to the accepted-token boundary. Prefix-cache reuse and speculative decoding can otherwise restore or retain state from the wrong boundary.

The expected four-way behavior is:

1. Plain decoding: correct
2. Prefix caching only: correct
3. MTP only: correct
4. MTP + prefix caching: currently regresses accuracy/corrupts generation

The proposed upstream boundary fix is small:

• [Bugfix][Core] MTP + enable prefix caching + mamba accuracy fix vllm-project/vllm#43650

However, other GDN speculative-state fixes are also relevant:

• async accepted-token state: [Bugfix] Avoid racy accepted counts in async spec decode vllm-project/vllm#45100
• GDN MTP scheduling/state correctness: [Bugfix] Fix PD disagg + MTP correctness for Qwen3.5(GDN) vllm-project/vllm#47466
• accepted-token index CUDA crashes: [Bugfix] Bound two num_accepted-derived indices in the GDN spec-decode path (MTP + hybrid GDN CUDA illegal access) vllm-project/vllm#50021

Therefore the prefix-boundary patch alone should not be treated as proof that the complete combination is production-safe.

Clean reproduction

Run the same Qwen3.6-27B checkpoint, data, seed, precision, and deterministic eval prompts in this 2×2 matrix:

Configuration	MTP speculative decoding	enable_prefix_caching
Plain	off	false
APC only	off	true
MTP only	on	false
MTP + APC	on	true

For a SkyRL config using the high-level MTP knob:

COMMON=(
  trainer.eval_before_train=true
  trainer.max_training_steps=1
  generator.eval_n_samples_per_prompt=2
  generator.eval_sampling_params.temperature=0
)

# Plain
SKYRL_DISABLE_SPEC=1 ... "${COMMON[@]}" \
  generator.inference_engine.enable_prefix_caching=false

# APC only
SKYRL_DISABLE_SPEC=1 ... "${COMMON[@]}" \
  generator.inference_engine.enable_prefix_caching=true

# MTP only
... "${COMMON[@]}" \
  generator.inference_engine.enable_prefix_caching=false

# MTP + APC
... "${COMMON[@]}" \
  generator.inference_engine.enable_prefix_caching=true

Required checks:

• Confirm APC arms have nonzero vllm/prefix_cache_hit_rate; otherwise cache reuse was not exercised.
• Compare deterministic outputs, reward/accuracy, generated length, episode depth, MTP acceptance, and invalid/early-EOS responses.
• Repeat with an A → B → A long-shared-prefix pattern, since stable A → A hits may not reproduce a poisoned-state transition.
• Run at least 100 cache-transition requests to catch CUDA faults.

Expected result: MTP + APC is output-equivalent to MTP-only at temperature 0.

Suggested immediate mitigation

For Qwen3.6-27B, explicitly disable prefix caching whenever MTP is enabled:

generator.inference_engine.enable_prefix_caching=false

Also consider warning or failing when SkyRL detects all three conditions:

• hybrid GDN/Qwen3.5/Qwen3.6 model
• speculative decoding enabled
• prefix caching enabled

The guard should be removed only after SkyRL pins and validates a vLLM build containing the necessary prefix-boundary and accepted-state fixes.