v0.19.4 Patch Release

What's Changed

• Validate warmup_type in WarmupCosineLR like WarmupLR by @sohumt123 in #8151
• Run pull request code only inside a Modal Sandbox by @tohtana in #8170
• [AutoTP] Allow ZeRO stage 3 inference with tensor parallelism by @delock in #8167
• Fix autotuning get_val_by_key to search all nested subdicts by @ebarkhordar in #8177
• Enable support for Tutel when k != 1 for shared moe by @jinyouzhi in #8174
• NVMe write warning by @sfc-gh-truwase in #8164
• Guard LRRangeTest and OneCycle schedulers against zero step sizes by @ebarkhordar in #8166
• Fix WarmupLR collapsing multi-group base LRs to group 0's by @ebarkhordar in #8171
• Stabilize fork-sensitive nightly tests and AutoSP coverage by @tohtana in #8172
• Make the OneCycle stair counts actually change the schedule by @vineethsaivs in #8179
• [AutoTP] Enable HF colwise_gather_output to support lm_head replace by @jinyouzhi in #8146
• Scope DeepCompile compiler state to graph and engine lifecycles by @tohtana in #8159
• Clarify merge commit exemption for signoff check in AGENTS.md and CLAUDE.md by @jinyouzhi in #8194
• [AutoTP] Preserve universal checkpoint metadata for HuggingFace tp_plan by @jinyouzhi in #8187
• Derive AutoEP rank splits from the per-expert count exchange by @yh0903 in #8190
• Warn when zero.Init silently falls back to a single rank (#8084) by @akshansh47 in #8089
• Release GIL in blocking AIO bindings by @tohtana in #8196
• Enable ZeRO-3 linear wrapper for existing models by @tohtana in #8189
• Skip CUDA transformer forward tests when TransformerBuilder is incompatible by @sfc-gh-truwase in #8200
• Clamp capacity to num_tokens in MoE gating drop branches (complete #5353) by @ebarkhordar in #8155
• Preserve tensor learning rates across scheduler updates by @n33levo in #8202
• [AutoTP] Fix ZeRO-3 checkpoint consolidation to gather across TP and DP by @delock in #8168
• Add managed_gradient_accumulation for ZeRO stage 0/1 by @sfc-gh-truwase in #8184
• MoE dispatch: fix silent Tutel + TP corruption, speed up native and Tutel paths by @jinyouzhi in #8195
• Add keyword argument support to activation checkpointing by @zupengwang in #8182
• Remove redundant DCO workflow by @tohtana in #8188
• Preserve tensor learning rates in OneCycle by @tohtana in #8205
• Add Triton grouped-GEMM for MoE experts on Ampere/Ada by @hwchen2017 in #8180
• Wiring up offload_opt_states by @pengdurice in #8186
• Fix DeepCompile ZeRO-3 gathered parameter ownership by @tohtana in #8157
• Consolidate LR scheduler tensor-LR tests into a contract suite by @delock in #8206
• Fix MLU_Accelerator conformance to the DeepSpeedAccelerator ABC by @ebarkhordar in #8208
• Reduce redundant work in AutoEP token routing by @hwchen2017 in #8209
• Make PipelineModule.set_checkpoint_interval actually change the interval by @vineethsaivs in #8178
• Share DeepNVMe pinned-tensor manager and route swap buffers through I/O handles by @sfc-gh-truwase in #8212
• Stop trim_mean sorting the caller's list in place by @vineethsaivs in #8199
• Unmanaged gradient accumulation: ZeRO stage 2 support by @sfc-gh-truwase in #8203
• Fix ZeRO-3 all_reduce param fetch stride for padded parameters by @ebarkhordar in #8158
• Unmanaged gradient accumulation: ZeRO stage 3 support by @sfc-gh-truwase in #8217
• Update version.txt pre-0.19.4 release by @loadams in #8218

New Contributors

• @sohumt123 made their first contribution in #8151
• @ebarkhordar made their first contribution in #8177
• @yh0903 made their first contribution in #8190
• @akshansh47 made their first contribution in #8089
• @n33levo made their first contribution in #8202
• @zupengwang made their first contribution in #8182
• @pengdurice made their first contribution in #8186

Full Changelog: v0.19.3...v0.19.4

v0.19.3 Patch Release

What's Changed

• Update version post release by @loadams in #8070
• Validate fp16 dynamic loss scaling parameters are positive by @aryanputta in #8050
• Add merge queue DCO workflow by @tohtana in #8086
• Stop obsolete CI jobs on workflow cancellation by @tohtana in #8034
• Fix ZeRO-3: Use per-param dtype for output buffers in _allgather_params_coalesced by @albertvillanova in #8073
• fix: close file descriptor in deepspeed_io_handle_t::wait() to prevent fd leak by @MarkCLChang in #8075
• Add type hints to top-level public API functions by @arunshar in #8079
• Default gradient_clipping to 1.0 by @sfc-gh-truwase in #8068
• activation_checkpointing: default num_layers to None so configure() assert fires by @Kymi808 in #8041
• Add configurable engine log level by @sfc-gh-truwase in #8067
• [CI] diff driven test selection by @stas00 in #8077
• Enable bf16 check_grad_overflow by default (matching fp16) by @yongzhe-wang in #8035
• Support AutoEP with ZeRO-3 zero.Init source modules by @tohtana in #8060
• Update publication list in README.md by @PKUWZP in #8095
• fix: AutoTP partition_config uses full hierarchical module path by @delock in #8088
• Reject Muon optimizer with reduce_scatter in ZeRO-1/2 by @whycoming in #8090
• Fix DeepCompile profile metadata backfill by @tohtana in #8094
• Add numerical-correctness test for Muon under ZeRO-1/2 by @whycoming in #8091
• Don't swallow KeyboardInterrupt/SystemExit in NPUOpBuilder by @ajinkyajawale14499 in #8096
• Avoid CUDA context initialization during op compatibility checks at import by @Achyuthan-S in #8078
• [DeepCompile] fix gather params in dynamo skipped frames for ZeRO3 by @XAheli in #8059
• Feat: zero3 deprecate elastic checkpoint by @nathon-lee in #8099
• Fix eigenvalue parsing for compression-only quantize configs by @sowndappan5 in #8057
• fix: log eigenvalue monitor values by @he-yufeng in #8049
• Fix DeepCompile profiling memory cleanup by @tohtana in #8106
• Add optional torchembed RoPE backend to apply_rotary_pos_emb by @py-ai-dev in #8052
• Fix minor comment/docstring typos in runtime and inference modules by @nathon-lee in #8046
• feat(zenflow): run the overlapped CPU optimizer in a native process by @Antlera in #8058
• fix: add buffer-length check in shm.cpp by @orbisai0security in #8082
• Fix FlopsProfiler crash when dp_world_size is None under sequence parallelism by @vineethsaivs in #8122
• Fix Python 3.9 import-time TypeError in AutoEP ep_router by @vineethsaivs in #8119
• Add input validation for save_checkpoint and warmup LR schedulers by @jahnavi-yelamanchi in #8126
• Fix ZeRO-3: use per-param dtype for quantize-scale buffer in _allgather_params_coalesced by @roycho96 in #8128
• Fix ZeRO-3 autocast gather with mixed parameter dtypes by @tohtana in #8113
• doc: Remove suggestion to build extensions in parallel by @Flamefire in #7899
• feat(types): add type hints to public comm API functions by @NovusEdge in #8118
• Add Hybrid Engine rollout in DeepSpeed to support On-Policy Distillation (OPSD) Trainer by @PKUWZP in #8027
• Add AutoEP + AutoTP parallel folding by @tohtana in #8064
• Make DCO workflow Probot compatible by @tohtana in #8110
• ZeRO 1/2: wait on all IPG-bucket producer streams in average_tensor (#8061) by @arunshar in #8080
• Fix CommsLogger.stop_profiling_comms not disabling global profiling by @vineethsaivs in #8137
• Guard WarmupCosineLR against total_num_steps == warmup_num_steps (ZeroDivisionError) by @vineethsaivs in #8142
• Add EXAONE 4.5 model support for Inference V2 by @Bias92 in #8121
• fix(autotp): when using autotp, ignore the consistency of certain data within the tp_group. by @1787648106 in #8125
• Reduce blockDim in fake_quantize_kernel for improved SM occupancy by @flutist in #8115
• Fix DeepSpeedInferenceConfig crash on bool moe backward-compat value by @winklemad in #8145
• fix(exceptions): narrow broad Exception clauses to specific types by @sowndappan5 in #8136
• fix(muon): support ZeRO-1/2 reduce scatter by @Micuks in #8141
• [DeepCompile] Add lightweight pass contracts for optimization passes by @jahnavi-yelamanchi in #8139
• Fix DeepCompile fallback handling for scalar outputs by @tohtana in #8149
• Use synthetic CIFAR data in AlexNet tests by @tohtana in #8143
• Update CI badge by @tohtana in #8150
• Update Zhipeng Wang affiliation to Google in COMMITTERS.md by @PKUWZP in #8153
• Fix ZeRO-3 hooks for attribute-delegating modules by @Functionhx in #8144
• Fix repeated gradient scaling across pipeline stages by @Functionhx in #8154
• Z3: Support for activation ckpt with frozen params by @sfc-gh-truwase in #8148
• Adopt PTF Code of Conduct by @sfc-gh-truwase in #8165

New Contributors

• @aryanputta made their first contribution in #8050
• @albertvillanova made their first contribution in #8073
• @MarkCLChang made their first contribution in #8075
• @arunshar made their first contribution in #8079
• @Kymi808 made their first contribution in #8041
• @yongzhe-wang made their first contribution in #8035
• @whycoming made their first contribution in #8090
• @ajinkyajawale14499 made their first contribution in #8096
• @Achyuthan-S made their first contribution in #8078
• @XAheli made their first contribution in #8059
• @he-yufeng made their first contribution in #8049
• @py-ai-dev made their first contribution in #8052
• @vineethsaivs made their first contribution in #8122
• @jahnavi-yelamanchi made their first contribution in #8126
• @NovusEdge made their first contribution in #8118
• @1787648106 made their first contribution in #8125
• @winklemad made their first contribution in #8145
• @Micuks made their first contribution in #8141
• @Functionhx made their first contribution in #8144

Full Changelog: v0.19.2...v0.19.3

v0.19.2 Patch Release

What's Changed

• fix(fp16): filter requires_grad in FP16 optimizer flat buffer init by @avicooper1 in #8029
• Run AutoSP compile tests sequentially by @tohtana in #8020
• Fix PR-target workflow concurrency groups by @tohtana in #8017
• Fix full CI test isolation for ZeRO chmod and NVMe quantization tests by @tohtana in #8008
• Keep required CI checks visible for ignored paths by @tohtana in #8019
• Bump version by @sfc-gh-truwase in #8030
• Add engine.coalesce_grad_reduction() for ZeRO 1/2/3 multi-backward by @roycho96 in #7992
• feat(zero): enable torch.func transforms on engine for ZeRO 0/1/2 by @roycho96 in #8026
• Simplify module_inject.transpose by @xbcReal in #8028
• Fix DeepCompile all-gather scheduler candidate selection by @tohtana in #8033
• Version fix to unblock pypi by @sfc-gh-truwase in #8039
• Bump version after 0.19.1 release by @tohtana in #8040
• Fix DeepCompile ZeRO-3 release parameter lifetime by @tohtana in #8032
• Fix ZenFlow ZeRO-3 selective optimizer crash with parameter offload on nvme by @Antlera in #8042
• Add test coverage for Muon muon_lr/adam_lr overrides by @sowndappan5 in #8047
• Avoid HF Hub access in CPU unit test setup by @tohtana in #8053
• Fix DeepCompile ZeRO-1 grad target lifetime by @tohtana in #8036
• Normalize ZeRO-3 DeepCompile grad dtype before reduction by @tohtana in #8038
• Remove AutoSP assertion against Transformers version by @tohtana in #8044
• fix(transformer): use correct stride in Transpose_Kernel shared memory indexing to eliminate bank conflicts by @flutist in #8055
• zero3: invalidate coordinator trace on hook re-registration by @roycho96 in #8043
• Consistent fp32 grads flow by @sfc-gh-truwase in #8056
• Add AutoEP by @tohtana in #7938
• Fix: ZenFlow Adam integration for updated PyTorch backward flow (#7759) by @Antlera in #7771
• Pass expected grad dtype to register_z3_param in ZeRO-3 release test by @tohtana in #8063
• Add Biren SUPA accelerator support by @frozenleaves in #8054
• Mixed-precision: per-policy param/buffer dtype cast (preserve fp32 buffers) by @sfc-gh-truwase in #8066

New Contributors

• @avicooper1 made their first contribution in #8029
• @sowndappan5 made their first contribution in #8047
• @flutist made their first contribution in #8055
• @frozenleaves made their first contribution in #8054

Full Changelog: v0.19.1...v0.19.2

v0.19.1

What's Changed

• Update version post-v0.19.0 release by @loadams in #7996
• Add office hours times/link on the README by @loadams in #8004
• Update topkgating probability-mask test expectation by @tohtana in #8007
• Optimize singleton MoE collectives by @Tianyi-Franklin-Wang in #7997
• zero3: SDMA allgather via mori (sdma_allgather) by @inkcherry in #7999
• fix(io): close aio_fd in FastFileWriter._fini to prevent fd leak by @jg-heo in #8005
• Auto-detect CUTLASS for EvoformerAttention by @MaxTretikov in #8000
• fix: use subprocess instead of os.system in data_analyzer.py by @orbisai0security in #7994
• Fix ZeRO-3 forward crash on modules with plain dict _parameters by @roycho96 in #8009
• Remove stale step() docstring from DeepSpeedCPUAdam by @lucaspirola in #8011
• Add configurable torch-latest dependency versions by @tohtana in #8016
• Run FastFileWriter fd-close test outside pytest-forked by @tohtana in #8015
• Make GitHub Actions job names unique by @tohtana in #8014
• Support bf16 optimizer states with CPU offload by @lucaspirola in #8010
• [fix] fix test_zf.py hang bug by @xbcReal in #8012
• [Blog] Muon Optimizer Support in DeepSpeed by @delock in #7962
• fix gemma4 num attention head bugs (from #7975) by @delock in #7990
• fix: add setup_context for torch.func compatibility by @roycho96 in #7916
• Sort and dedupe -gencode flags emitted by op_builder.builder by @adityasingh2400 in #8021
• fix(zero): enable vmap on LinearFunctionForZeroStage3 by @roycho96 in #8023
• Support flash-attn 2.7.0 in FPDT attention by @xbcReal in #8022
• Fix DeepCompile AOT kwargs patching for PyTorch >= v2.11 by @tohtana in #8024

New Contributors

• @Tianyi-Franklin-Wang made their first contribution in #7997
• @jg-heo made their first contribution in #8005
• @MaxTretikov made their first contribution in #8000
• @orbisai0security made their first contribution in #7994
• @lucaspirola made their first contribution in #8011
• @xbcReal made their first contribution in #8012
• @adityasingh2400 made their first contribution in #8021

Full Changelog: v0.19.0...v0.19.1

v0.19.0

What's Changed

• Update version after latest release (v0.18.9) by @loadams in #7936
• Refactor consolidate transpose by @nathon-lee in #7934
• Fix/fix autotp universal checkpoint ci by @tohtana in #7937
• Fix process hang in process-group shutdown by @Flamefire in #7941
• Zero3 defragment utility by @nathon-lee in #7940
• [SP] add SP deny list instead of allow by @kashif in #7887
• fix(zero): detach flat buffer to prevent autograd inplace error on CP… by @delock in #7948
• Fix FPQuantizer build by @Flamefire in #7963
• Fix zero 1 and 2 CPU-offloaded gradient norm by @alek6kun in #7967
• Fix overlap-comm buffer lifetimes by @tohtana in #7965
• Fix DeepCompile+Z3 on PyTorch v2.9/2.10 by @tohtana in #7951
• Fix WarmupCosineLR multi-group initialization by @tohtana in #7969
• Enable PyTorch version selection for full test by @tohtana in #7968
• fix(fp_quantizer): fix UB and negative shift warnings in fp_quantize_impl.cu by @Cursx in #7973
• fix(op_builder): avoid duplicate/wrong -gencode flags by @Cursx in #7974
• Rename dequantization template parameters by @Flamefire in #7976
• Avoid CUDA reinit error in CI tests by @tohtana in #7977
• Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per step by @roycho96 in #7981
• deepcompile: Fix backward graph recompilation due to unbalanced forward/backward visits by @eternalNight in #7980
• Fix Adam subgroup inconsistency by @st-bang97 in #7982
• Dynamic offload compatible with static optimizer offload by @sfc-gh-truwase in #7979
• Fix modal ci timeout by @sfc-gh-truwase in #7989
• Fix BF16_Optimizer last-microbatch grad leak under ZeRO-1 by @maxyu1115 in #7985
• fix: topkgating major bug by @excepshenal in #7986
• Add DeepSpeed NVTX domain support by @heurry in #7988
• Add Gram Newton-Schulz orthogonalization for Muon optimizer by @delock in #7953
• [AutoSP] (Sequence Parallelism) support for Multimodal Models (ViT + LLM) by @nathon-lee in #7984
• Update version.txt before 0.19.0 release by @loadams in #7995

New Contributors

• @alek6kun made their first contribution in #7967
• @Cursx made their first contribution in #7973
• @roycho96 made their first contribution in #7981
• @st-bang97 made their first contribution in #7982
• @maxyu1115 made their first contribution in #7985
• @excepshenal made their first contribution in #7986
• @heurry made their first contribution in #7988

Full Changelog: v0.18.9...v0.19.0

v0.18.9 Patch Release

What's Changed

• Respect $TRITON_HOME by @Flamefire in #7907
• Add Feature Universal Checkpoint for AutoTP by @nathon-lee in #7908
• fix: remove unnecessary shell=True in ROCm GPU architecture detection by @instantraaamen in #7915
• Don't detect local GPU if $DS_IGNORE_CUDA_DETECTION is set by @Flamefire in #7896
• Add HuggingFace tp_plan support for AutoTP by @delock in #7901
• fix: handle non-existent path in is_nfs_path for Triton autotune cache by @Krishnachaitanyakc in #7921
• Fix backward compatibility of torch.amp.custom_fwd for PyTorch < 2.4 by @tohtana in #7920
• Extending Muon Optimizer Support for ZeRO Stage 3 by @PKUWZP in #7919
• Add news item for ASPLOS 2026 Best Paper Award by @PKUWZP in #7923
• fix(superoffload) preserve multi-group updates with shared cpu buffers (#7905) by @xylian86 in #7906
• AGENTS.md: Add pre-commit command to existing CI requirements line by @delock in #7930
• Update README with latest news from DeepSpeed by @PKUWZP in #7931
• Merging AutoSP into DeepSpeed by @neeldani in #7860
• Add fallback to full test by @tohtana in #7933
• Remove Microsoft Corporation copyright from AGENTS.md and CLAUDE.md by @PKUWZP in #7932
• Update version.txt for latest incoming release 0.18.9 by @loadams in #7935

New Contributors

• @instantraaamen made their first contribution in #7915
• @Krishnachaitanyakc made their first contribution in #7921
• @neeldani made their first contribution in #7860

Full Changelog: v0.18.8...v0.18.9

v0.18.8 Patch Release

What's Changed

• Suppress see_memory_usage logs by @sfc-gh-truwase in #7891
• [Bloom] Fix hangs of bloom test by @k-artem in #7890
• double reduction user-friendly error by @stas00 in #7895
• Fix async_io ops building error on Huawei Ascend NPU by @huangyifan0610 in #7894
• Fix Evoformer's multi-arch dispatch root cause by @tohtana in #7881
• fix: Validate fp16.loss_scale is finite and non-negative by @nathon-lee in #7889
• Add AGENTS.md and CLAUDE.md with project rules for AI coding agents by @delock in #7902
• fix(zero3): use current_stream() instead of default_stream() for grad… by @michaelroyzen in #7898
• Update version by @loadams in #7903

New Contributors

• @huangyifan0610 made their first contribution in #7894
• @michaelroyzen made their first contribution in #7898

Full Changelog: v0.18.7...v0.18.8

v0.18.7 Patch Release

What's Changed

• Update version post release by @loadams in #7850
• Z1/2 init: flatten params on device by @ksugama in #7828
• Enable shm_comm support for arm by @phalani-paladugu in #7800
• Add news entry for DeepSpeed updates by @PKUWZP in #7854
• Add EXAONE 4.0 model support for Inference V2 by @Bias92 in #7853
• Fix ROCm BF16 conversion intrinsics in inference v2 (#7843) by @tohtana in #7846
• Fix compilation of Evoformer by @Flamefire in #7862
• Throw error when parameter is modified in GatheredParameters by @tohtana in #7832
• Fix Zero-3 static scale assertion in fp16 test by @tohtana in #7866
• Schedule nightly full test by @tohtana in #7870
• Fix broken links and add AutoTP Training tutorial to sidebar nav by @tohtana in #7874
• fix: replace 35 bare except clauses with except Exception by @haosenwang1018 in #7873
• perf: use deque for FIFO queues in sequence parallel, superoffload, and compile by @giulio-leone in #7880
• Fix: only add parameter with grads to parameter group by @delock in #7869
• Fix no-grad grad-fn lookup in ZeRO hook counting on PyTorch 2.3 (#7830) by @tohtana in #7841
• Fix import deepspeed crash on PyTorch v2.3 + Python 3.12 by @tohtana in #7875
• XPU use stock pytorch instead of Intel Extension for PyTorch by @delock in #7877
• Remove amp() from abstract accelerator by @delock in #7879
• Add document section explaining autocast nesting by @tohtana in #7883
• Fix hook count performance regression from v0.18.5 by @tohtana in #7886

New Contributors

• @ksugama made their first contribution in #7828
• @phalani-paladugu made their first contribution in #7800
• @Bias92 made their first contribution in #7853
• @haosenwang1018 made their first contribution in #7873
• @giulio-leone made their first contribution in #7880

Full Changelog: v0.18.6...v0.18.7

v0.18.6 Patch Release

What's Changed

• Update version.txt to 0.18.6 after latest release by @loadams in #7826
• Fix leaf module race condition by @tohtana in #7825
• Skip sequence parallel operations during eval by @jp1924 in #7821
• Support custom partitioning patterns for AutoTP by @tohtana in #7806
• Fix gradient is ready with z2 by @sfc-gh-truwase in #7829
• Fix AutoTP custom patterns: respect use_default_specs by @tohtana in #7827
• Support new python 3.14 annotation handling by @sdvillal in #7831
• fix: replace deprecated fractions.gcd with math.gcd by @Mr-Neutr0n in #7845
• Fix bf16 gradient norm divergence with ZeRO stage 0 by @tohtana in #7839
• Replace torch.jit.script with torch.compile (#7835) by @tohtana in #7840

New Contributors

• @jp1924 made their first contribution in #7821
• @Mr-Neutr0n made their first contribution in #7845

Full Changelog: v0.18.5...v0.18.6

v0.18.5 Patch Release

What's Changed

• Update version.txt after 0.18.4 release by @loadams in #7765
• Various fixes to run on mps by @jeffra in #7767
• Udpate workflow trigger by @tohtana in #7768
• fix: delete using namespace std. by @nathon-lee in #7766
• fix: update Megatron-DeepSpeed tutorial to match current repo structure by @nathon-lee in #7761
• Add timeout to test workflows by @tohtana in #7774
• Remove cron/PR triggers for outdated V100 tests by @loadams in #7777
• [Docs] Fix docs/_pages/config-json.md format by @ooooo-create in #7779
• Update CLA to refer to DCO by @loadams in #7778
• Fix multiprocessing testcase by @k-artem in #7743
• fix: skip compressed allreduce for empty tensors by @T1mn in #7769
• docs: update README.md by @eltociear in #7781
• Fix gradient checkpointing with use_reentrant=True / PyTorch-style backward / ZeRO-3 by @tohtana in #7780
• Fix Ulysses PEFT test by @tohtana in #7784
• Fix Evoformer compilation by @sdvillal in #7760
• fix checkpointing/loading of z0+bf16 by @tohtana in #7786
• Add sequential allgather optimization for ZeRO-3 by @aeeeeeep in #7661
• Fix AutoTP test numerical tolerance with rtol by @tohtana in #7794
• Fix backward for pipeline engine by @tohtana in #7787
• Skip empty parameters in gradient reduction by @tohtana in #7789
• Fix issue with BF16 optimizer selection by @tohtana in #7788
• Fix BF16_Optimizer being used without ZeRO by @tohtana in #7790
• Add full test suite workflow by @tohtana in #7795
• Fix Muon optimizer module path by @tohtana in #7802
• Fix ping-pong buffer index reset and removing redundant stream sync by @undersilence in #7805
• Fix ZeRO stage to choose BF16 optimizer in test by @tohtana in #7803
• Run Evoformer tests sequentially by @tohtana in #7810
• Improve engine's cleanup by @tohtana in #7813
• Ignore evoformer test by @tohtana in #7815
• Fix typos in accelerator setup guide by @nathon-lee in #7818
• Raise clear error on in-place GatheredParameters edits without modifier_rank by @tohtana in #7817
• [Bugfix] Resolve Rank index out of range during BWD when sp_size < world_size in Ulysses by @Flink-ddd in #7809
• Update PyTorch to v2.9 for modal tests by @tohtana in #7816

New Contributors

• @ooooo-create made their first contribution in #7779
• @T1mn made their first contribution in #7769
• @sdvillal made their first contribution in #7760
• @undersilence made their first contribution in #7805

Full Changelog: v0.18.4...v0.18.5