DeepSpeed Comm. Backend v1 - #1985

Merged

jeffra merged 24 commits into

staging-comms-next

Jun 10, 2022

Commits on May 27, 2022

Staging comms v1 (#301)

authored
867a853
Delete stage1.py
awan-10
authored
c93fcfe
Delete distributed.py
awan-10
authored
7f8ca01

Commits on May 31, 2022

Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1
Quentin-Anthony
committed
9976681
Simplify ZeRO stage 1 check for previous commit
Quentin-Anthony
committed
09063a3
Remove misleading world_size prints
Quentin-Anthony
committed
656b415

Commits on Jun 1, 2022

Remove custom backend references
Quentin-Anthony
committed
859c5f8
Merge branch 'master' into staging-comms-next
jeffra
authored
4d710b4
Delete nccl.cpp
Quentin-Anthony
authored
108064e
Delete alltoall.py
Quentin-Anthony
authored
ce5001f
Delete runall.py
Quentin-Anthony
authored
73afaa8
Delete all_reduce.py
Quentin-Anthony
authored
e51021f

Commits on Jun 7, 2022

Merge branch 'master' into staging-comms-next
jeffra
authored
6678344
Add isend/irecv comm ops
Quentin-Anthony
committed
604eae7

Commits on Jun 8, 2022

Add pre-commit check and clean up references to torch.dist in comments
Quentin-Anthony
committed
2dd23f3
Updated pre-commit error
Quentin-Anthony
committed
b902744

Commits on Jun 9, 2022

Clean up comm.py logic
Quentin-Anthony
committed
3c5e065
Added comm constants file to avoid raw string assignment
Quentin-Anthony
committed
971c349
Revert "Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1"
Quentin-Anthony
committed
5e0c8c1