BLOOM Inference via DeepSpeed-Inference, Accelerate and DeepSpeed-ZeRO - #308

Merged

stas00 merged 58 commits into

mainfrom

bloom-inference

Aug 10, 2022

Commits on Jul 10, 2022

hardcode the dtype depending on the model
stas00
committed
efa354d
change the mp based on the world_size
Reza Yazdani
committed
cafc3f5
remove hardcoded world_size
stas00
committed
daeb293
add bigscience/bigscience-small-testing
stas00
committed
7d5f7d4
Merge branch 'bloom-inference' of https://github.com/bigscience-workshop/Megatron-DeepSpeed into bloom-inference
Reza Yazdani
committed
2d3d271
fixes
stas00
committed
1ff0f69
add zero-inference script
stas00
committed
56b24ed

Commits on Jul 11, 2022

fixes
stas00
committed
67aab37
fix
stas00
committed
328ab0c

Commits on Jul 12, 2022

Commits on Jul 13, 2022

fix for offline use
stas00
committed
6c5c23b
add benchmark
stas00
committed
6b19227
add benchmark
stas00
committed
10cbb2d
update
stas00
committed
494c212
cleanup
stas00
committed
2b67c0d
update
stas00
committed
3853724
msecs
stas00
committed
1896739
cleanup
stas00
committed
7c9daaf
improve
stas00
committed
dca2c8f
fix benchmark, add warmup
stas00
committed
85580c0
update
stas00
committed
5ea3dee
fix; thanks Michael Wyatt
stas00
committed
737c681
clarify
stas00
committed
6be0cca
Merge branch 'bloom-inference' of https://github.com/bigscience-workshop/Megatron-DeepSpeed into bloom-inference
Reza Yazdani
committed
fea3902
add bloom batch-inference script
Reza Yazdani
committed
fc9b458
removed the names :-)
Reza Yazdani
committed
7b0edef
fold the bs functionality from the other script
stas00
committed
2120dd2
fix
stas00
committed
78bcbb7
restore do_sample
stas00
committed
e7468cd
dump generate args
stas00
committed
68f5ca6

Commits on Jul 14, 2022

fix
stas00
committed
1eca7c5
fix
stas00
committed
8815fc3
support any batchsize
stas00
committed
034cc6f
div by bs
stas00
committed
155c3c3
mul by bs
stas00
committed
73a8b7b
add cpu_offload; sync scripts
stas00
committed
09d7408
wip
stas00
committed
695265d

Commits on Jul 15, 2022

improvements
stas00
committed
1a7e891
fixes
stas00
committed
aba4055
fixes
stas00
committed
5e92d55
add accelerate script
stas00
committed
3992112
fix
stas00
committed
5a7057b

Commits on Jul 16, 2022

wip
stas00
committed
4758531
wip
stas00
committed
7550ee0

Commits on Jul 18, 2022

stats
stas00
committed
5153c40

Commits on Jul 19, 2022

add OnDevice and remove zero-inference (#316)
jeffra
authored
cb50ea5
wip
stas00
committed
a53fcaa
rework generate + benchmark
stas00
committed
7252879
figure out the memory map dynamically
stas00
committed
2aa419d
bug fix
stas00
committed
4bd8ca5
fix ds-zero-inference wrt device
stas00
committed
b76e516

Commits on Jul 20, 2022

bug fix
stas00
committed
ecfd577
update
stas00
committed
fd26b9c

Commits on Jul 22, 2022

update
stas00
committed
e2bfe91

Commits on Aug 9, 2022

fix
stas00
committed
b9a67ea
Merge remote-tracking branch 'origin/main' into bloom-inference
stas00
committed
3862ef0