These optimizers adjust the learning rate by the number of optimization steps, which is problematic if `update_on_kv=False` is set and multiple GPUs share the same optimizer object (leading to incorrect count of optimization steps). For example: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/optimizer/optimizer.py#L1093 cc @szha @sandeep-krishnamurthy
These optimizers adjust the learning rate by the number of optimization steps, which is problematic if
update_on_kv=Falseis set and multiple GPUs share the same optimizer object (leading to incorrect count of optimization steps). For example: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/optimizer/optimizer.py#L1093cc @szha @sandeep-krishnamurthy