research
∙
08/18/2023
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
Stochastic gradient descent (SGD) performed in an asynchronous manner pl...
share
research
∙
05/25/2023
Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training
Deep learning is experiencing a rise in foundation models that are expec...
research
∙
12/08/2021
SASG: Sparsification with Adaptive Stochastic Gradients for Communication-efficient Distributed Learning
Stochastic optimization algorithms implemented on distributed computing ...
research
∙
10/05/2021
