research
∙
07/12/2023
Locally Adaptive Federated Learning via Stochastic Polyak Stepsizes
State-of-the-art federated learning algorithms such as FedAvg require ca...
share
research
∙
06/08/2023
Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates
Distributed and federated learning algorithms and techniques associated ...
research
∙
02/27/2023
Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions
Single-call stochastic extragradient methods, like stochastic past extra...
research
∙
06/12/2022
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
Algorithms designed for single-agent reinforcement learning (RL) general...
research
∙
02/15/2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods
Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent a...
research
∙
11/16/2021
Stochastic Extragradient: General Analysis and Improved Rates
The Stochastic Extragradient (SEG) method is one of the most popular alg...
research
∙
10/28/2021
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize
We investigate the convergence of stochastic mirror descent (SMD) in rel...
research
∙
10/08/2021
Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivity
Extragradient method (EG) Korpelevich [1976] is one of the most popular ...
research
∙
06/30/2021
Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity
Two of the most prominent algorithms for solving unconstrained smooth ga...
research
∙
06/30/2021
On the Convergence of Stochastic Extragradient for Bilinear Games with Restarted Iteration Averaging
We study the stochastic bilinear minimax optimization problem, presentin...
research
∙
02/19/2021
AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods
We present an adaptive stochastic variance reduced method with an implic...
research
∙
07/08/2020
Stochastic Hamiltonian Gradient Methods for Smooth Games
The success of adversarial formulations in machine learning has brought ...
research
∙
06/20/2020
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization
We present a unified theorem for the convergence analysis of stochastic ...
research
∙
06/18/2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation
We provide several convergence theorems for SGD for two large classes of...
research
∙
03/23/2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates
Decentralized stochastic optimization methods have gained a lot of atten...
research
∙
02/24/2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
We propose a stochastic variant of the classical Polyak step-size (Polya...
research
∙
09/26/2019
Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip
In the era of big data, one of the key challenges is the development of ...
research
∙
05/20/2019
Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols
In this work we present a new framework for the analysis and design of r...
research
∙
03/19/2019
Convergence Analysis of Inexact Randomized Iterative Methods
In this paper we present a convergence rate analysis of inexact variants...
research
∙
01/27/2019
SGD: General Analysis and Improved Rates
We propose a general yet simple theorem describing the convergence of SG...
research
∙
01/27/2019
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
In this work we present a randomized gossip algorithm for solving the av...
research
∙
11/27/2018
Stochastic Gradient Push for Distributed Deep Learning
Large mini-batch parallel SGD is commonly used for distributed training ...
research
∙
10/31/2018
Provably Accelerated Randomized Gossip Algorithms
In this work we present novel provably accelerated gossip algorithms for...
research
∙
09/23/2018
Accelerated Gossip via Stochastic Heavy Ball Method
In this paper we show how the stochastic heavy ball method (SHB) -- a po...
research
∙
12/27/2017
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
In this paper we study several classes of stochastic optimization algori...
research
∙
10/30/2017
