Apr 3, 2024 · Abstract:We prove new convergence rates for a generalized version of stochastic Nesterov acceleration under interpolation conditions.
scholar.google.com › citations
A combination of interpolation and growth conditions has been used to prove fast convergence rates for SGD with line-search [14] , with the stochastic Polyak ...
People also ask
Why does stochastic gradient descent converge faster?
Which gradient descent converges the fastest?
Which is faster gradient descent or stochastic gradient descent?
What are the sufficient conditions for stochastic gradient descent (SGD) to converge?
Apr 3, 2024 · New convergence rates for a generalized version of stochastic Nesterov acceleration under interpolation conditions are proved and are ...
May 29, 2024 · Bibliographic details on Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation.
Apr 3, 2024 · We prove new convergence rates for a generalized version of stochastic Nesterov acceleration under interpolation conditions.
Preprints. Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation. A. Mishkin, M. Pilanci, M. Schmidt. 2024 [arXiv] ; Full Papers.
Under interpolation, we further show that all smooth loss functions with a finite-sum structure satisfy a weaker growth condition. Given this weaker condition, ...
Jan 12, 2021 · Faster convergence rates for SSGD under interpolation condition, which match the non-accelerated bounds for the smooth case. This can shed ...
[PDF] Fast and Faster Convergence of SGD for Over-Parameterized Models
vaswanis.github.io › SR-poster
▷ Under this growth condition, we prove that SGD converges as fast as full-batch gradient descent for (strongly)-convex and non-convex functions. ▷ We show that ...
These considerations demonstrate that (i) SGD shows rapid convergence in the interpolation regime and that (ii) data parallelism should be computationally ...