Jun 22, 2023 · Our experiments show that nonmonotone methods improve the speed of convergence and generalization properties of SGD/Adam even beyond the previous monotone line ...
Nonmonotone stochastic line search methods outperform SGD, Adam and their monotone counterpart for training over-parameterized models.
Our experiments show that nonmono- tone methods improve the speed of convergence and generalization properties of SGD/Adam even beyond the previous monotone ...
May 30, 2024 · Our experiments show that nonmonotone methods improve the speed of convergence and generalization properties of SGD/Adam even beyond the ...
Recent works have shown that line search methods can speed up Stochastic Gradient Descent (SGD) and Adam in modern over-parameterized settings.
Don't be so Monotone: Relaxing Stochastic Line Search in Over ...
publications.rwth-aachen.de › print
|a Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models |h online. 260, _, _, |a [Erscheinungsort nicht ermittelbar] |b [Verlag ...
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models · In Advances in Neural Information Processing Systems 36 (NeurIPS 2023) / ...
May 3, 2024 · Recent works have shown that line search methods can speed up Stochastic Gradient Descent (SGD) and Adam in modern over-parameterized settings.
Jun 26, 2023 · Leonardo Galli's Post · Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models · Explore topics · Sign in to view more ...
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models. L Galli, H Rauhut, M Schmidt. Advances in Neural Information Processing ...