Parallelizing stochastic gradient descent for least squares regression: mini-batching, averaging, and model misspecification

The implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in terms of ...

Stochastic gradient descent (SGD) algorithm and its variations have been effectively used to optimize neural network models. However, with the rapid growth of big data and deep learning, SGD is no longer the most suitable choice due to ...

• Its convergence rate of O 1 ρ T + σ 2 shows its applicability for the real-time systems.

This paper presents a new approach to solving the optimization task that arises when L2-SVM in its primal form is considered. In particular, we propose the application of a Barzilai-Borwein (BB) update step in five variants for the classic Stochastic ...