In this paper we present a simple method, based on opposite transfer functions which greatly improve the convergence rate and accuracy of gradient-based ...

In this paper we present a simple method, based on opposite transfer functions which greatly improve the convergence rate and accuracy of gradient-based ...

Apr 7, 2016 · Implement some adaptive learning rate algorithm such as adadelta, adagrad or rmsprop. Finally, a last thing you may want to try is batch normalization.

Numerical condition affects the learning speed and accuracy of most artificial neural network learning algorithms. In this paper, we examine the influence ...

Nov 9, 2021 · In this work, we propose a novel method named short circuit to enhance gradient learning in deep neural networks.

We propose efficient numerical schemes for implementing the natural gradient descent (NGD) for a broad range of metric spaces with applications to PDE-based ...

The improved backpropagation algorithm helps alleviating the problem of slow convergence and oscillations. The analysis indicates that the backpropagation with ...

Improving the training efficiency of neural network based algorithms is an active area of research and numerous papers have been proposed in the literature.

Missing: scale | Show results with:scale

This chapter is meant as a practical guide with recommendations for some of the most commonly used hyperparameters, in particular in the context of learning ...