The vanishing gradient problem during learning recurrent neural nets and problem solutions

Conventional recurrent neural networks (RNNs) have difficulties in learning long-term dependencies. To tackle this problem, we propose an architecture called segmented-memory recurrent neural network (SMRNN). A symbolic sequence is broken into segments ...

Recurrent neural networks (RNN) unfolded in time are in theory able to map any open dynamical system. Still, they are often blamed to be unable to identify long-term dependencies in the data. Especially when they are trained with backpropagation it is ...

The authors propose a new type of recurrent neural network for handwritten character recognition. The proposed recurrent neural network differs from Jordan and Elman recurrent neural networks in view of functions and architectures because it was ...