Recurrent neural networks (RNNs), including long short-term memory (LSTM) RNNs, have
produced state-of-the-art results on a variety of speech recognition tasks. However, these
models are often too large in size for deployment on mobile devices with memory and
latency constraints. In this work, we study mechanisms for learning compact RNNs and
LSTMs via low-rank factorizations and parameter sharing schemes. Our goal is to
investigate redundancies in recurrent architectures where compression can be admitted …

Abstract Recurrent Neural Networks (RNNs) are powerful sequence modeling tools.
However, when dealing with high dimensional inputs, the training of RNNs becomes
computational expensive due to the large number of model parameters. This hinders RNNs
from solving many important computer vision tasks, such as Action Recognition in Videos
and Image Captioning. To overcome this problem, we propose a compact and flexible
structure, namely Block-Term tensor decomposition, which greatly reduces the parameters …