Browse Books

Reviewer: John A. Fulcher

I first became interested in artificial neural networks (ANNs) a decade or so ago. At the time I was struck by how ANNs could perform so well as pattern recognition and classification tools, despite the simplistic models used. Something else that occurred to me was that the future of ANNs did not lie down the path of incremental improvements to these simple models (for example, backpropagation), but rather in entirely new models. It is my contention that the ANN field hit a brick wall during the mid to late 1990s. I wondered why just about everyone was focusing on level-output neurons, rather than the more biologically realistic pulse-output neurons. It is in this context that I find this book, which originated in a two-day workshop held in August 1997 at Cambridge University, so refreshing, although I hasten to add that it represents the first few tentative steps in this new direction. This book comprises three sections: “Basic Concepts and Models,” “Implementations,” and “Design and Analysis of Pulsed Neural Networks.” Section 1 consists of four chapters: “Spiking Neurons,” by W. Gerstner, “Computing with Spiking Neurons,” by W. Maass, “Pulse-based Computation in VLSI Neural Networks,” by A. Murray, and “Encoding Information in Neural Activity,” by M. Recce. Section 2 consists of five chapters: “Building Silicon Nervous Systems with Dendritic Tree Neuromorphs,” by J. Elias and D. Northmore, “A Pulse-Coded Communications Infrastructure,” by S. Deiss et al., “Analog VLSI Pulsed Networks for Perceptive Processing,” by A. Mortara and P. Venier, “Preprocessing for Pulsed Neural VLSI Systems,” by A. Hamilton and K. Papathanasiou, and “Digital Simulation of Spiking Neural Networks,” by A. Sahnke et al. Part 3 consists of five chapters: “Populations of Spiking Neurons,” by W. Gerstner, “Collective Excitation Phenomena and their Applications,” by D. Horn and I. Opher, “Computing and Learning with Dynamic Synapses,” by W. Maass and A. Zador, “Stochastic Bit-Stream Neural Networks,” by P. Burge et al., and “Hebbian Learning of Pulse Timing in the Barn Owl Auditory System,” by W. Gerstner et al. In the preface, T. Sejnowski states that “recent advances in experimental techniques are opening new ways to test theories for how information is encoded and decoded by spiking neurons in neural systems.” The relative timing of spikes in a population of neurons could also encode information. He goes on to speculate “as more evidence is found for the importance of spike timing in the cortex, the question shifts from whether spike timing carries information to how it is used. This volume provides a rich source of ideas that will serve as the starting point for many research directions.” I was particularly impressed with the introductory material presented in Section 1. In chapter 1, Gerstner distinguishes between rate codes (measured in terms of spike count, spike density, or population activity), and pulse codes (measured in terms of time-to-first-spike, phase, or correlation). A simple threshold-fire spiking model is then developed, and contrasted with the Hodgkin-Huxley neuron model. In chapter 2, Maass expands upon this simple spiking model and compares it with the familiar McCulloch and Pitts neuron model. He points out that “a network of spiking neurons computes a function which maps a vector of several time series onto a vector of several other time series” (in contrast to a conventional ANN, which maps vectors of numbers). Maass proceeds to show that such spiking networks can be used for storing and retrieving information, and indeed can act as universal approximators of continuous functions. He concludes with the statement, “They can carry out computations under different modes for coding information in spike trains. In particular, they can carry out analog computation not only under a rate code, but also under temporal codes where the timing of spikes carries analog information…through the use of temporal coding a network of spiking neurons may gain for certain computational tasks more computational power than a traditional neural network of comparable size.” Murray provides an overview of VLSI considerations in chapter 3. He makes passing reference to “the diversity of opinion that exists in the neural research community over the importance (or otherwise) of the biological exemplar. For some, it is merely the inspiration behind the neural paradigm, and sometimes embarrassing, while others regard it as a splendid working example, that should be adhered to in detail, as well as in general terms.” I favor the former view. After contrasting analog, digital, and hybrid approaches, Murray discusses coding methods , complexity, communication, memory, accuracy and training. He concludes, “It is entirely feasible that pulses offer opportunities for totally new forms of neurally-inspired computation and that pulse-based computers could be fundamentally superior to other forms in some applications. Time will tell.” This statement only serves to underscore that pulsed neural networks are a new—albeit exciting—research field. I was also taken with Maass and Zador's account of learning with dynamic synapses (chapter 12). They conclude that “one is likely to lose a substantial amount of computational power if one models biological networks of spiking neurons by artificial pulsed neural nets that employ the same type of static synapses that are familiar from traditional neural network models.…Hence if one wants to mimic adaptive mechanisms of biological neural systems in artificial pulsed neural nets, one is forced to go beyond the traditional ideas from neural network theory and look for new types of learning algorithms.” Indeed, I suspect that new learning algorithms will need to be developed in order to fully exploit the potential of pulsed neural networks generally. I hope I have been able to convey some of the excitement and enthusiasm demonstrated by the authors of this volume. I look forward to further developments in the field.