Harnessing stochasticity for superconductive multi-layer spike-rate-coded neuromorphic networks

SNNs attempt to replicate biological spiking behavior so as to unlock the ultra-low-power computation observed in biological systems. Biological neural networks encode information based on the frequency and relative timing of neuronal spiking events. This behavior is counter to conventional software neural networks, which represent information as single-valued numbers. Spiking neuromorphic networks are an attempt to more richly encode information with equivalent or cheaper hardware for more efficient, biomimetic computation.

Neurons in SNNs mimic biology by aggregating spiking activity from upstream neurons and firing when sufficiently stimulated, creating a spike event that is propagated downstream. A common neuronal model, the LIF neuron, integrates input spikes into an internal potential which leaks over time if not stimulated. When the potential crosses the action potential threshold, the neuron fires, resets, and begins integrating again. The LIF model is a very common neuronal model for SNNs [1].

Synapses in SNNs mimic biology by modulating the strengths of the connections between upstream and downstream neurons. While most SNNs employ synapses that modulate the amplitude of spiking events, this work utilizes an alternative approach wherein synapses modulate spike rate by—inspired by the prevalent stochasticity observed in biological systems [19, 21–23] – stochastically gating individual synaptic spiking events.

Stochasticity in synapses has appeared throughout the literature for both spiking [19, 20, 22, 24] and level-based networks [25, 26]. In level-based networks synapse weights are sampled several times per inference in a neural sampling machine whereas in spiking networks, stochastic synapses are effectively integrated with spike-rate information encodings [19, 20]. In this work, synapses stochastically pass a fraction of incoming spikes with a probability w, known as 'blank-out' gating [19, 20, 25]. Given an input spike train following a Poisson process with rate λ_in , the synapse output spike rate is given by: $\lambda_{\textrm{out}} = w * \lambda_{\textrm{in}}$, similarly following a Poisson process [19, 20].

Spike-rate-coded networks are a subset of SNNs in which information is encoded solely in average neuronal spike rates on wires throughout the network. Spike rate coded spiking neural networks may be trained similar to level-based feed-forward ANNs via backpropagation, assuming the following:

• information is only coded in the average spike rate and no useful information is present in the spike timing,
• the synapses modulate the spike rate by a trainable constant multiplier, and
• the neuronal output spike rate depends only on the input spike rate according to a continuous transfer or activation function [19].

Under these assumptions, the output rate of a synapse randomly passing a fixed proportion of fluxons is equal to multiplying the input rate by that synapse weight [19, 20]. Furthermore, confluence of post-synaptic spike trains result in a single spike train whose rate is the sum of the input rates. This rate may saturate at large frequencies which may be treated like saturation of an activation function. There therefore exists a one-to-one mapping between feed-forward spike-rate-coded SNNs and level-based ANNs, enabling identical training via backpropagation for both networks.

The benefits of this equivalence are manifold: in the case where spiking SFQ networks may be lower cost than level-based ANNs, previously trained networks may be based on SFQ hardware, and new networks may be effectively trained with extant software tools. Furthermore, backpropagation under spike-rate-coding assumptions may be used to train starting point networks for advanced learning techniques that may be evolved using unsupervised Hebbian learning techniques to begin encoding information in spike timing, further approaching the richness of information coding in the brain.

Single flux quantum logic is an emerging cryogenic technology for highly energy efficient computing [27]. SFQ circuits are based on JJs and superconducting quantum interference devices (SQUIDs), and operate with magnetic flux quanta. The quanta are typically represented by voltage pulses of quantized area equal to the magnetic flux quantum ($\Phi_0\sim$ 2.07 mV · ps) [28]. These pulses are generated in a process often referred to as JJ switching—a shift of superconducting phase between the terminals of the JJ by 2π.

The primary advantage of SFQ circuits for digital logic is the unparalleled energy efficiency and speed. Each 2π transition of a typical 100 µA JJ dissipates energy on the order of $2\times10^{-19}$ J [29]. Although a logic operation requires several switches, the energy per operation is several orders of magnitude lower than state-of-the-art CMOS logic even considering the cryogenic cooling to 4.2 K (liquid helium temperature) [30]. Additionally, as SFQ circuits operate on quantized pulses instead of voltage levels, SFQ systems are essentially very deep pipelines, as many distinct pulses can concurrently travel along different points of the same wire. This deep pipelining enables spike frequencies from tens to hundreds of gigahertz, one to two orders of magnitude higher than the fastest CMOS ASICs.

Whereas in conventional SFQ logic, information is encoded as the presence or absence of an SFQ pulse within a specific time period, alternative JJ-based circuits can generate and operate on more complex SFQ pulse sequences. This includes the generation of pulse sequences with a controllable frequency and stochastic switching induced by thermal noise. Both properties are exploited in this work.

Several approaches for neuromorphic computing with superconducting circuits have been proposed, although none utilize stochasticity in the synapses. A non-spiking superconducting XOR neural network was experimentally demonstrated [31]. Several proposals demonstrate individual SFQ gates for neuronal [5–7, 15], synaptic [7–9, 32], or interconnect [10] functionality. Two-neuron oscillatory Hopfield networks [11, 12] and single layer feed-forward spiking networks [13, 16–18] have been proposed, and a multi-layer feed-forward spiking network with heterogeneous CMOS-SFQ circuitry has been demonstrated in simulation [14]. Some of the proposed approaches utilize magnetic JJs to gradually modify the internal state of the gates, enabling online learning [8, 13, 16]. Fabrication processes used to manufacture these devices are, however, not well established, and the resulting circuits are limited in scale. In [33], an SFQ-based methodology for building neuromorphic networks is proposed, where a bipolar current is used to represent a logic state, which is converted into a train of SFQ pulses for transmission. Stochasticity in neuromorphic SFQ neurons has been studied [17], but to the authors' best knowledge, there are no studies exploring stochasticity in SFQ synapses.

Because of the phenomenological similarity between neuronal spikes and SFQ pulses, information in the network is represented in the timing and frequencies of SFQ pulses. Upon sufficiently stimulating a JJ with voltage and/or bias current, a 2π phase shift is created around a superconducting loop, producing a corresponding SFQ voltage pulse across the JJ, which can be propagated to stimulate downstream JJs. As an SFQ is the smallest possible non-zero voltage pulse, fluxons are ideal information carriers for SNNs, enabling extremely low power neuromorphic processing. Furthermore, information in a sequence of fluxons is not encoded in the magnitude of the spike but rather in the spike timing as is the case in SNNs and, more notably, the brain. Fluxons have non-volatile attributes as well, enabling short-term memory and the construction of low power LIF neurons.

A wide range of SNN topologies and encodings is available, and this work is particularly focused on spike-rate-coded architectures because of simplified training (section 2.3) and robustness to the stochasticity of the synapses (section 3.2) although the hardware here is amenable to other encoding methodologies. Among the various SNN topologies are perceptron networks and feed-forward deep neural networks; both of which are amenable to spike-rate-coding for classification tasks. Recurrent neural networks may be constructed as well, enabled by internal neuronal time delays in fluxon propagation and encoding information in timing between spikes. Strong biological evidence suggests that stochastic synapses may be well-suited for these timing-coded networks [19, 21–23] although more information is lost by not propagating an input spike, indicating that online learning using spike timing—such as spike-time-dependent plasticity [1]—is also amenable to SFQ networks [32] and may similarly ease training costs.

Inspired by the stochasticity in biological systems [19, 21–23], the proposed synapse stochastically gates incoming spikes, encoding the synaptic weight in the probability that the synapse will propagate an incoming fluxon to the output, implementing the first SFQ 'blank-out' [19, 20, 25] synapse in the literature. Stochastically dropping spikes at synapses is a well-known phenomenon in biology [19, 21–23], and biomimetic hardware will likely lead to efficient hardware.

These stochastic synapses are amenable to rate-coded networks as they directly modulate the rate of a spike-train as described in section 2.2. Additionally, as stochastic biological systems often utilize spike-timing to encode to information, it is conceivable that future neuromorphic timing-coded SNNs would benefit from stochasticity like that proposed here, although the exact function of such stochasticity in biological systems has yet to be understood completely and is not explored further here.

Whereas other synapse approaches modulate the effective amplitude of spiking events, due to the quantized nature of fluxons, it is difficult to accomplish this modulation without conversion between the fluxons and analog current, thereby decreasing system efficiency when amplitude-weighting is used. In the stochastic scheme, as both the inputs and outputs to synapses are fluxons, these conversions are not necessary making them potentially more efficient in scaled systems. The corresponding trade-off is that incorporating stochasticity into network behavior requires an increase in redundancy, whether longer spike-trains in rate-coded networks or redundant data-paths in timing-coded networks.

The proposed synapse is constructed from a Josephson balanced comparator [34, 37], harnessing thermal noise for true stochasticity. A Josephson balanced comparator is a pair of serially connected JJs with the bias current, I_B , applied between these JJs, as shown in figure 2(a). When an SFQ pulse is applied to the input, one of the JJs within the comparator undergoes a 2π phase shift, depending on the magnitude of I_B . For small I_B , $J1$ switches, absorbing the input pulse, whereas for large I_B , $J2$ switches, propagating the input pulse to the output. In the absence of noise current I_T , the gating functionality depends deterministically on the applied currents, however in the presence of thermal fluctuations in the applied currents, the balanced comparator exhibits stochastic behavior—a 'grey zone' [34]. A similar approach has been proposed for SFQ based synapses [33], where a C-SQUID [38] is used rather than a balanced comparator.

Zoom In Zoom Out Reset image size

The synapse weight is encoded in the probability that an input spike will propagate to the output and can be modulated after fabrication by adjusting I_B . Figure 2(b) depicts the probability of passing an incoming SFQ pulse to the output as a function of bias current for specific circuit parameters, demonstrating a continuous swing of weights between 0 (all fluxons blocked) and 1 (all fluxons propagated). The range of synapse weights may be extended to [−1, 1] using differential pairs of synapses connected to the excitatory and inhibitory inputs of each neuron (see section 3.3).

Neuronal LIF circuits—integrating incoming SFQ pulses until an internal threshold is reached and outputting a resultant SFQ spike sequence—are described here. A leaky SQUID loop is inductively coupled with a firing SQUID loop [39] to implement integrating, leaking, and firing behavior.

A conventional SQUID loop with a large inductance is used to integrate incoming SFQ pulses. The input SFQ pulses are applied across $J1$ or $J2$ (indicated in figure 3) as excitatory or inhibitory inputs respectively. As $J1$ switches, the fluxon energy is stored as current in inductor $L1$. This energy is accumulated with additional inputs increasing the flux stored in the loop. A fluxon applied to the inhibitory input switches $J2$, reducing the flux stored in the loop and allowing neurons to have both excitatory and inhibitory interactions.

Zoom In Zoom Out Reset image size

Leaking may be implemented with the addition of a resistive element R dissipating the energy stored in inductor $L1$. A resistance on the order of a few ohms produces a linear leakage characteristic, the timing of which can be tuned by changing the size of the resistor. Utilizing the dynamic SFQ (DSFQ) leakage mechanism, additional JJs may be introduced into the loop to provide a faster reset of the circulating current [40, 41].

When the current in the SQUID is sufficiently large, an inductively coupled SQUID produces a firing pulse. A JJ partially biased by an inductively coupled neuron loop current, is shown in figure 3. Due to the coupling between $L1$ and $L2$, as $L1$ integrates input pulses, current in $L2$ simultaneously increases. Eventually current in $L2$—with contributions from $L1$ and I_B —is sufficiently large to switch $J4$ producing an output pulse. In the case where inhibitory inputs are more frequent than excitatory inputs, $J3$ will switch instead of $J4$ sans output spike. Bias currents through the JJs help to regenerate fluxons, enable signal fan-out, and improve the cascade characteristics.

The relationship between the neuron input and output rates is non-linear, a necessary condition for proper neural network functionality. Figure 4 depicts the relationship between input and output spike rates (equivalent to the activation function of a conventional level-based neuron if spike-rate-coding is employed). Note the distinct non-linearity akin to a rectified linear unit (RELU), scaled exponential linear unit (SELU), or sigmoidal activation function. The width and slope of the threshold region as well as the saturation characteristics are adjusted by tuning I_B , $L1$, $L2$, R, and $I_C (J1)$ (see figure 3). The threshold input rate may be tuned after fabrication through the application of bias input sequences. At higher frequencies, output rate saturates as $J4$ moves toward the resistive regime adding additional non-linearity.

Zoom In Zoom Out Reset image size

The input fan-in is realized using conventional RSFQ confluence buffers (pulse mergers) [28]. These buffers exhibit a saturating spike rate, as multiple input pulses arriving in close succession produce only one output pulse. This property assists in the saturation of the neuronal output spike rate.

Negative weights are applied as inhibitory inputs to the neuron. A single synapse can therefore be implemented as a differential pair of synapse circuits connected to the excitatory and inhibitory inputs of a neuron. Inhibitory or excitatory bias input spike sequences may be added to adjust the neuron threshold.

Fan-out of signals is challenging, although solutions for SFQ fan-out have been proposed in the literature. As a fluxon is minimal and cannot be split, fluxons must be regenerated whenever a transmission line branches. Fan-out is therefore managed by splitter trees, which could incur well-known area and delay overhead [10, 18]. Multiple techniques exist to reduce the overhead of the signal fan-out in large scale SFQ circuits [42, 43]. These techniques are primarily based on utilizing a splitter with more than two outputs at the cost of reduced parameter margins. Large-scale fan-out trees are therefore possible although costly, and sparse network architectures should be employed. This constraint therefore encourages the deployment of pre-trained networks that may be thoroughly pruned before SFQ design as described in section 3.5.

The SFQ neuromorphic primitives can be cascaded to construct deep neuromorphic networks enabling straightforward circuit design with minimal layer-to-layer tuning. As inputs and outputs from synapses, neurons, fan-in, and fan-out circuits are all SFQ pulses there is no need for costly signal conversion to construct large multilayer networks, and there is no need for a clock network as all of the primitive circuits operate asynchronously.

In terms of reliability, the stochasticity of the synapses will not cause unwanted noise in downstream neuronal, fan-in, and fan-out circuits. The synapses are biased to a stochastic grey zone such that thermal energy at 4.2 K is sufficient for synaptic stochasticity, whereas the neuronal, fan-in, and fan-out circuits are designed to be reliable at these standard SFQ temperatures. As synapses do not create any additional noise, the only thermal noise that the synapse propagates to the rest of the circuit is via the spike rate and timing, which is the intended function of the synapses.

To ensure that spiking activity is similar from layer to layer, spiking activity can be regenerated through the use of signal confluence and additional input spike sequences, mitigating that the synapse output spike rate will always be less than or equal to the input rate. When an RSFQ confluence buffer merges two or more spike sequences, the output spike rate is the combined rate of the input sequences, increasing signal activity. Furthermore, additional bias spike sequences may be incorporated to further regenerate signal activity.

Layer-to-layer spiking activity can be tuned at a network-architecture level, mitigating the need to individually tune circuit parameters for each layer or for a specific network architecture. Specifically, synapse weights should be tuned to ensure that spiking activity remains within proper regions of operation for the circuits. Additionally, the rates of the bias input spike sequences may be tuned through the use of synapse circuits, and neuronal bias currents may be programmed to adjust the neuronal firing threshold.

The authors propose two ways to set weights and neuronal biases, both of which are controlled by DC current. Firstly, weights may simply be hardcoded during fabrication using standard SFQ bias current distribution techniques such as resistive trees or current-limiting JJs [44]. While the network would not be tunable after fabrication, it can still be useful in mass-produced edge-sensing devices where extreme resource efficiency is crucial, as pre-trained networks can be pruned and optimized before fabrication and do not need tunability, which can incur large energy, area, and routing costs. The case study of section 4 assumes hardcoded weights and biases.

The second possible weight-storage technique involves the circuit shown in figure 5(a), in which a SQUID loop—very similar to that presented in the neuron of figure 3—is inductively coupled to the synapse output inductor, similarly to a blocking gate [44]. As the SQUID loop of figure 5(a) does not incorporate a resistor, the device is non-volatile; fluxons may be added to the SQUID loop through input A or removed by pulsing input B enabling tunable control of the synapse weight. The relative inductance M will control the impact of each stored SFQ pulse on the synapse weight, enabling fine- or coarse-grained precision, and combinations of fine- and coarse-grained circuits can be used to jointly adjust the same synapse for increased weight range and precision. As demonstrated in the simulation of figure 5(b), a tuning circuit connected to the output inductor of a synapse allows for tunable spike pass probability. Finally, as inputs A and B may be connected to other SFQ circuits in the network, this circuit is amenable to direct implementations of online learning in which SFQ pulses may be sent to input A (B) to (depress) potentiate the weight during network operation.

Zoom In Zoom Out Reset image size

A trained neuromorphic network to compute the two bit XOR function requires at least two layers in an ANN along with negative weights and bias inputs. The chosen network is depicted in figure 6. Input neurons 0 and 1 normalize the network inputs rates to the neuronal output levels. Due to trained neuronal biases, neuron 2 will have a high output when one of the two inputs is on while neuron 3 requires both inputs to be on in order to activate. Neuron 4 uses inhibitory weights to compute the XOR functionality, activating only if neuron 2 is active while neuron 3 is not. Neuron 4 will therefore only turn on when exactly one of the inputs is on: the XOR functionality. A similar non-spiking architecture was demonstrated experimentally in superconducting hardware in [31].

Zoom In Zoom Out Reset image size

For this demonstration, spike-rate-coding is chosen to map the SFQ neuromorphic primitives to the chosen network. As described in sections 2.3 and 3.1, spike-rate-coding supports training similar to a level-based ANN and allows direct mapping of architecture and trained weights between the level-based ANN and the spike-rate-coded SNN. Weights are trained offline and are treated as constants in the network simulations. The two network inputs are stationary pulse trains with rates close to neuronal saturating rates, encoding a logic 0 (1) as a low (high) spike rate. The synapse weights are encoded in the probability of the spike propagation. The bias input sequences are trained to make the neurons sensitive to different combinations of inputs, tuning the threshold input rate of figure 4 to ensure large separability between the input patterns. The network outputs may be interpreted by the average spike rate relative to the thresholding spike rate of the neurons, which is close to 33.3 GHz in this circuit.

The XOR network is directly implemented with the SFQ primitives described in section 3. A schematic of the neuromorphic network based on these components, computing a two bit XOR function (presented in figure 6) is shown in figure 7. The synaptic weights and neuronal biases are tuned by changing the bias current.

Zoom In Zoom Out Reset image size

As shown in figure 8, the network correctly computes the XOR function. Note that the output signals can spike more frequently than the inputs, indicating that although the neurons and synapses reduce the spike rate to downstream devices, the splitters and confluence buffers regenerate spiking activity (see section 3.4), enabling deeper neuromorphic networks. Note also that there is a non-negligible propagation delay between the arrival of the input spikes and the computation of the output; the driving circuitry of the neuromorphic network would therefore need to account for delays from network signal propagation and stabilizing output. As delays and output signal encoding are strongly application- and network architecture-dependent, design of these external circuits is beyond the scope of this work, which is intended to be device, architecture, and encoding agnostic.

Zoom In Zoom Out Reset image size

Additionally the network is robust to variations in the input rates, showing a large separation between classes. A phase diagram of classification for the XOR network is shown in figure 9(a), where the dependence of the network output rate is shown as a function of the two input rates. It is desirable for the high output rates corresponding to the output of logic 1 (shown in figure 9 in red) to map to the input rates corresponding to logic 10 and 01. The input rates corresponding to logic 00 and 11 should produce low spike activity (shown in blue). This classification diagram displays good separation of the output states with respect to the input spike rates. The proposed network is therefore robust to variations in the input rates.

Zoom In Zoom Out Reset image size

This small network therefore works properly, and it is expected that the same network primitives may be used to design larger networks as described in sections 3.3 and 3.4. Fan-out may be designed using splitter trees as described in [10, 18], and directly implementing pruned, sparse, pre-trained networks can reduce the burden of fan-out and save area and energy cost associated with implementing tunable weights (see section 3.5).