Model description

Fig 1A shows our computational framework: it includes three levels of processing: Peripheral (Fig 1A; left panel), Central (Fig 1A; middle panel) and Object (Fig 1A; right panel). Within each of these levels, we apply a given amount of adaptation, inhibition and noise, shown in Fig 1B. Full details of the model design can be found in Materials and methods. Overall the results show that a bistable response—the alternations of the model output—emerge with the introduction of adaptation, inhibition and noise in any one of these three stages independently, or when all three stages include these three terms. Furthermore, it generates a bistable response across the broadest range of variations of adaptation and inhibition in the Object-level stage of analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Model design.

Each model in the ensemble includes three hierarchical analysis stages: Peripheral, P(t), Central, C(t), and Object, O(t), over which we apply the same form of adaptation, inhibition and a small amount of noise, F[x(t)] A. The three analysis stages. The Peripheral analysis, P(t), computes a log-frequency spectrum using cochlear-like filter shapes. The Central analysis, C(t), computes multiple spectral scales of P(t), to capture dependencies across different frequencies. The Object analysis, O(t), computes multiple interpretations of the auditory scene into separate masks of C(t), selecting the interpretation most consistent with the data at each time frame. B. The application of adaptation, inhibition and noise. Adaptation, a(t), reduces the input, x(t), by a low pass version of the output history; inhibition, b(t), reduces the input by a low pass version of the output history of distant neighbors; noise, n(t), adds a small amount of variation to the output weights. The result is a set of output weights, y(t), applied within each model along the frequencies (for Peripheral), scales (for Central) or scene interpretations (for Object). C. Illustration of model unit responses to a given auditory streaming input when adaptation, inhibition and noise are applied to the given stage, and only that stage. The Central units have been averaged across the lower and upper half of the scales. The Object units have been selected from the top 4 possible interpretations, along with an approximate description of the given interpretation.

https://doi.org/10.1371/journal.pcbi.1007746.g001

During the Peripheral analysis stage (Fig 1A; left panel) the model computes a time-frequency representation of the sound. This analysis resembles a typical short-time Fourier transform, but includes several features which are more biologically plausible for the auditory periphery: this includes log-frequency cochlear-shaped filters, half-wave rectification and firing rate limitations.

During the Central analysis stage (Fig 1A; middle panel) the model computes a series of time-frequency analyses that vary in their spectral scale. Each scale is defined by a series of band-pass filters, all with the same width, applied along the dimension of log-frequency of the auditory spectrogram from the Peripheral analysis. Spectral scale captures the spectral dependency of measured receptive fields found in the inferior colliculus (IC) [78–81] and primary auditory cortex (A1) [69, 82, 83]. For our purposes in this report, we focused on this single set of frequency-based features to limit complexity during the systematic exploration of a large ensemble, though it could conceivably be extended to any number of additional relevant perceptual features.

During the Object analysis stage (Fig 1A; right panel) the model computes multiple interpretations of the acoustic scene. Each interpretation consists of one or more groupings of Central features into streams (or objects). The different scene interpretations emerge on the basis of the hyper-parameters of this stage, which vary in value from levels that imply rapid changes over time, to gradual changes over time. These varying scene interpretations are weighted on the basis of their posterior probability. Consistent with the evidence concerning the formation of auditory objects in the brain, these features are grouped through a process of temporal coherence [8, 9, 70, 84, 85] and strung together over time on the basis of object continuity [9, 71, 86, 87]. Temporal coherence means that the formation of initial groupings depends on the short-term correlations in the scene across different time scales: as such, acoustic events that occur simultaneously or nearly simultaneously will be grouped together. Object continuity means that events grouped during temporal coherence that appear similar across time are grouped together as one object. These two stages were selected with the goal of showing a range of plausible grouping behaviors across the stimulus of interest here (the A-B-A repeating tones), while adhering relatively closely to existing principles of scene analysis. The temporal-coherence stage has proven effective on its own in existing work [70] within a similar framework. We required the additional, second object-continuity stage to operate across the longer time scales necessary to examine perceptual bistability.

For the present, auditory streaming stimulus, the two components of the Object-level analysis play complementary roles: first, temporal coherence allows us to identify two distinct clusters of responses in the output of the central analysis, corresponding to the two tones (A and B) from the multitude of frequency-scale outputs of the Central stage. Second, the object continuity stage allows these two tones to either be treated as two separate streams or grouped into a single stream, depending on the prior. Over the course of the stimulus the final posterior probability of these two possible scene interpretations changes, due both to evidence accumulation and to fluctuations in the strength of different interpretations caused by adaptation, inhibition and noise.

We vary the behavior of these three stages across a large ensemble of models by systematically varying the magnitude of three terms—adaptation, inhibition and noise (Fig 1B)—across the three analysis stages. These terms are applied to a set of weights which determine the relative strength of different frequencies (Peripheral), scales (Central) or scene interpretations (Object). Adaptation reduces each weight by a delayed, low-pass version of itself (self-referential link in Fig 1B). Inhibition reduces each weight by a delayed, low-pass version of neighboring weights (lines to and from second unit in Fig 1B). Noise modulates each weight randomly (line to noisy response in Fig 1B). These basic components have been applied in various forms throughout the years to model perceptual bistability [18, 19, 41–43, 47–50].

Fig 1C provides a simplified illustration of the response of each analysis stage to the input. It aimes to provide some intuition of the effects of adaptation, inhibition and noise within the model. Each output shows how the given stage responds when adaptation, inhibition and noise are applied to that stage, and that stage only. For ease of interpretation we have opted to simplify the unit responses for the Central stage and Object stage outputs: the Central stage is averaged over all frequencies, and over multiple scales—grouped into the lower and upper halves—and the Object stage includes a set of the most probable interpretations and a schematic representing the approximate scene configuration. From this panel it can be seen that adaptation, inhibition and noise can induce distinct periods of alternating activity at each stage of the model. Fig 1C also provides a demonstration of an important principle of distributed bistability—that the earlier alternations of unit strength do not directly correspond to bistable perception, but instead are interpreted by the Object stage and ultimately generate a bistable response.

Model evaluation

To evaluate each model in our ensemble we compare its ability to predict human behavior on two data sets, shown in dark blue in the panels of Fig 2; together these data sets capture two key hallmarks of perceptual bistability.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Best-fitting models.

The performance of humans and models that best fit human behavior. Models included here fell in the top 25^th percentile according to an aggregate measure of model fitness (the model:human deviation ratio). A. The percent of segregated (2 or more object) responses. Human data were taken from two existing studies [16, 88]. Error bars denote the 95% confidence intervals estimated by bootstrap. Human data include a total of N = 72 participants. B. The distribution of response lengths for the 6 semitone stimulus across all listeners, and the best fitting models. The lengths of each individual model and human listener were first Z-scored in log space. Human data include a total of N = 35 participants, with an average of 211 samples per listener (minimum = 20, maximum = 1232). Percept lengths were combined across the two percept types: fused and segregated. C. Individual data for 5 example models (columns) across the three stimulus conditions (rows). The “None” model includes no adaptation, inhibition or noise. The “Peripheral”, “Central”, and “Object”, include adaptation, inhibition and noise in their respective levels, and the “Combined” model includes these terms in all three stages. Each panel represents periods of fused (red) and segregated (blue) responses over five simulation runs (y-axis) of a 10-second time period during the steady state response of the model output (x-axis).

https://doi.org/10.1371/journal.pcbi.1007746.g002

The first hallmark of bistability is that it is selective to stimulus ambiguity: for the A-B-A stimulus employed here this means that bistability is most pronounced at an intermediate (6 semitone) frequency separation. Fig 2A shows human data which captures this hallmark [16, 88]. It depicts behavior in terms of the percent of segregated responses (y-axis), across the three stimulus conditions tested (x-axis). A segregated response is one that indicates the listener heard two or more sounds, and a fused response indicates only one sound was heard. The pure tones in the ABA pattern presented to listeners were separated by either 3, 6 or 12 semitones (st). Only the 6 st stimulus elicited a clearly bistable response: listeners reported the 3 st stimulus was mostly heard as a single sound source, and the 12 st stimulus as mostly two streams of sound.

The second hallmark of bistability is the characteristic distribution of response lengths. The response length is the amount of time between when a listener reports hearing one stream and when they report hearing two streams of sound, and vice versa. Fig 2B shows human data which captures this hallmark (c.f. Human data). These data depict behavior in terms of the distribution (y-axis) of Z-scored response lengths (x-axis), but only for the most ambiguous stimulus (6st). The figure shows that there were periods of relatively stable perception (the distribution has a long tail), with some characteristic length (the distribution peaks above zero). Z-scores were applied to each individual configuration of the model and each human listener in log space. This normalization is applied because (1) the rate of switching varies across individual listeners [21, 25, 77], and (2) our focus in this study is on evaluating model fit to the overall pattern of responses, rather than evaluating a quantitative match to the temporal dynamics of individual listeners. We also find that there is a similar match between model and data when the lengths are mean-normalized (following the normalization used in [4]).

We chose to use different human data sets across these panels because neither data set could capture the characteristics of the other: in the first two studies [16, 88] the trials were too short to avoid cutting off the tail of the distribution shown in Fig 2B, and so the pattern of response lengths could not be determined; the data presented in Fig 2B focused on bistable responses, using only the 6 st stimulus, and employed longer trials; these data did not include the 3 or 12 st stimulus.

To evaluate each model’s ability to capture the first hallmark of bistability we are interested in—the selectivity of bistability—we compute a quantity referred to as the response deviation: the deviation of model predicted responses from that of the mean human listener. Specifically, for each stimulus (3, 6 and 12 st), we found the proportion of “segregated” responses (ala Fig 2) for each individual simulation run and for the average human listener, and then computed the root-mean-squared difference between model and human responses across all three stimuli.

To evaluate each model’s ability to capture the pattern of response lengths, we compute the response-length deviation: across all simulations of a given model we compute the Kolmogorov-Smirnov statistic [89] of the model’s distribution of Z-scored response lengths vs. that same distribution for the human listeners. We selected the Kolmogorov-Smirnov statistic—the maximum difference between the empirical cumulative distribution function of two samples—for its sensitivity to subtle differences between two distributions.

These two measures of deviation are combined to provide an overall fitness score for each model, referred to as the model:human deviation ratio. To accomplish this, the two deviation measures are first placed on a comparable scale by dividing model deviation by the deviation of individual human listeners. The two deviation ratios are then averaged. We compute deviation for each individual human listener from the overall sample using the same procedure described above for computing model deviation. Specifically, each individual’s response deviation is the root-mean-squared difference from individual to mean proportion of “segregated” responses. An individual’s response-length deviation is the Kolmogorov-Smirnov statistic of their Z-scored response lengths compared to the Z-scored response lengths of all listeners.

To provide some intuition about this measure: the larger the model:human deviation ratio, the greater the model deviation is relative to the deviation of the average human listener from the mean. A deviation of 1 would indicate the model has the same amount of deviation from the mean as the average individual human listener.

To determine whether our ensemble includes models that are consistent with human behavior, we examine the behavior of the top 25% of models, according to this model:human deviation ratio (light blue circles in Fig 2A; light blue region in Fig 2B). These model data support the merits of our aggregate measure, confirming that when models score well (low) by this measure, they collectively show behavior quite similar to the mean human data.

As a further check on model behavior, we examine several representative models in more detail (Fig 2C): one including no adaptation, inhibition or noise, one including these three terms in just the Peripheral-level, just the Central-level and just the Object-level analysis, and a final model with the three terms in all three levels simultaneously. When the Peripheral level includes these terms, their magnitudes are set as c_a = 15 (adaptation) and c_b = 130 (inhibition). For Central and Object, their magnitudes are set as c_a = 5 (adaptation) and c_b = 5 (inhibition). These are selected from the top 25% of the ensemble for the within-stage model variations (discussed in Model behavior across the ensemble below), and such that their combination yields a reasonable model for the Combined example (informed by the across-stage model variations, also discussed below).

In Fig 2C the individual behavior for 5 simulation runs are shown. This behavioral output was generated by a thresholding of the time-frequency-mask output of the Object level stage (see Interpretation of model output). Each panel in this figure represents whether a model reports a fused (red) or segregated (blue) response at the given time (x-axis) and simulation run (y-axis), across the three different stimuli tested (rows). This figure displays the reported percepts of a model simulation, starting at 5 seconds to avoid the “buildup” phase of the responses (c.f. S1 Fig). The response to a given stimulus is constant for the “None” model. The remaining four models demonstrate the most balanced alternations for the 6 st stimulus, and show less ambiguous, but still bistable responses for the 3 st and 12 st stimuli, consistent with the indications that these stimuli can also include some bistable fluctuations [90]. This figure demonstrates that the inclusion of adaptation, inhibition and noise, at each individual analysis level, or at all levels at once can generate reasonable bistable behavior.

In a supplementary figure (S1 Fig), we also examine the early responses of the same five models from Fig 2C: specifically from 0 to 10 seconds. For each model with some amount of adaptation, inhibition and noise, we find a rough qualitative match to a phenomenon sometimes referred to as “buildup” (e.g. [4, 91, 92]): a consistent transition from “fused” responses at the start of a run to a more average balance between “fused” and “segregated” response. The results are fairly coarse, showing just a qualitative match to the human outcomes, and some deterministic behavior in early alternations of the Object stage analysis. These outcomes are representative of the buildup curves of other models we examined, though not all show this buildup property. Since we were focused on the magnitudes rather than the time constants of adaptation, inhibition and noise, we did not seek out a more precise temporal fit to the human data. Though we find only a rough qualitative match, it remains reassuring that the model, not specifically designed to handle buildup, is able to show an outcome reminiscent of buildup.

Model behavior across the ensemble

We now evaluate the behavior of the entire model ensemble, using the aggregate model:human deviation ratio and the two separate deviation measures—response and response-length deviation (see Model evaluation). We focus on the systematic variation of adaptation and inhibition both within and across the levels of the hierarchy. Varying the amount of noise across levels did not appear to change the key conclusions of our results (see Sensitivity analysis).

Fig 3 shows the within-stage model variations of adaptation and inhibition: in these variations, adaptation, inhibition and noise are applied to only one of the levels of the hierarchy at a time, setting their parameters to zero for the other two stages.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Within-stage variations of adaptation and inhibition.

Each model shown includes terms for adaptation, inhibition and noise in one of the three analysis stages, leaving the other two stages with no such terms. This configuration lead to a total of 3 × 10² = 300 model variants corresponding to the 3 perceptual stages, with 10 different levels of adaptation and inhibition. A The model:human deviation ratio for all levels of inhibition and adaptation at each of the three analysis stages. Each square represents average deviation for all simulations of a model with the given level of adaptation and inhibition. Values above one (pale blue to red) indicate models with more deviation than the average human listener; values below one (blue colors) indicate models with the same or less deviation than the average human listener. These measures are averaged across the ratio of both response deviation (panel B) and the response-length deviation (panel C). B The response deviation for all models and the human listeners from [16, 88] (N = 72). Each point for the three model variations represents the average response deviation for all simulations of a single model from the ensemble. Offsets from the central x-axis of each condition are used as a visual aid, to ensure that all data points are at least partially visible. The dotted line marks the mean of the human data: the deviation ratio is greater than one above this line and less than one below this line. C The response-length deviation for all models and human listeners. Human data are the same as those shown in Fig 2B (N = 35).

https://doi.org/10.1371/journal.pcbi.1007746.g003

Fig 3A shows behavior for the within-stage model variations in terms of the model:human deviation ratio. This measure is shown for each level of adaptation (x-axis) and inhibition (y-axis) tested within each of the three levels of the hierarchy (columns). The figure supports two key findings: that some variations in the parameters of each level lead to human-like model behavior (there is blue in all three panels); and more such variations of the higher-level model parameters than lower-level parameters lead to human like behavior (there is more blue moving from left to right).

The overall outcomes reflected by the model:human deviation ratio are also supported when we examine the two deviation measures separately: response deviation—in Fig 3B—and response-length deviation—in Fig 3C. Note that points are displaced along the x-axis within this figure to ensure that each datum is visible; thus, the overall shape of this plot provides some indication of the distribution of the data. Both figures show that some individual models (data points) fall at or below the mean human deviation (dotted lines). Further, as we move up along the hierarchy, the mass of points (the mean) is lower, meaning that there are increasingly more models more consistent with the human data.

Fig 4 summarizes the results of the across-stage variations of adaptation and inhibition: in these models, adaptation and inhibition are varied simultaneously across all three levels. Despite the presence of these terms across all three layers in many possible configurations, the results remain quite similar to the within-stage variations of Fig 3: there are non-zero levels of adaptation and inhibition at all three stages that lead to human-like behavior (blue in all three panels), and there are more such accurate model variations as we move up the stages of analysis (blue increases from left to right). Note that each datum in these figures (square in A or point in B and C) represents the best (minimum) mean deviation ratio across all models with the given magnitude of adaptation and inhibition at a specific analysis stage. Therefore, for a given stage, each point is representative of the minimum deviation ratio of 625 models (5² × 5²), one model for each level of adaptation and inhibition at the other two analysis stages.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Across-stage variations of adaptation and inhibition.

Each model in the ensemble includes some amount of adaptation, inhibition and noise at all three analysis stages. This configuration lead to a total of (5²)³ = 390, 625 model variants corresponding to the 5 different levels of adaptation and inhibition across the three analysis stages. A The minimum model:human deviation ratio for each value of adaptation and inhibition at each level. Each square represents the average for all simulations of the best performing (minimum deviation) model with the given level of adaptation and inhibition. Values above one (pale blue and red colors) indicate models with more deviation than the average human listener, values below one (blue colors) indicate models with the same or less deviation than the average human listener. B Response deviation for the data shown in A: that is, the minimum percent-streaming deviation for each pairing of adaptation and inhibition, tested at each level of the model hierarchy. The dotted line indicates the average deviation of the human listeners. C Response-length deviation for the data shown in A. Figure follows the same format as described for panel B.

https://doi.org/10.1371/journal.pcbi.1007746.g004

Sensitivity analysis

To determine how sensitive our results are to specific parameter values, we examine the effects of varying these parameters on the model:human deviation ratio (Fig 5). We vary the time constants for adaptation (τ_a) and inhibition (τ_b), the magnitude of noise (c_σ), and the breadth of inhibition (Σ_b). For each parameter considered we vary that parameter, while leaving all other parameters at their default value. For each parameter setting we examine the within-stage model variations, but using a coarser grid than shown for the within-stage model variations from Fig 3: instead we use the 25 parameter combinations shown for Fig 4. An inspection of the individual data revealed no large differences between the results, regardless of the parameter varied. (That is, we re-plotted Fig 3 for all 13 model variants)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Sensitivity of model.

A summary of the overall model behavior across different parameter variations. The default-parameter model essentially summarizes the results from Fig 3A, though the models tested here use a coarser grid of variations across adaptation and inhibition, consistent with the grid used for the remaining parameter variations shown in the present figure. For each parameter varied (columns), we consider multiple possible alternatives (numbers above each group of bars), changing it from a default value (shown above each group, within the gray region). Note that the values of inhibition breadth (Σ_b) differ across the levels because their effect on behavior differs across the levels.

https://doi.org/10.1371/journal.pcbi.1007746.g005

Fig 5 provides a summary of all model variants. It shows the mean model:human deviation ratio (y-axis) for all the within-stage model variations when using the default model parameters (far left column) and when using a number of different parameter variants (x-axis). For ease of reference, the default model parameters are also shown (upper gray row). Note that the final parameter (Σ_b) had distinct values within each level of the hierarchy because its meaning depends on the number of competing units, which varied across level.

This figure shows that one of the key results of our analysis is not altered by variations in the model parameters. Regardless of parameter variant, more variations of adaptation and inhibition are consistent with the human data as we move up the hierarchy. As such, the mean model:human deviation ratio across the variations of adaptation and inhibition decreases as we move up the hierarchy.

Furthermore, all of these parameter variants have some model variations consistent with the human data. The average minimum model:human deviation ratio across these variants is 0.88 (SD = 0.18).

Distributed sources of bistability

The notion that perceptual bistability may emerge from many different stages of analysis has received converging support from a number of different sources [3, 10, 12–14, 16–20, 48, 50–63]. However, existing efforts to model the locus of bistability across a hierarchy [11, 13, 14, 17, 41] differ from the present effort in that they (1) do not consider multiple possible loci of bistability within this hierarchy and (2) do not introduce adaptation and inhibition within a system that actually computes a scene interpretation from raw sensory input. These two differences strengthen the support for the pervasiveness of perceptual bistability because they show that (1) bistability can emerge from a large variety of different model configurations and that (2) this emergence can scale from more idealized models to a more complete system capable of generating perceptual inferences relevant to behavior. Outside the region of an ambiguous stimulus presented to our model (6 st), the model computes a useful mask that either fuses a scene into one stream (for 3 st) or extracts it into two streams (12 st).

Past computational studies of bistability have focused on a single locus at either relatively early stages [65, 66] or late stages of processing [64, 93]. None of them systematically vary the locus. In Rankin et al [65] (and also [66]), a relatively low-level source of auditory bistability is examined: three broadly tuned frequency channels compete and these can be used to explain bistability during the ABA tone pattern. This approach is similar in spirit to introducing bistability during the Central or Peripheral analysis stage of our model: in both examples there are a set of competing, broadly tuned tonotopic features, and there is no inference process which groups features into objects. In Mill et al [64] and in Barniv and Nelson [93] a higher-level source is evaluated, in which probabilistic interpretations of the acoustic scene compete with one another. The implicit assumption is that bistability occurs over some representation of auditory objects. In this way it is similar to introducing bistability during the Object stage of our model. Note that any one of these accounts of competitive dynamics might apply at an earlier or later stage of stimulus representation. Yet none systematically explore this possibility.

Existing bistable models of auditory streaming also represent input in a manner that is far more constrained than the present model: a small number of units represent the stimulus. This approach limits the ability to compare the merits of different bistable loci within a single system, because the transformations across stages of analysis are precisely when the details of how input is transformed from a raw acoustic input start to be important. Existing auditory models of this streaming paradigm that do include a more detailed perceptual hierarchy do not appear to demonstrate any form of perceptual bistability [70, 85, 94].

The present model can be understood as a hybrid of past approaches. It incorporates a form of adaptation, inhibition and noise, of a simpler form but similar in spirit to past competitive models [65, 66]; it incorporates a form of evidence accumulation during the Object-level analysis, akin to more probabilistic models [64, 93]; and furthermore, the competitive dynamics are included across a hierarchy, akin to past hierarchical perceptual models [70, 85, 94].

This modeling approach also provides an illustration of an important concept of distributed bistability: that the most obvious neural correlates of bistability may only represent a read-out of some earlier, more opaque progenitor of bistability. For example, the fluctuations in emphasis between units near A- and B-tone frequencies, shown in Fig 1C, do not directly match a specific scene interpretation, yet their presence is required to induce behavioral bistability (Fig 2C).

A systematic increase of variation up the perceptual hierarchy?

The existing evidence for individual differences in perceptual bistability [5, 7, 18, 21–23, 25, 30, 32, 43, 47–50, 77, 95, 96] suggest that the breadth of functional variations represented by our model ensemble may also be present in the human population. That is, there may be more variation in the sources of bistability at higher levels of the perceptual hierarchy. Most likely, this observed pattern was due to the change in how the signal is represented at these stages. This intriguing prediction suggests that one promising line of future work would be to compare this model or its extension to human data at the individual level.

Relation to physiology

Our analysis indicates that plausible forms of bistability can be generated by the dynamics of a number of distinct functional representations (peripheral, central and object), and that it is possible for all of these generators of perceptual bistability to exist simultaneously. To the extent that these representations are employed within a given physiological substrate, they could support perceptual bistability. The key point is that these are functional characterizations of the auditory system, rather than precise physiological analogues. Our conclusions rely on a systematic evaluation of the amount of adaptation, inhibition and noise present across many models. Other parameters of the simulation were not precisely tuned, and our key conclusions appear relatively robust to variations in these other model parameters. The model thus provides a qualitative assessment of how the magnitude of adaptation and inhibition across a perceptual hierarchy may induce bistable behavior. Therefore, appropriate comparisons between this model and physiology should be at the level of functional process, rather than any precise match between model parameters and physiology.

One natural mapping between the present model and physiology would be to relate the Peripheral stage to brainstem, Central to midbrain or primary cortex, and Object to primary and/or secondary areas of auditory cortex. Concordant with the idea that each area contributes to bistability, several long-term averages of neural activity are consistent with the buildup of auditory streaming within early brainstem (cochlear nucleus) [92], and cortex [91]. These sites could be showing an appropriate correlate for auditory streaming because each contributes, in a distributed fashion, to the final bistable report of perception.

If there are multiple generators of these bistable dynamics arising from these different loci, this could occur in a number of ways. Emergent properties of a population of neurons [97], the dynamics of sub-threshold membrane potentials [98], exploration-exploitation trade-offs in perceptual decision making [23, 59], short-term plasticity, predictive coding [7, 99], and top-down modulation [7, 41] may all have a role to play in placing the system in a state where these dynamics could emerge within each site, or across sites.

The dynamics of bistability would likely not arise solely from the intrinsic rates of adaptation and inhibition of isolated neurons, as these can be much shorter in the brainstem and midbrain [100, 101] compared to even the shortest time constants employed in the present case. However, the rates of stimulus-specific adaptation in the midbrain fall along a more realistic time scale [97, 102–105]. None-the-less, we would be wary of any direct analogies to single-unit recordings and the present model. How the population- and system-wide processing stages simulated here map to individual neurons remains unclear. One must be especially cautious given the possibility that, within a distributed system, the progeintors of bistable behavior may not show an obvious relationship to perceptual alternations; these alternations may only be apparent in the read-out these progenitors. A precise mapping from individual neurons to this model would require further careful study.

Future work

Our focus in this report is on the magnitude of adaptation, inhibition and noise during auditory streaming at three stages of perceptual analysis. While this has proven fruitful, it necessarily excludes many other fascinating aspects of perceptual bistability.

First, there are a number of other plausible sources of bistability: alternations in top-down interactions [5, 7, 106], alternations in pathways outside the auditory system [23, 107], and alternations at multiple time scales [108]. Given the ability of the three forms of competition examined in the present model to co-exist within a single system, we speculate that these additional forms of competition may all co-exist to varying degrees.

Second, there are many details of the temporal dynamics of bistable models that we have omitted [18, 19, 48, 50, 65, 109–111]. The innovations considered from this past work, such as recurrent excitation [110, 111], noise-dominant dynamics [50, 109], multiple oscillators within the same level of analysis [19] or low-level background activity [18, 48], may be necessary to fully capture the dynamics of human data [2]. Future efforts along these lines will likely require additional measures of model performance that can assess the quantitative match to the time course of model and human behavior, such as a precise match to the “buildup” phenomena, rather than the very coarse qualitative match found in the present model (S1 Fig).

Third, there are many additional bistable and multistable paradigms to consider modeling [6, 33, 34, 112–117]. A first step towards handling these paradigms could be to extend the features included in the Central analysis to spectral-temporal features. These features are probably important for both Shepard tones [112], and speech stimuli [118]. Both stimuli can lead to clear examples of multistability—when the harmonics are evenly split between two possible pitch-change directions [114], and when the same word is repeated, leading to a series of verbal transformations [6, 23, 33, 34]. Note that multistable outputs, as opposed to simple bistable outputs, could be readily computed from our model, because the output of the object-level analysis is a time-frequency mask which, in the present case, we have interpreted heuristically as a bistable response.

Fourth, bistable perception may help us examine differences between conscious and unconscious states [38–41]. The present model may provide a potential baseline upon which to develop computational models of conscious vs. unconscious perception in auditory scenes. Given the indications that correlates of conscious perception arise within an order of several hundred milliseconds [38, 39, 119](but see [120]), and that conscious states appear to be associated with more comprehensive integration of information [39, 119], this present feed-forward model likely reflects early unconscious processing. In a similar vein, there are likely contributions to bistability in areas well past the auditory cortex (e.g. [23, 33, 107]), functions not modeled in the present case. Additional sources of bistability such as top-down, predictive error signals [5, 7] could have more of a role to play in any consciously driven aspects of perceptual bistability [39, 85].

Fifth, one implication of Fig 3 is that whether adaptation or inhibition is necessary to generate bistability depends on the stage of analysis: the Peripheral stage required both adaptation and inhibition, the Central stage did not require adaptation, and the Object did not require inhibition. (Note that a systematic removal of noise was not possible with the current design: even without the introduction of a noise term, the signal processing of the Peripheral and Central stages introduces apparent noise and the Object stage clustering is probabilistic.) The implication of the cross-stage differences in the requirements for adaptation and inhibition is that the representation over which these terms operate is vital: future work could systematically explore how aspects of the signal representation determine whether adaptation and inhibition are necessary to induce a plausible form of perceptual bistability.

These numerous potential elaborations of the model will benefit from our work here, in that it establishes that several foundational components of perceptual bistability can be distributed throughout a larger system capable of segmenting simple auditory scenes. In particular, because this model consumes raw acoustic input, this will allow future elaborations of the model to be validated across multiple experimental paradigms and stimuli.

Conclusion

The present computational study indicates that auditory bistability can plausibly emerge from a multitude of different sources, from very early following the auditory periphery to later, object-level encodings of the stimulus. Given the pervasiveness of adaptation, inhibition and, arguably [121], noise, throughout the nervous system, this suggests that bistability arises from these sources throughout the auditory hierarchy. The results also indicate that there are more variations of the later stages consistent with human data than earlier stages of processing. This could suggest that, while distributed, there is a more dominant role for Object-level sources of bistability. Furthermore, when interpreted in light of the significant individual differences found in perceptual bistability [24–32, 96], it is possible that there is greater variability in bistable function for the higher-level stages of auditory processing, than lower-level stages of processing.

Ethics statement

All participants who contributed empirical data as a part of this report provided written consent to participate in the study. Procedures were approved by the University of Nevada, Las Vegas Office of Research Integrity, approval number 867626.

Model design

In each model in the ensemble the three stages of analysis are a Peripheral analysis, P(t), a Central analysis, C(t), and an Object analysis, O(t)—shown in Fig 1A. Into each level we apply adaptation, a(t), inhibition, b(t), and noise, n(t)—collectively denoted F[x(t)], shown in Fig 1A—with varying magnitudes.

All model parameters are described in Tables 1 and 2. Unless otherwise stated, these parameters were manually selected, to provide a reasonable fit to human data while minimizing computational cost. This occurred during a series of pilot model simulations. Minimal adjustments to these parameters were applied during piloting, to avoid model over-fitting, and we carefully explored the effect of changing a number of these parameters during our analysis (see Sensitivity analysis).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Model parameters.

Some of these took on multiple values. Multiple, consecutive integers are denoted with “…”.

https://doi.org/10.1371/journal.pcbi.1007746.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Model parameters that varied by level of analysis.

https://doi.org/10.1371/journal.pcbi.1007746.t002

To assist in the perusal of the model description, Table 3 provides a list of the most important functions, and their meaning. In cases where a variable is left out of a function (e.g. x(t) instead of x(t, f)) the implication is that the output is multi-valued (e.g. x(t) = [x(t, f₁), x(t, f₂), …]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Major functions and their meaning.

https://doi.org/10.1371/journal.pcbi.1007746.t003

Model evaluation

We evaluate model responses on an ABA stimulus, created to exactly match the stimulus used in Snyder et al [16]. It consists of two different frequencies: one at 500 Hz (A) and another either 3, 6 or 12 semitones (st) above 500 Hz (B). Each triplet consisted of three equally spaced 50 ms tones, with onsets separated by 120 ms. The triplets are separated by 480 ms each. Pure tones included a 10 ms cosine ramp at the onset and offset.

The key model parameters we manipulate during our experiments are the magnitude of adaptation (c_a) and inhibition (c_b) across the values 0, 5, 15, 44, 130, 130, 390, 1100, 3400, 10000 and 10⁵, for the within-level variations, and across 0, 5, 63, 790 and 10⁵, for the across-level variations. This results in 300 total pairings of the within-stage parameters (10² × 3), and 390, 625 pairings of the across stage models (). For each pairing of the within-stage model parameters we run 20 simulations, each with a 48 second stimulus composed of 100 repetitions of the ABA pattern. This is repeated three times, for the three stimulus conditions (3, 6 or 12 st). For each of the across-stage model variations, we use the same procedure, but it terminated early for all poorly performing model variations (Fig 4): following the first 10 simulations of the 6 semitone stimulus (always the first stimulus evaluated), if the model:human deviation ratio is greater than one, no more simulations for that model are run. This procedure ensures that our finite computational resources were not spent evaluating the merits of models that clearly performed poorly on this most essential stimulus condition.

Interpretation of model output.

To interpret the model output for each simulation, we use the following strategy. During the streaming task human listeners are often asked to respond continuously, reporting if they heard a “fused” (1 stream) or “segregated” percept (2 or more streams). To obtain a similar response for the model we interpret the maximum amplitude time-frequency mask found at each frame of O(t) as a “fused” or “segregated” response using an automated heuristic. Specifically, we compare the estimated frequency bandwidth of this dominant mask to that of the input’s time-frequency representation, P(t), and reported a 2 stream percept only if this ratio was greater than Θ_r (c.f. Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Heuristic parameters (c.f. Eq 30).

https://doi.org/10.1371/journal.pcbi.1007746.t004

To estimate the frequency bandwidth, we use the following procedure. The frequency bandwidth of the dominant source is found for multiple overlapping windows of analysis w_b(t). For each time window (of length Θ_w, at time step Θ_Δ) this bandwidth is estimated by finding the maximum distance between frequency channels whose quantile falls above a data-driven threshold, determined by the quantile of the entire mask. Specifically, using Q_f(p) to denote the p^th quantile of frequency channel f, we estimate the bandwidth as follows (30)

The values for all Θ parameters (shown in Table 4) are selected by hand to provide model responses consistent with a visual inspection of sampled output masks. We eliminate any brief switches: any response that returns back to the original response within 250 ms is discarded as a spurious output of our decision-making heuristic.

Before analyzing the model responses during the computation of our model:human deviation ratio, we discard the first response. This same procedure has been used on human data to examine the steady state behavior, to avoid any “buildup” in the first response to the stimulus [4], a feature our model appears to replicate (S1 Fig).

Human data

We use human data from three sources: [16, 88] and a new group of participants. From the two existing data sources, we use the context phase of Experiment 1A of Snyder et al [16] and the context phase of Experiment 1 and 2 of Yerkes et al [88]. The stimuli are identical to those described above, but the base frequency was 300 Hz instead of 500 Hz in some of the conditions (the measures of interest here did not appear to differ across these two different frequencies). We found the proportion of segregated responses by selecting the current response during the time range of 4 to 6 seconds for each individual. If multiple responses occurred during this window, a weighted average was calculated, based on the proportion of time spent on a given response within the 4 to 6 second window. This window was selected to fall within a steady state period of the experimental stimulus on the basis of Fig 1 of [16]. Before this steady state period listeners tend to be biased towards fused percepts, and gradually transition to a given balance between fused and segregated responses in a manner specific to the stimulus employed, a process called “buildup”. Note that the rate of buildup in [16] differs from several other recent studies [65, 91, 92]), probably due to variations in stimulus and/or experimental design. When estimating the distribution of individual percept lengths across all listeners (Fig 2B), we followed the same procedure used for the model output: we removed the first percept and treated any responses less than 250 ms long as spurious (this latter step eliminated %1.2 of the data).

The new group of listeners included a total of 35 normal-hearing participants (22 females) with an average age of 24.5. Listeners were recruited from the University of Nevada, Las Vegas community and paid for their participation. During the condition documented here, participants were presented the stimulus described above with a base frequency of 400 Hz, presented over 3M E-A-RTONE insert ear phones in a sound attenuated booth, at a level of 60 dB. There were a total of 24 trials broken into three 8-trial blocks. Each trial of the stimulus was 48 seconds long, consisting of a total of 100 repetitions of the ABA sequence. For the duration of each trial, listeners were asked to continuously report whether they heard fused or segregated percepts. Participants began the experiment with a short practice session during which the galloping rhythm of the fused percept, and the more steady rhythm of the segregated percept were explained.

Statistical analyses

All reported confidence intervals are computed using a non-parametric bootstrap [125] with 10,000 samples. The density curves shown in Fig 2B were computed using a method for log-space density estimation [126] with a Normal kernel, where kernel bandwidths were determined by the Silverman rule [127].