,Journal of J. Math. Biology (1985) 21:273-283 Mathematical Biology 9 Springer-Verlag 1985 Eye movements and the enhancement of edges Steven D. Woods, ~ Richard H. Rand, 2 H. David Block*, 2 and Donald C. Lewis 2 Department of Mathematics, Statistics, and Computer Science, Marquette University, Milwaukee, WI 53233, USA 2 Department of Theoretical and Applied Mechanics, Cornell University, Ithaca, NY 14853, USA Abstract. A mathematical examination of retinal photochemistry leads to a hypothesis for Mach band phenomena based on eye movements. This retinal model suggests why minimally distinct borders fade under eye fixation and agrees qualitatively with subjective measures of border contrast as a function of overall field luminance. Key words: Eye movements - - edge enhancement - - mach bands Introduction When one's visual environment is divided into uniform-luminance half-fields of white and black as shown in Fig. 1, the border between the fields is enhanced by the appearance of Mach bands on either side. On the dark side of the edge, the band appears blacker than the black half-field, and on the white side of the edge the band appears brighter than the white half-field. These two bands are a perceptual rather than a luminance or reflective p h e n o m e n o n and were attributed by Mach [8] to the presence of lateral inhibition mechanisms in the visual system. Solid evidence for the existence of these lateral inhibition (or center-surround) mechanisms has been found in both retinal and cortical unit recordings taken Fig. 1. The Mach pattern * Deceased. 274 s.D. Woods et al. from many species [9, 7, 13, 8] and consequently Mach's original explanation of this p h e n o m e n o n has become generally accepted. We will present an explanation for Mach bands quite different than Mach's own. Our approach depends explicitly on eye movements and properties of retinal photochemistry. We will show that as an image of these uniform half-fields moves across the retina the cells which are stimulated by both fields produce a time varying output which differs from the steady state outputs which are produced by cells stimulated exclusively by one region or the other. When the image shifts so as to place one of these transiently responding cells on the white side of the border, we will show that it responds at a level greater than any cell receiving steady state stimulus from the white half-field. Likewise when the image shifts and places a transiently stimulated cell on the dark side of the border, its response will be less than any cell receiving steady state stimulus from the dark half-field. These transiently stimulated photoreceptors are thus responsible for the presence of Mach bands. Model of photochemistry Rhodopsin consists of a protein called opsin which carries the chromophore, retinaldehyde. This aldehyde of vitamin A is referred to simply as retinal. In the resting state the retinal is in its specific 11-cis form and fits closely into the opsin part of the rhodopsin molecule. When this chromophore absorbs light, it isomerizes and goes to its all-trans configuration. After a number of thermal reactions molecule is eventually hydrolyzed into all-trans retinal a n d opsin. After hydrolysis, the retinal is reduced to an equilibrium state with vitamin A by means of an enzymatic reaction. Before rhodopsin can be regenerated, retinal must be available in its 1 l-cis form. This energy-consuming reaction proceeds via an enzyme which promotes the re-formation of retinal from vitamin A. As all-trans is converted to 11-cis retinal, the equilibrium is disturbed and more retinal is in its 11-cis form it will combine spontaneously with available opsin to yield rhodopsin. This spontaneous combination gives up energy which promotes the oxidation of vitamin A to retinal. Because there is generally a reserve of vitamin A in the pigment epithelium (with which the cones have contact)this regeneration is stopped when all the opsin present has combined with retinal [3, 5]. We will not model the full complexity of this photochemical process. It proves sufficient merely to distinguish between light-sensitive versus light-insensitive chemical types. Rhodopsin (with its 11-cis form of retinal) is light-sensitive and we denote its concentration as a function of time as r = r(t). We will consider all other chemical species, including all-trans retinal, opsin, vitamin A and so forth, to be light-insensitive even though there is some photoreversal and some iso-rhodopsin formation, and we denote their concentration collectively as r ' = r'(t). The number of light quanta incident per unit volume of retinal cell we denote by z = z(t). First, we treat the light-induced isomerization of retinal as k r + z ~r' (1) Eye movements and the enhancement of edges 275 and we suppose that the chemical reisomerization is given by u l r t , r. ) (2) u2 A system of differential equations which corresponds to (1) and (2) is obtained as usual [12] and can be written as i = - k r z + ul r' - u2r (3) i"= + k r z uzr. (4) r + r ' = r r = constant (5) - ulr'q- The first integral, allows (3) and (4) to be reduced to f = - k r z + u l ( r r - r) - u2r (6) i = - k r z + ~ ( S * - r) (7) /3 = ul + u2 (8) S* = u l r T / ( u l q- U2). (9) T = ( u l + u2)t (10) S('r) = r("r/(u, + u2)) = r(t) (11) Z ( . r ) : z(~'/ (u, + u=) ) : z( t) (12) or, upon regrouping where and By rescaling time and letting and we have (13) S= -aNZ+(S*-S) where dS IS=--o dr (14) The system determined by (1) and (2) is represented by a linear, first-order differential equation. This equation (13) was derived by Cornsweet [2] in a somewhat different manner. With initial condition S(0) = So the solution to this equation is 1 , fof(C) J ' dt,] (15) 276 S . D . W o o d s et al. where f(t) = e ~p(')+t (16) and p(t)= z(r) dt'. (17) Since we have the closed form solution (15), we need only specify Z(t) in order to obtain S(t), the concentration of photopigment as a function of time. This concentration level affects how sensitive the receptor cell is to incoming light, but is not itself proportional to the output of the cell. For example, if the intensity of the light stimulus increases we expect that the output of the cell should also increase. However the effect of a brighter stimulus is to deplete the amount of pigment available. In the next section we propose an explicit model of cell output. A model of cell output Let Z(t) be the light intensity falling on the photoreceptor cell (cell input) and let h(t) be the electrical activity of this cell (cell output). See Fig. 2. We wish to find h(t), given Z(t). Now let S(t) be the concentration of rhodopsin in the cell which is available for interaction with light. At time t, the cell amplification may be defined as amplification = output = h (t) input Z ( t )" (18) We shall assume that the amplification is proportional to the concentration of rhodopsin: h(t) z(t) = kS(t). (19) This assumption is based on the argument that the more rhodopsin there is present, the higher the probability will be that a photon entering the cell will strike a rhodopsin molecule. In fact, the slope of this amplication function may Z ( t ) = tight intensity ( Receptor cett h ( t ) = cett output Fig. 2. S c h e m a t i c d i a g r a m of a single retinal cell Eye movements and the enhancement of edges 277 change during the course of adaptation but we use the linear form for the sake of mathematical simplicity. Using the closed form solution (15) of the linear reaction equation we can easily examine the cell output which results from a constant light stimulus. According to (19), the instantaneous cell output corresponding to the concentration of retinal S(t) is h( t) = kZ( t)S( t). (20) If the cell is kept in total darkness for t < 0 so that So = S* and if, for t i> 0, we suppose that Z(t)= Zo = constant, then p(t)= Zot and we find that h( t) = ho[1 + aZo e -(~'z~ (21) with kZoS* ho - 1 + a-Zo" (22) Equation (21) is plotted in Fig. 3 as a function of t. Note that h(t) = ho is the steady state output level corresponding to Z(t)= Zo. Figure 4 displays ho as a function of Zo. Note here the saturation p h e n o m e n o n common to m a n y biological systems. h(tl l kZoS*-~~. ho1 , t Fig. 3. Cell response to a constant light stimulus Z o h0 kS ~ Z0 Fig. 4. Steady state cell output as a function of light intensity 278 S . D . Woods et al. Eye movement The light stimulus which falls on the retina is, in any familiar circumstance, changing in time. The movement of objects and changes in the position and intensity of illumination contribute to this time dependence, but also the eye itself is in constant motion. Even though the perception of a retinal image can remain stationary during these involuntary eye movements, it is incorrect to think that these movements are incidental to perception. In remarkable experiments performed by Ditchburn and Ginsborg [4] and by Riggs, Ratliff, Cornsweet and Cornsweet [10] it was found that when a retinal image is stabilized all the contours and discontinuities of the image fade from view. Eventually the image vanishes entirely and the visual field appears uniformly grey. This effect demonstrates that continual eye motion is in fact crucial to visual perception. Of course, these same remarks apply to the sensation of Mach bands in that if Mach's pattern is stabilized on the retina, the entire pattern along with the bands will steadily grow dim and vanish [11]. But even more can be said. In experiments by Bittini et al. [1] it was found that the Mach bands were enhanced by oscillating the pattern perpendicular to the line of sight. In fact, the spatial luminance gradient which is required for the bands to be just visible can be reduced if the pattern is oscillated. Likewise, a Mach pattern with any given luminance gradient will have its Mach bands enhanced by oscillatory lateral movement of the pattern. We will now discuss a model consisting of a one dimensional array of retinal cells each behaving as described previously. This model will provide an explanation of Mach bands based on eye movements and as a result will show why minimally distinct borders fade under fixation. Consider a line of rod cells as in Fig. 5. We shall idealize this situation by considering a continuum of such cells, a given cell corresponding to a position x. We suppose that these cells are exposed to a picture described by Z(x) which is shown in Fig. 6. Here Z1 represents a black region of the picture (low light intensity) while Ze > Z1 represents a white region. The edge is therefore represented by x = 0. If there were no eye movements, the steady state rhodopsin concentration S(x) and the output response h(x) would be as in Fig. 6. In particular, (for i = 1, 2) we have from (20) and (21) that S* Si- - l + aZi' kZiS* hi - - _ 1-l- a Z i kZiSi. Note that $2 < S~ while h2> h~ since Z2> Z~. Fig. 5. One dimensional array of retinal cells. Zk(t) is the light input to the kth cell and hk(t) is the cell output of the kth cell Eye movements and the enhancement of edges 279 Z(x) I Z2 I Z1 ~--- X S(x} I S1 IS h(x) lh I hl X Fig. 6. Stimulus, pigment concentration, and cell output for a motionless eye N o w s u p p o s e that the eye oscillates from left to right across the edge x = 0 in Fig. 6, or equivalently, suppose that the pattern o f Fig. 6 is m a d e to oscillate across the retinal rod cells so that Z ( x ) b e c o m e s Z ( x - A cos wt) as in Fig. 7. Here A > 0 is the amplitude o f the oscillation and o) is its circular frequency. R o d cells in the region x > A receive Z(x, t) = Z] and hence have S(x, t) = &, h(x, t)= hi as before. Similarly rod cells with x < - A receive Z = Z 2 > Z] and have S = S 2 < & , h = h 2 > hi as before. Cells in the " e d g e " region - A < x < A, however, receive a periodic light input. A cell at position x receives Z~ for a duration T~, then Z2 for a duration T2, this alternating pattern repeating ad infinitum. Here T1 and T2 d e p e n d on x, but T~ + T2 = 2rr/w. See Fig. 8. Cells in the edge region near x = A receive Z = Z] most o f the time, with only short bursts o f Z = Z2 (and vice versa for cells near x = - A . ) The cell at x = 0 receives Z~ for half the period and Z2 for the other half. The steady state r h o d o p s i n concentration S(x, t) in an edge region cell at x will behave as in Fig. 9. During the Z] portion o f the cycle, S begins to a p p r o a c h Z(x,t) z21 t= I 2nl~ - w zl: _1 A A -I Fig. 7. Proximal stimulus for lateral eye motion described by x(t) = A cos (wt) 280 S.D. Woods et al. A cos (wt) N \ T1 T 2~ \ b \ \ f -A 3/ \ \ N T~ \ I A Fig. 8. a Light stimulus for a given cell in the edge region as a function of time. b The period of light stimulus for all cells in the edge region -A ~<x ~<A $1 but always remains less than Sl, As soon as Z2 comes on, S begins to approach $2 < $1 but always remains greater than $2. We require S to be continuous in t at the transition times when the input signal jumps between Z1 and Za. (For convenience in Fig. 9 we have shown as dotted lines the steady state values for cells not in the edge regions, i.e. S = $1 and S = $2.) Now consider the resulting steady state cell output h(x, t), also shown in Fig. 9. Here h(x, t) = kZ(x, t)S(x, t) as in (20), so that although S(x, t) is continuous in t, Z(x, t) is not continuous in t and hence h(x, t) is not continuous in t. Moreover, from Fig. 6, h2> h~. Therefore during the Z~ portion of the cycle, h < hi while during the Z2 portion of the cycle h > h2. That is, cells in the edge region produce output signals h(x, t) which are (i) dimmer than the signal hi coming from non-edge black regions ( x > A) and which are also (ii) brighter than the signal h2 coming from nonedge white regions (x < - A ) . When c o m p a r e d with the black region, a region adjacent to the edge seems blacker; when compared with the white region, a region adjacent to the edge Eye movements and the enhancement of edges Z(x,t) 281 ] Z 2- Zl S(x,t) S 1S h(x,t) h h1 TI T1 Fig. 9. Light stimulus, pigment concentration, and cell output for a moving eye seems whiter: this is the phenomenon observed in Mach bands. Note also that the model predicts that the edge effect diminishes as one moves from the middle of the edge region, x = 0, towards the boundaries of the edge region, x = • (since cells which are in the edge region but near x = A ( - A ) have large (small) ratios T1/7"2, are loaded mostly with Z1 (Z2), and hence respond with an average h ~ h I (h2).) We have shown that the enhancement of boundaries between dark and light regions (the Mach band effect) results, at least in part, from eye movement and from the chemical dynamics of the retinal photoreceptors. This approach does not contradict explanations of Mach bands based on lateral inhibition, but note that it has two advantages over such a theory. First, it makes no specific requirements of the connections between retinal cells; it depends only on the first layer of photoreceptors. Second, it incorporates the effects of eye movements, which 282 s.D. Woods et al. are experimentally known to be necessary to visual perception, into the response of photoreceptor cells. It is appropriate to note that this model does not explain the disappearance of contrast which is typical when a retinal image is stabilized. However the model does explain the enhancement of border contrast. Thus, in the case where a border is adjusted to be minimally distinct under normal viewing and an observer is asked subsequently to fixate on the border, the resulting decrease in enhancement will cause the border to fade from view as the photoreceptors tend toward their steady state. Contrast as a function o f luminance In addition to a retina-based explanation of Mach bands this model also provides a context to understand the apparent contrast of a border as a function of overall field luminance. We denote the luminance of the dark half-field as Z1 = Z0 and the luminance of the white half-field as Z2 = yZo with y i> 1. The multiplicative factor 3' measures the luminance contrast between these fields and when y = 1 the two fields are identical (Zo = yZo), therefore having no perceivable border. For a fixed value of y a low luminance field corresponds to a small value of Zo and a high luminance field corresponds to a large value of Zo. Under these stimulus conditions a photoreceptor cell responds maximally when its proximal stimulus has just changed from the low level Zo to the high level yZo. If, in addition, the cell has been sufficiently exposed to the dark half-field then its concentration of light-sensitive pigment will have risen to the steady state level S * / ( a Z o + 1). Using this as the initial condition in equation (1) we calculate that the instantaneous response output of this cell will be Hmax ~ S(O)" (TZo) - S*(TZ~ aZo+ 1 " The corresponding situation in which a cell has been adapted to the white half-field and then exposed to the dark region at t = 0 produces an instantaneous response Hmi n ~ S(O)" Z o - - S*(Zo) - otyZo + 1" As a measure of the perceived contrast at the edge, we use the ratio S* R ( y ; Zo) = Hmax Hmin - - . aZo+ 1 - - S* . c~TZo + 1 (yZo) (Zo) y ( a y Z o + 1) aZo+l ' y~l. Eye movements and the enhancement of edges 283 We can obtain the perceived contrast R at low and high luminances by successively taking Zo = 0 and Zo = co. In those case we have R(3'; 0) = lim R(% Zo) Zo-,O 3'(0+1) 3' 0+ 1 and R(7;co) = lira R(y;Z0) = lira zo~oo zo~ 3'(ar+l/Zo) ~+l/Zo 7a3" _ - - a 2 _ 3' 9 Because. y t> 1, this establishes that R( T; co) = 72>! Y= R( T; O) whicb means that the perceived contract R is much larger at the high luminances signified by Z o ~ c o than it is at the low luminances signified by Z o o 0 . This result supports the observation by Frome et al. [6] that the visibility of a border increases with an increase in overall luminance. Moreover it provides an alternative to their speculation that this effect is due to an increase in the strength of lateral inhibition at high luminances. References 1. Bittini, M., Ercoles, A., Fiorentini, A., Ronchi, L., Toraldo di Francis, G.: Enhanced contrast of an indefinitely contoured object by movement or intermittent illumination. AH~ Fond Giorgio Ronchi 15, 62-84 (1960) 2. Cornsweet, T. N.: Changes in the appearance of stimuli of very high luminance. Psych. Rev. 69, (No. 4) 257-273 (1962). 3. Davson, H.: The Eye. Vol. 2B, 2nd Ed. New York: Academic Press 1977 4. Ditchburn, R. W., Ginsborg, B. L.: Vision with a stabilized retinal image. Nature 170, 36 (1952) 5. Fein, A., Szuts, E. Z.: Photoreceptors, their role in vision. Cambridge: Cambridge University Press 1982 6. Frome, F. S., Buck, S. L., Boynton, R. M.: Visibility of borders: separate and combined effects of color differences, luminance contrast, and luminance level. J. Opt. Soc. Sin. 71,145-150 (1981 ) 7. Hubel, D. J., Wiesel, T. N.: Receptive fields of single neurons in the cat's striate cortex. J. Physiol. 148, 574-591 (1959). 8. Hubel, D. H., Wiesel, T. N.: receptive fields and functional architecture of monkey striate cortex. J. Physiol. London 195, 215-243 (1968) 9. Ratliff, F.: Mach /lands: Quantitive studies on neural networks in the retina. Sar~ Francisco: Holden-Day Inc., 1965 10. Riggs, L. A., Ratliff, F., Cornsweet, J. C., Cornsweet, T. N.: The disappearance of steadily fixated objects. J. Opt. Soc. Am. 43, 495-501 (1953) 11. Riggs, L. A., Ratliff, F., Keesey, U. T.: Appearance of Mach bands with a motionless retinal image. J. Opt. Soc. Am. 51,702-703 (1961) 12. Rubinow, S.: Introduction to Mathematical Biology. New York: John Wiley and Sons 1975 13. Schiller, P. H., Finlay, B. L., Volman, S. F.: Quantitative Studies of Single Cell Properties in Monkey Striate Cortex. I. Spatiotemporal Organization of Receptive Fields. J. Neurophysiol. 39, 1288-1319 (1976) Received April 17/Revised September 4, 1984