Visual Image Processing: We developed a framework to simulate and compute human visual performance based on the ideas of implicit masking, nolinear processes, and other well-known properties of the visual system that have been used in many models. The basic functional components of this model include a front-end low-pass filter, a retinal nonlinearity, a cortical frequency representation and a frequency-dependent nonlinear process, and finally a decision stage.

Low-pass Filtering: When the light modulated information of an image enters into human eyes, it passes through the optical lens of the eye and is captured by photoreceptors in the retina. One function of photoreceptors is to sample the continuous spatial variation of the image discretely.The cone signals are further processed through horizontal cells, bipolar cells, amacrine cells, and ganglion cells with some re-sampling. From an image processing point of view, the effects of optical lens, sampling, and re-sampling in the retinal mosaic are low-pass filtering.          We estimate the front-end filter from psychophysical experiments. It has been shown that the visual behavior at high spatial frequencies follows an exponential curve. Yang et al. (1995) extrapolated this relationship to low spatial frequencies to describe the whole front-end filter with an exponential function of spatial frequency:

LPF(f)  = Exp(-a f),                                                                             (1)

Where a is a parameter specifying the rate of attenuation for a specific viewing condition. Yang and Stevenson (1997) modified the formula to account for the variation in a with the mean luminance of the image:

a = a₀ + d L₀^0.5,                                                                                  (2)

where a₀ and d are two parameters and L₀ the mean luminance of the image.

Retinal Compressive Nonlinearity: In the retina, there are several major layers of cells, starting from photoreceptors including rods and three types of cones, to horizontal cells, bipolar cells, amacrine cells, and finally to ganglion cells where the information is transmitted out of the retina via optic nerve fibers to the central brain. Retinal processes include a light adaptation, where the retina becomes less sensitive if continuously exposed to bright light. The adaptation effects are spatially localized . In the current model, the adaptation pools are assumed to be constrained by ganglion cells with an aperture window:          

W_g(x, y) =Exp[-(x² + y²)/(2r_g²)]/(2pr_g²),                                            (3)

where r_g is the standard deviation of the aperture.  The adaptation signal at the level of ganglion cells I_g is the convolution of the low-passed input image I_c with the window function W_g.  In this algorithm, the window profile is approximated as spatially invariant by considering only foveal vision. The retinal signal I_R is the output of a compressive nonlinearity.  The form of this nonlinear function is assumed here to be the Naka-Rushton equation, which has been widely used in models of retinal light adaptation.  One major difference here is that the adaptation signal I_g in the denominator is a pooled signal, which is similar to a divisive normalization process:         

I_g = w₀ (1 + I₀ⁿ)I_cⁿ/(I_gⁿ + I₀ⁿw₀ⁿ),                                                                                         (4)

where n and I₀ are parameters that represent the exponent and the semi-saturation constant of the Naka-Rushton equation, respectively, and w₀ is a reference luminance value. In conditions where I_c and I_g are all equal to w₀, the retinal output signal is the same as the input signal strength.

Cortical Compressive Nonlinearity: Simple cells and complex cells in the visual striate cortex usually respond to stimuli of limited ranges in spatial frequency and orientation. To capture this frequency- and orientation-specific nonlinearity, one can transform the image I_R from a spatial domain to a frequency domain representation via a Fourier transform to T(f_x, f_y), and divided by n_x and n_y to normalize the amplitude in the frequency domain.  Here f_x and f_y are the spatial frequencies in x and y directions, respectively, and n_x by n_y is the number of image pixels.
     These cells also exhibit nonlinear properties; their firing rate does not increase until the stimulation strength is above a threshold level and the firing rate saturates when the stimulation strength is very strong. In the model calculation, the signal in the frequency domain passes through the same type of nonlinear compressive transform as it did in the retinal processing.Following the concept of frequency spread in implicit masking (see Fig 2), one major step here is to compute the frequency spreading that affect the masking signal in the denominator of the nonlinear formula. In this model, the signal strength in the masking pool, T_m(f_x, f_y), is the convolution of the absolute signal amplitude |T(f_x, f_y)| and an exponential window function:

W_c(f_x, f_y) = Exp[-(f_x² + f_y²)^0.5/s],                                                         (5)

Where s correlates with the extent of the frequency spreading and the bandwidth of frequency channels. As the bandwidth of frequency channels increases with the spatial frequency, one should expect that the s value increases with spatial frequency. To simplify the computation, however, this value is approximated as a fixed value in the current algorithm. Applying the same form of compressive nonlinearity as in the retina, the cortical signal in the frequency domain is expressed as:                                           

T_c = sign(T) w₀ (1 +T₀^v) |T|^v/(T_m^v + T₀^vw₀^v) ,                                        (6)

where v and T₀ are parameters that represent the exponent and the semi-saturation constant of the Naka-Rushton equation for the cortical nonlinear compression, respectively. The term T_m in the denominator includes the energy spread of the DC component (i.e., at 0 cpd) of the spatial pattern.  This component is processed in the same way as other frequency maskers, if there are any, under Eq. 6.  Thus, the concept of implicit masking is naturally implemented in the image processing framework. In summary, the major process in the cortex is modeled by a compressive nonlinearity applying to the spatial frequency and orientation components. The cortical image representation in the frequency domain is given by the function T_c.This function can be used to calculate visual responses and to simulate visual performance on detecting spatial patterns, as well as for estimating perceived brightness.