How do foveal representations support high acuity vision during eye movements?
Primate vision is unique among mammals in its specialization for high acuity foveal vision. The fovea subtends less than a visual degree in central vision, yet it is critical to how we use vision and is massively over-represented at the level of cortex. Eye movements bring objects of interest to their fovea for detailed inspection typically 2-3 times each second. When there are deficits specific to foveal vision, as in macular degeneration, it is debilitating to everyday activities such as driving, reading, or any skilled movement. Despite the importance of the fovea in human vision, few neurophysiology studies have examined it at the level of cortical processing, primarily due to technical limitations in eye tracking. For most eye trackers the noise floor and calibration error exceeds the size of V1 foveal receptive fields, making it impossible to position stimuli inside receptive fields during traditional behavioral paradigms with central fixation.
Figure 4. Investigations of foveal vision during free-viewing. A: Marmosets free-view full-field noise stimuli and natural images. B: Advanced eye tracking enables us to recover spatio-temporal receptive fields in foveal V1. C: Brief post-saccadic transients during natural image foraging carry the most informative signals to distinguish preferred or non-preferred stimuli in a foveal receptive field.
We have recently overcome these limitations through a collaboration with Michele Rucci’s laboratory. We adapted methods for digital dual-purkinje eye tracking for the marmoset eye and used free-viewing tasks to estimate visual receptive fields of V1 neurons (Yates et al, Beyond Fixation, bioXriv). Marmosets viewed either full-field noise stimuli (flashed Gabors or dots) or natural images as they foraged a screen for a low contrast target (Fig. 4A). Free-viewing tasks were critical for recovering foveal receptive fields because they allow stimuli to appear in the fovea naturally, as opposed to traditional tasks which place a fixation point over the fovea. This allowed us to recover for the first time the spatio-temporal structure of foveal receptive fields (Fig. 4B). In continuing work we are using convolutional neural networks to model complex cell receptive fields and measure how they scale relative to peripheral V1. These models will enable us to understand key differences between foveal and peripheral processing and link them to acuity measured behaviorally (Nummela et al., 2017, Dev. Neuro.). Our preliminary results suggest that the small scale of foveal receptive fields alone is not sufficient to explain acuity achieved in behavior, but will also requires considering the dynamics of eye movements and how they impact visual information.
A key advantage of our free-viewing approach is that it enables us to study how eye movements shape information processing. For example, when marmosets are foraging on natural images we find that V1 neurons undergo strong post-saccadic transients that carry the best visual information about the foveal stimulus (Fig. 4C). In ongoing work we are building models that incorporate how eye movements impact visual encoding which will ultimately enable us to link the neural mechanisms to behavioral acuity.
In a separate line of work, we are investigating whether and how top-down predictions influence post-saccadic foveal processing. Human psychophysics studies show that pre- and post-saccadic views of objects influence perception. However, it is unclear at what level in the visual system that information may be combined. One possibility is that foveal predictions influence early visual processing for the target. We have found some evidence to support a role for prediction in early visual processing at the level of foveal MT. These predictive signals could support trans-saccadic integration for perception or could also influence the planning of predictive eye movements like PFR.
How does marmoset visual behavior compare to humans?
How do top-down attention signals modulate early visual processing?
How do foveal representations support high acuity vision during eye movements?
What role does visual prediction play in guiding eye and hand movements?