An introduction to mechanisms of visual attention

An Introduction to“Mechanisms of Visual Attention:

A Cognitive Neuroscience Perspective”

Werner X. Schneider

Departm ent of Experim ental Psychology,L udwig-Maxim ilians-University,

Munich,Germany

Thereis agrowing consensus thatthecomputationof the“internal visual world”does not occur in a purely automatic and obligatory way.Humans process part of the available visual information with priority dependent either on the current task(intention)or the structure of the stimulation. This assignment of priority in visual processing is usually termed“visual attention”.Most of what we currently know about visual attention is based on behavioural studies in humans.Such experimental paradigms as“spatial pre-cueing”(e.g.Posner& Petersen,1990),“visual search”(e.g.Wolfe,1994),“partial report”(e.g.van der Heijden,1992),“flanker interference”(e.g.Eriksen,1995)and“negative priming”(e.g.Fox,1995)have been developed in the tradition of experimental psychology over the last decades.During the past few years,an increasing number of neuropsychological and neuroscience studies on visual attention have extended the database.Neuropsychological experiments with brain-dam-aged patients and animals have attempted to identify the role of different brain areas for the control of visual processing priority(for overviews,see Driver& Mattingley,1995;Duncan,Humphreys,Ward,&Shapiro,1997;Humphreys& Bruce,19;Posner&Petersen,1990);the dependent variables in the main have been behavioural measures,such as reaction time and accuracy.In the neurosciences,the effects of visual attention have been measured in modula-tions of brain activity.At the level of individual neurons,single-unit recordings have revealed firing rate changes of attentional manipulations(for overviews, see Desimone & Duncan, 1995;Duncan et al.,1997).Event-related potentials (ERP)derived from very large populations of neurons have provided insights

R equests for reprints shouldbeaddressed to Werner X.Schneider,Department of Experimen-tal Psychology,Ludwig-Maximilians-University,Leopoldstr.13,80802Munich,Germany.E-mail: wxs@mip.paed.uni.muenchen.de

I would like to thank Heiner Deubel,Martin Eimer and especially Martin Fischer for constructive comments on an earlier draft of the manuscript.

1998 Psychology Press Ltd2SCHNEIDER

into the temporal structure of prioritized processing(for overviews,see Eimer, this issue;Hillyard,Mangun,Woldorff,&Luck,1995).Information about the brain site of attention-dependent neural activity modulations has been gained by functional brain imaging methods such as PE T and fMRI(e.g.Corbetta, Shulman, Miezin, &Petersen,1995; Posner &Raichle,1994).

In summary,our scientific knowledge about visual attention has increased considerably in the last decades and now consists of behavioural as well as neurobiological findings.However,there is still not much consensus about the mechanisms that underlie the data from experiments on attention. The critical issues include:Where within the processing system does priority assignment and selection occur(“early”and/or“late”)?How is selection realized?Is it space-and/orobject-based?How is visual attentioncontrolled?(Foroverviews, see Allport,19,1993;Johnston&Dark,1986;Schneider,1993.)Further-more,the assumption of a single selection mechanism has recently been challenged(see Allport,19,1993).It cannot be taken for granted that the behavioural measures of“visual attention”in different experimental para-digms,such as spatial pre-cueing,partial report and negative priming,and in neuropsychological“attentiondisorders”(suchas differentversions of neglect) refer to the same mechanism.Simply using the same term,namely“visual attention”, does not verify the assumption of a single selection mechanism. Depending on the answers to the above questions,different and partly incompatible theories of visual attention have been postulated over the last 20 years(see Bundesen,1990;Duncan&Humphreys,19;LaBerge&Brown, 19;Posner&Petersen,1990;Treisman,1988;Treisman&Gelade,1980; van der Heijden,1992;Wolfe,1994;for overviews,see Allport,19,1993; Schneider,1993).Fortunately,the recent inclusion of neurobiological data on attentional effects and on the architecture of the visual system has helped to constrain this theorizing.Based in part on these additional neurobiological constraints and on new behavioural data,a move to a consensus on the central issues has taken place.One example of suchconvergence is the assumption that at least one attentional mechanism in vision is space-based(see Bundesen,this issue;Humphreys&Heinke,this issue;LaBerge&Brown,19;Logan,1996; Olshausen,Andersen,&VanEssen,1993;Schneider,1993;van der Heijden, 1992; Wolfe,1994).

This approach to explicitly relate data and theories from cognitive psychol-ogy andneuroscienceinanattempttounderstandclassical“cognitive”domains such as visual attention has been labelled“cognitive neuroscience”(for over-views,see Gazzaniga,1995;Humphreys&Bruce,19;Kosslyn&Koenig, 1992;Posner&R aichle,1994).As noted above,an advantage of this approach is that information about the architecture of the visual system and about the effects of visual attention manipulations at the level of brain activity(single cells,activation of whole brain areas)provide additional constraints for resolv-ing debated issues on the nature of visual attentional mechanisms.INTRODUCTION3 A second promising prospective to tackle these issues is to ask what kinds of functions visual attentionprocesses might serve.Usually,empirical research focuses on certain experimental paradigms,such as visual search or spatial pre-cueing,but does not consider the possible functions of attention(for discussions of this issue,see Allport,1987;Neumann,1987;Prinz,1983; Schneider, 1993). However, it is reasonable to assume that attentional mecha-nisms have evolvedtofulfil certain basic functions.“Selection-for-action”(e.g. Allport,1987)and“selection-for-object-recognition”(e.g.LaBerge&Brown, 19)are examples of such basic functions.Given several candidate functions, further questions about the attentional mechanisms arise.Are the suggested underlying mechanisms for different functional domains(e.g.spatial–motor action,object recognition) identical or different?If they are different,are they coupled or independent?Attempts to answer these questions(e.g.Deubel, Schneider,&Paprotta,this issue)might generate new and productive con-straints for conceptualizing attentional mechanisms.

With these two perspectives in mind,namely the cognitive neuroscience approach and an emphasis on attentional functions,Sabine Maasen and I organized a symposium on visual attention mechanisms in Holzhausen,Ger-many.Consequently,we invited neural network and mathematical modellers, as well as experimentalists using notjustbehavioural but alsoneuropsychologi-cal methods and neuroscience methods,to attend.Furthermore,researchers focusing either on mechanisms in different functional domains(e.g.selection-for-action,selection-for-object-recognition)or on certain aspects of these pu-tative mechanisms(e.g. control) were invited.

The papers in this special issue are based on the invited talks at the symposium.The structure of the issue consists of four sections on visual attention mechanisms.The first three section reflect different functional do-mains of attentional mechanisms,whereas the fourth section deals with issues of mechanisms beyond individual functions.

The first section is entitled“Visual attention mechanisms for shape-based object recognition”and contains papers by Humphreys and Heinke and by Hummel and Stankiewicz.Humphreys and Heinke present a neural network (connectionist)model of a visuospatial attention mechanism,called the Selec-tive Attention for Identification Model(SAIM).The function of the suggested attentional mechanism is to allow translation-invariant shape-based object recognition.One goal of SAIM is to explain neuropsychological data on different versions of attentional disorders,that is,“space-and object-based”neglect.By lesioning different parts of network modules,the simulations show how these attentional disorders can arise.

Hummel and Stankiewicz introduce a neural network model that is based on the assumption that shape-based object recognition works by relying on struc-tural descriptions of parts and their spatial relationships.An attentional process is introduced that is different from the spatial attention window proposed by4SCHNEIDER

Humphreys and others.Its function is to mediate part-based recognition at the class-and instance-level.The network simulations show,for instance,that reduced“attention”still leads toclass-level recognitionbutnottoinstance-level recognition.

The second section is entitled“Visual attention mechanisms for perception and for the control of spatial-motor actions”and contains papers by Deubel, Schneider and Paprotta and by Craighero,Fadiga,Rizzolatti and Umiltà.The paper by Deubel et al. investigated if and how the selective programming of a reaching movementtoanobjectinspace(selection-for-action)andtheselective perceptual processing of an object(selection-for-object-recognition)are cou-pled.The experimental paradigm required subjects to recognize a discrimina-tion target among distractors while simultaneously preparing a movement to a reaching target.Their results show that discrimination performance is best by far when the discrimination target and the movement target refer to the same object in space.The results indicate a coupling of visual attention functions by one common mechanism.It may be the same kind of mechanism Humphreys and Heinke have proposed for solving the problem of translation-invariant object recognition.

Craighero et al.investigated whether perceptual processing of an object influences motor programming,or,in terms used by the authors,whether “visuomotor priming”occurs.In two experiments,subjects had to grasp an oriented bar.Before a“go”signal allowed this motor action,a visual prime stimulus was presented.The orientation of the prime was either congruent or incongruent with the orientationof the to-be-graspedbar.Craighero etal.found a priming effect,that is,grasping initiation time was shorter in the congruent that in the incongruent condition.The authors discuss their results in terms of the“premotor theory of attention”,which claims that a spatial attention mecha-nisms is controlled by brain areas responsible for motor programming of saccadic eye movements and arm movements.

The third section is entitled“Visual attention mechanisms for the construc-tion(binding)of integrated internal objects”.It contains papers by Irwin and Gordon,by Wojciulik and Kanwisher and by Hommel.Irwin and Gordon present experimental data on how the selective construction(priority assign-ment)of visual object representations might work across saccadic eye move-ments.Two experiments show first that a limited number of objects are stored in“transsaccadic memory”—the site of the visual representation of the world across saccadic eye movements—and,second,that a spatial attention mecha-nism determines the content of this memory.Only attended objects can be represented in transsaccadic memory.Furthermore,a neurocognitive atten-tional mechanism is suggested that is assumed to regulate access to transsac-cadic memory and the setting up of integrated object representations.This mechanism might also be used for selection-for-object recognition and for selection-for-spatial-motor-action.INTRODUCTION5 The paper by Woljciulik and Kanwisher presents the results of a neuropsy-chological study of the role of the parietal lobe in the selective binding of individual features into representations of an integrated object.The abilities of the patient RM(who suffered bilateral parieto-occipital damage)for implicit and explicit feature binding were investigated in two experiments using a Stroop task.The result show a dissociation between both forms of binding;that is,no explicit but implicit binding.This hints at the possibility that the parietal lobe might be necessary for explicit but not for implicit binding.These results represent a real challenge for any neurocognitive theory of visual attention mechanisms(e.g. Duncan,1996; LaBerge, 1995; Schneider, 1995).

The paper by Hommel also suggests a role for attention in binding visual features into integrated object configurations,called“object files”by Kahne-man,Treisman and Gibbs(1992).Three experiments focus on whether action-related information is part of these object files.Compatible with Hommel’s expectation,his results show that response features(spatial patterns for keypress)are also part of the internal object configuration—the term“event file”is suggestedforthis structure.Hommel concludes thatthebinding function of attention not only consists of setting up visual perceptual representations of objects,but also includes action-or response-related representations within the same object representations.

The fourth section is entitled“Visual attentional mechanisms beyond indi-vidual functions”and contains papers by Hahn and Kramer,by Eimer and by Bundesen.Hahn and Kramer ask whether the often suggested visuospatial attention mechanism can simultaneously be allocated to several locations in space.Fivereactiontime experiments are reportedthat requiredsame–different matching with target letters embedded among distractor letters. The results of these experiment are hard to reconcile with the claim that“attentional re-sources”can only be allocated contiguously in space and they can be taken as evidence for the ability to“split attention”among several locations.In the discussion,these data are related to a neurocognitive model of attention by LaBerge (1995).

The paper by Eimer reviews a number of experiments that measured atten-tional effects in form of event-related brain potentials(ERPs).Distinct ER P components(P1, N1, Nd1,N2pc)are described which were noted in different attentional tasks,such as sustained and transient spatial pre-cueing.The author argues that these distinct ERP components can be localized in different parts of the brain and may in part reflect distinct attentional mechanisms.How the underlying mechanisms of these effects might be conceptualized is also dis-cussed in terms of the neuroanatomically based model of LaBerge (1995). The final paper,by Bundesen,presents a mathematical theory of attentional mechanisms with a large explanatory range.The development of the theory from a simple choice and race model towards a computational Theory of Visual Attention(TVA)is outlined.The TVA assumes two attentional mechanisms,6SCHNEIDER

called“filtering”and“pigeonholing”,that together determine the final result of prioritization in vision.Very recent extensions of the TVAby Logan(1996), which allow the integration of perceptual grouping effects on visual selection efficiency,are also described.Finally,by localizing some of TVA’s basic components in different parts of the brain,neuropsychological data on neglect are explained.

The symposium took place in Holzhausen near Munich between3and5 August1996.The following researchers participated as invited speakers and/or discussants:AlanAllport,Claus Bundesen,Leonardo Chelazzi,Heiner Deubel, Martin Eimer,Bernhard Hommel,John Hummel,Glyn Humphreys,Dave Irwin,Gordon Logan,Wolfgang Prinz,Werner Schneider,Carlo Umiltàand Ewa Wojciulik.Art Kramer was also invited to give a talk but could not attend; however,he and his colleague Sowon Hahn presented an exciting paper. Furthermore,the following were guests at the symposium:Bruce Bridgeman, Dietmar Heinke, Don MacKay,Lex van der Heijden and Rainer Wolff. All of these people madethe symposium avery stimulating and productive enterprise. However,two individuals require a special mention.First,Wolfgang Prinz can be considered the father of this project:He explicitly encouraged us toorganize the symposium.Also,he made available money from the Max-Planck Society to finance the whole project.The other person without whom the symposium would not have been possible is Sabine Maasen.She managed all the financial and practical aspects, including finding the venue, and helped to create a very lively and productive atmosphere that shaped the overall symposium.More-over,Sabine did an excellent job as co-editor of this special issue.

Review procedures and reviewers

All contributions to this special issue were reviewed by two or three reviewers. For each paper,one of the reviewers was a symposium participant,while the other one or two were not.All reviewers,except those for the paper by Deubel, Schneider and Paprotta,were selected by myself;Dave Irwin selected the reviewers for the paper by Deubel et al.The reviewers are listed below in alphabetical order;the reviewers chosen by Dave Irwin are not listed because they are not known to the editors (WXS,SM).Overall,this special issue owes much to the comments and recommendations of all the reviewers.

Alan Allport, University of Oxford,UK

Claus Bundesen, University of Copenhagen, Denmark

Leonardo Chelazzi, University of Verona, Italy

Heiner Deubel, Ludwig-Maximilians-University, Munich, Germany Martin Eimer, Ludwig-Maximilians-University, Munich, Germany Martin Fischer, Ludwig-Maximilians-University, Munich,Germany John Henderson, Michigan State University,USA

INTRODUCTION7 Bernhard Hommel, Max-Planck-Institute for Psychological Research,

Munich, Germany

John Hummel, University of California,Los Angeles,USA

Glyn Humphreys,University of Birmingham,UK

Dave Irwin, University of Illinois, Champaign, USA

Sten F. Larsen, University of Aarhus,Denmark

Kevin O’Regan, CNRS,Paris,France

Bruno Olshausen,University of California, Davis, USA

Hal Pashler,University of California, San Diego, USA

Wolfgang Prinz,Max-Planck-InstituteforPsychological R esearch,Munich, Germany

Carlo Umiltà, University of Padova, Italy

Lex van der Heijden, University of Leiden,The Netherlands

Dirk Vorberg,Technische Universität Braunschweig,Germany

Ewa Wojciulik,Harvard University,Cambridge,MA, USA

Josef Zihl,Max-Planck-Institute for Psychiatry,Munich, Germany

REFERENCES

A llport,D.A. (1987).Selection for action: Some behavioral and neurophysiological considera-

tions of attention and action.In H.Heuer&A.F.Sanders(Eds),Perspectives on perception and action(pp.395–419). Hillsdale, NJ:Lawrence E rlbaum A ssociates Inc.

A llport,D.A.(19). Visual attention.In M.I.Posner(Ed.),Foundations of cognitive science

(pp.631–682). Cambridge, MA: MIT Press.

A llport,D.A.(1993). A ttentionandcontrol.Have webeenasking thewrong questions?Acritical

review of twenty-five years.In D.E.Meyer&S.Kornblum(Eds),A ttention and performance X IV:Synergies in experimental psychology,artificial intelligence,and cognitive neuroscience (pp.183–218). Cambridge, MA: MIT Press.

Bunde sen,C.(1990). A theory of visual attention.Psychological Review, 97,523–547. Corbetta,M.,Shulman,G.L.,Miezin,F.M.,&Petersen,S.E.(1995). Superior parietal cortex activation during spatial attentionshifts andvisual feature conjunction.Science,270,802–805. Desimone,R.,&Duncan,J. (1995). Neural mechanisms of selective visual attention.A nnual R eview of N euroscience, 18, 193–222.

Driver,J.,&Mattingley,J.B.(1995). Selective attention in humans:Normality and pathology.

Current Opinion in N eurobiology,5, 191–197.

Duncan,J. (1996). Cooperating brain systems in selective perception and action.In T.Inui& J.L.McClelland(Eds),A ttention and performance X VI:Information integration in perception and communication(pp. 549–578).Cambridge, MA:MIT Press.

Duncan,J.,&Humphreys,G.W. (19). Visual search and stimulus similarity.Psychological R eview, 96, 433-458.

Duncan,J.,Humphreys,G.W.,&Ward,R.(1997). Competitive brain activity in visual attention.

Current Opinion in N eurobiology,7, 255–261.

E geth,H.E. (1977). A ttention andpreattention.In G.H.Bower(Ed.),The psychology oflearning

and motivation(Vol. 7, pp.277–320).New York: A cademic Press.

E riksen,C.W. (1995).Theflankers task andresponsecompetition:Auseful tool forinvestigating

a variety of cognitive problems.Visual Cognition, 2, 101–118.8SCHNEIDER

Fox, E.,(1995). Negative priming from ignored distractors in visual selection: A review.

Psychonomic B ulletin and Review, 2, 145–173.

Gazzaniga, M.S.(1995).T he cognitive neurosciences.Cambridge,MA:MIT Press. Hillyard,S.A., Mangun,G.R., Woldorff,M.G.,& Luck,S.J.(1995). Neural systems mediating selective attention.In M.S.Gazzaniga(Ed.).The cognitive neurosciences.Cambridge,MA: MIT Press.

Humphreys,G.W.,&Bruce,V.(19).Visual cognition.Hove:Lawrence Erlbaum A ssociates Ltd.

Johnston,W.A.,&Dark,V.J.(1986). Selective attention.A nnual Review of Psychology,37, 43–75.

Kahneman,D.,Treisman,A.M.,&Gibbs,B.J.(1992). The reviewing of object files:Object-specific integration of information.Cognitive Psychology,24,175–219.

Kosslyn,S.M.,&Koenig,O.(1992).Wet mind:T he new cognitive neuroscience.New York: The Free Press.

LaBerge, D.(1995).A ttentional processing.Cambridge,MA:Harvard University Press. LaBerge,D.,Brown,V.(19).Theory of attentional operations in shape identification.Psy-chological Review, 96, 101–124.

Logan,G.D.(1996). The CODE theory of visual attention:An integration of space-based and object-based attention.Psychological R eview, 103, 603–9.

Neumann,O.(1987).Beyond capacity:A functional view of attention.In H.Heuer&A.F.

Sanders(Eds),Perspectives on perception and action(pp.361–394).Hillsdale,NJ:Lawrence

E rlbaum A ssociates Inc.

Olshausen,B.A.,Anderson,C.H.,&Van E ssen,D.C. (1993).Aneurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information.The Journal of N euroscience, 13, 4700–4719.

Posner, M.I., & Petersen,S.E.(1990). The attention system of the human brain.A nnual Review of N euroscience, 13,25–42.

Posner,M.I.,&Raichle,M.E.(1994).Images of mind.New York:Scientific A merican Library. Prinz,W. (1983).Wahrnehmung und Tätigkeitssteuerung.Berlin:Springer-Verlag. Schneider,W.X.(1993). Space-based visual attention models and object selection:Constraints, problems,and possible solutions.Psychological Research,56,35–43.

Schneider,W.X.(1995). VAM:Aneuro-cognitive model for visual attention control of segmen-tation,object recognition,and space-based motor action.Visual Cognition,2, 331–375. Treisman,A.(1988). Features and objects:The fourteenth Bartlett memorial lecture.Quarterly Journal of E xperimental Psychology,40, 201–237.

Treisman,A.,&Gelade,G.(1980). Afeature-integration theory of attention.Cognitive Psychol-ogy, 12, 97–136.

Van der Heijden, A.H. (1992).Selective attention in vision.London:R outledge.

Wolfe,J.M.(1994). Guided search2.0:Arevised model of visual search.Psychonomic B ulletin and Review, 1, 202–238.