Ординатура / Офтальмология / Английские материалы / books.google.com / Visual Perception Fundamentals of awareness multi-sensory integration and high-order perception_Martinez-Conde_2006

16

location relative to a contextually inconsistent location to be significantly greater for objects whose identities have been contextually primed than for contextually unprimed objects, suggesting a unified representation for both nonspatial and spatial factors. In this situation, a similar interaction might also be expected for corresponding cortical activity in brain regions associated with contextual processing (i.e., the PHC and/or RSC) and in object-re- lated processing areas (e.g., fusiform gyrus), with the largest differential cortical activation when the target is consistent with both spatial and nonspatial contextual information. If, however, nonspatial and spatial contextual representations operate independently, their contribution to response latencies, as well as to brain activation, will be additive. In addition, a possible anatomical distinction within the contextual network may be found between the two types of representations (as found previously with spatial and nonspatial associations, Bar and Aminoff, 2003). With such an outcome, one could conclude that the two types of information reside in separate representational ‘‘stores,’’ suggesting dissociable influences in con- text-based top-down facilitation of object recognition. Finally, because spatial and nonspatial contextual associations may involve the top-down influence of both independent and unified representations of spatial and nonspatial contextual associations, it is also possible that the effects of these representations will be additive in contextand ob- ject-related regions, but will interact at higher-level brain regions associated with postrecognition processes. Future research is needed to address these issues, and further increase our understanding of the nature of contextual representations and their effect on visual object recognition.

Integrated objectand context-based top-down facilitation of recognition

In this overview, we have described how top-down facilitation of recognition can be achieved either

(1) through an object-based mechanism that generates ‘‘initial guesses’’ about an object’s identity using rapidly analyzed coarse information about the input image or (2) through the predictive

information provided by contextual associations between an object or a scene and the other objects that are likely to appear together in a particular setting. However, it is clear that objects do not appear in isolation, and that both object-based and context-related information is typically available to the visual system during recognition. It therefore makes sense that an optimized system might take advantage of both forms of top-down facilitation to maximize the efficiency of recognition processes. How is this achieved? In this section, we consider how both object-based and context-based influences might operate together to facilitate object recognition.

Central to this discussion is the observation that recognition of an object can be enhanced through neural mechanisms that sensitize the cortical representations of the most likely candidate interpretations of that particular object before information processed through the bottom-up stream has sufficiently accumulated. We propose two key mechanisms through which this ‘‘sensitization’’ is achieved. As reviewed in the first part of this paper, an objectbased mechanism is proposed to capitalize on a rapidly derived LSF representation of the object itself to generate ‘‘initial guesses’’ about its identity. The back-projection of these candidate interpretations to object-processing regions thereby sensitizes the corresponding cortical representations in advance of the bottom-up information that continues to activate the correct representation. The contextbased mechanism, reviewed in the second part of this chapter, is proposed to sensitize the representations of contextually related objects whose associations are activated through context frames stored in the PHC (see Fig. 9). Importantly, we propose that context frames can be activated following prior recognition of strongly contextual object (as in Fenske et al., submitted), or through early, coarse information about a scene or an object. This includes the possibility that contextual analysis begin promptly, even before a scene or an object is fully recognized. Considered together, the mechanisms for top-down facilitation of object recognition that we describe include one that is based on information from the ‘‘to-be-recognized’’ object itself and the other based on information about the context in which the object appears. Given the intricate links between

objects and the settings in which they appear, these mechanisms typically should not be expected to operate in isolation. Indeed, provided that sufficient information is available for activating the appropriate context frame, and that an LSF image of a single target object is sufficient for limiting its possible interpretations, the intersection of these two sources of information would result in a unique, accurate identification (Bar, 2004). An exciting direction for future research will be to assess how information about additional objects or the scene in which a target object appears may be processed in parallel to increase the opportunities for such valuable intersections of objectand context-based information.

The interactive nature of the two sources of topdown facilitation of object recognition that we have described emerges when the input to either

17

the objector context-based mechanism is ambiguous. For example, when the coarse object-related input to the prefrontal cortex is ambiguous (e.g., a blurred image of an umbrella also resembles a mushroom, lamp, and parasol), the benefit of having activated the appropriate context frame will be relatively greater than if the LSF profile of an object is completely diagnostic of the object’s identity (e.g., a blurred image of a giraffe only resembles a giraffe). In addition, if ambiguous information is projected to the PHC, then this can result in the activation of multiple context frames. From this possibility emerges a rather counterintuitive prediction of our model. Consider, for instance, that a picture of a gun, when projected rapidly in a blurred (i.e., LSF) form to the PHC, may be interpreted also as a drill and a hairdryer (Fig. 11).

18

These three objects are associated with three different context frames, and will subsequently trigger the activation of three sets of objects. Consequently, a gun will not only prime the recognition of a police car (i.e., contextual priming), but also the recognition of a hairbrush (i.e., a member of the context frame activated by the hairdryer). Importantly, because there is a complete lack of any other perceptual, semantic, or contextual relation between a gun and a hairbrush, finding significant priming for this condition is best explained through the interactive model of top-down facilitation that we propose. Indeed, we found significant priming for this condition (Fenske and Bar, submitted), and found further that this form of indirect priming existed for relatively short prime exposures (120 ms) but not for longer prime exposures (2400 ms). This finding underscores the interactive nature of the objectand context-based mechanisms we have described, and supports our notion that the arrival of additional information leaves active only the most relevant ‘‘initial guess’’ about an object’s identity and only the most relevant context frame.

In conclusion, the models and research findings reviewed here emphasize the importance of top-down facilitation in visual object recognition. Building on the previous work (Bar, 2003), we have examined evidence for an object-based mechanism that rapidly triggers top-down facilitation of visual recognition using a partially analyzed version of the input image (i.e., a blurred image) that generates reliable ‘‘initial guesses’’ about the object’s identity in the OFC. Importantly, we have also shown that this form of topdown facilitation does not operate in isolation. Work from our lab indicates that contextual associations between objects and scenes are analyzed by a network including the PHC and the RSC, and that the predictive information provided by these associations can also constrain the ‘‘initial guesses’’ about an objects’ identity. We propose that these mechanisms operate together to promote efficient recognition by framing early information about an object within the constraints provided by a lifetime of experience with contextual associations.

Acknowledgments

This work was supported by NINDS R01NS44319 and RO1-NS050615, NCRR P41RR14075, and the MIND Institute.

References

Aguirre, G.K., Detre, J.A., Alsop, D.C. and D’Esposito, M. (1996) The parahippocampus subserves topographical learning in man. Cereb. Cortex, 6: 823–829.

Aguirre, G.K., Zarahn, E. and D’Esposito, M. (1998) An area within human ventral cortex sensitive to ‘‘building’’ stimuli: evidence and implications. Neuron, 21: 373–383.

Aminoff, E. Gronau, N. and Bar, M. (submitted for publication). The parahippocampal cortex mediates spatial and nonspatial associations.

Bachevalier, J. and Mishkin, M. (1986) Visual recognition impairment follows ventromedial but not dorsolateral prefrontal lesions in monkeys. Behav. Brain Res., 20: 249–261.

Bar, M. (2003) A cortical mechanism for triggering top-down facilitation in visual object recognition. J. Cogn. Neurosci., 15: 600–609.

Bar, M. (2004) Visual objects in context. Nat. Rev. Neurosci., 5: 617–629.

Bar, M. and Aminoff, E. (2003) Cortical analysis of visual context. Neuron, 38: 347–358.

Bar, M., Kassam, K.S., Ghuman, A.S., Boshyan, J., Schmidt, A.M., Dale, A.M., Hamalainen, M.S., Marinkovic, K., Schacter, D.L., Rosen, B.R. and Halgren, E. (2006) Topdown facilitation of visual recognition. Proceedings of the National Academy of Science, 103: 449–454.

Bar, M., Tootell, R., Schacter, D., Greve, D., Fischl, B., Mendola, J., Rosen, B. and Dale, A. (2001) Cortical mechanisms of explicit visual object recognition. Neuron, 29: 529–535.

Bar, M. and Ullman, S. (1996) Spatial context in recognition. Perception, 25: 343–352.

Barcelo´, F., Suwazono, S. and Knight, R.T. (2000) Prefrontal modulation of visual processing in humans. Nat. Neurosci., 3: 399–403.

Biederman, I. (1972) Perceiving real-world scenes. Science, 177: 77–80.

Biederman, I. (1981) On the semantic of a glance at a scene. In: Kubovy, M. and Pomerantz, J.R. (Eds.), Perceptual Organization. Erlbaum, Hillsdale, NJ, pp. 213–253.

Biederman, I., Mezzanotte, R.J. and Rabinowitz, J.C. (1982) Scene perception: detecting and judging objects undergoing relational violations. Cogn. Psychol., 14: 143–177.

Biederman, I., Rabinowitz, J.C., Glass, A.L. and Stacy, E.W. (1974) On the information extracted from a glance at a scene. J. Exp. Psychol., 103: 597–600.

Brewer, J.B., Zhao, Z., Desmond, J.E., Glover, G.H. and Gabrieli, J.D. (1998) Making memories: brain activity that predicts how well visual experience will be remembered. Science, 281: 1185–1187.

Bullier, J. and Nowak, L.G. (1995) Parallel versus serial processing: new vistas on the distributed organization of the visual system. Curr. Opin. Neurobiol., 5: 497–503.

Cave, C.B. and Kosslyn, S.M. (1993) The role of parts and spatial relations in object identification. Perception, 22: 229–248.

Chun, M.M. and Jiang, Y. (2003) Implicit, long-term spatial contextual memory. J. Exp. Psychol. Learn. Mem. Cogn., 29: 224–234.

Davachi, L., Mitchell, J. and Wagner, A. (2003) Multiple routes to memory: distinct medial temporal lobe processes build item and source memories. Proc. Natl. Acad. Sci. USA., 100: 2157–2162.

Davenport, J.L. and Potter, M.C. (2004) Scene consistency in object and background perception. Psychol. Sci., 15: 559–564.

Desimone, R. (1998) Visual attention mediated by biased competition in extrastriate visual cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci., 353: 1245–1255.

Du¨zel, E., Habib, R., Rotte, M., Guderian, S., Tulving, E. and Heinze, H.J. (2003) Human hippocampal and parahippocampal activity during visual associative recognition memory for spatial and nonspatial stimulus configurations. J. Neurosci., 23: 9439–9444.

Engel, A.K., Fries, P. and Singer, W. (2001) Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci., 2: 704–716.

Epstein, R., Graham, K.S. and Downing, P.E. (2003) View- point-specific scene representations in human parahippocampal cortex. Neuron, 37: 865–876.

Epstein, R. and Kanwisher, N. (1998) A cortical representation of the local visual environment. Nature, 392: 598–601.

Fenske, M.J., Boshyan, J. and Bar, M. (submitted) Can a gun prime a hairbrush? The ‘‘initial guesses’’ that drive top-down contextual facilitation of object recognition. Paper presented at the 5th Annual Meeting of the Vision Sciences Society, Sarasota, FL.

Fenske, M.J. and Bar, M. (submitted for publication). Can A Gun Prime A Haitbrush? ‘‘Initial Guesses’’ that Mediate Contextual Facilitation of Object Recognition.

19

Freedman, D.J., Riesenhuber, M., Poggio, T. and Miller, E.K. (2001) Categorical representation of visual stimuli in the primate prefrontal cortex. Science, 291: 312–316.

Friston, K. (2003) Learning and inference in the brain. Neural Networks, 16: 1325–1352.

Gibson, E.J. (1969) Principles of Perceptual Learning and Development. Appleton-Century-Crofts, New York.

Gilbert, C.D., Sigman, M. and Crist, R.E. (2001) The neural basis of perceptual learning. Neuron, 31: 681–697.

Goodale, M.A. and Milner, A.D. (1992) Separate visual pathways for perception and action. Trends Neurosci, 15: 20–25.

Gordon, R.D. (2004) Attentional allocation during the perception of scenes. J. Exp. Psychol. Hum. Percept. Perform., 30: 760–777.

Grabowska, A. and Nowicka, A. (1996) Visual-spatial-fre- quency model of cerebral asymmetry: a critical survey of behavioral and electrophysiological studies. Psychol. Bull., 120: 434–449.

Grill-Spector, K., Kourtzi, Z. and Kanwisher, N. (2001) The lateral occipital complex and its role in object recognition. Vision Res., 41: 1409–1422.

Grossberg, S. (1980) How does a brain build a cognitive code? Psychol. Rev., 87: 1–51.

Henderson, J.M. (1992) Identifying objects across saccades: effects of extrafoveal preview and flanker object context. J. Exp. Psychol. Learn. Mem. Cogn., 18: 521–530.

Henderson, J.M., Pollatsek, A. and Rayner, K. (1987) Effects of foveal priming and extrafoveal preview on object identification. J. Exp. Psychol. Hum. Percept. Perform., 13: 449–463.

Henson, R.N. (2003) Neuroimaging studies of priming. Prog. Neurobiol., 70: 53–81.

Hock, H.S., Romanski, L., Galie, A. and Williams, C.S. (1978) Real-world schemata and scene recognition in adults and children. Mem. Cogn., 6: 423–431.

Hollingworth, A. and Henderson, J.M. (1998) Does consistent scene context facilitate object perception? J. Exp. Psychol. Gen., 127: 398–415.

Hopfinger, J.B., Buonocore, M.H. and Mangun, G.R. (2000) The neural mechanisms of top-down attentional control. Nat. Neurosci., 3: 284–291.

Humphreys, G.W., Riddoch, M.J. and Price, C.J. (1997) Topdown processes in object identification: evidence from experimental psychology, neuropsychology and functional anatomy. Philos. Trans. R. Soc. Lond. B Biol. Sci., 352: 1275–1282.

Jackson, O. and Schacter, D.L. (2004) Encoding activity in anterior medial temporal lobe supports subsequent associative recognition. Neuroimage, 21: 456–462.

Kirwan, C.B. and Stark, C.E. (2004) Medial temporal lobe activation during encoding and retrieval of novel face-name pairs. Hippocampus, 14: 919–930.

Kosslyn, S.M. (1994) Image and Brain. MIT Press, Cambridge, MA.

Kosslyn, S.M., Alpert, N.M. and Thompson, W.L. (1995) Identifying objects at different levels of hierarchy: a positron emission tomography study. Hum. Brain Mapping, 3: 107–132.

20

Kosslyn, S.M., Alpert, N.M., Thompson, W.L., Chabris, C.F., Rauch, S.L. and Anderson, A.K. (1993) Visual mental imagery activates topographically organized visual cortex: PET investigations. J. Cogn. Neurosci., 5: 263–287.

Koutstaal, W., Wagner, A.D., Rotte, M., Maril, A., Buckner, R.L. and Schacter, D.L. (2001) Perceptual specificity in visual object priming: functional magnetic resonance imaging evidence for a laterality difference in fusiform cortex. Neuropsychologia, 39: 184–199.

Lee, T.S. and Mumford, D. (2003) Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am., 20: 1434–1448.

Leonova, A., Pokorny, J. and Smith, V.C. (2003) Spatial frequency processing in inferred PCand MC-pathways. Vision Res., 43: 2133–2139.

Levy, I., Hasson, U., Avidan, G., Hendler, T. and Malach, R. (2001) Center-periphery organization of human object areas. Nat. Neurosci., 4: 533–539.

Logothetis, N.K., Leopold, D.A. and Sheinberg, D.L. (1996) What is rivalling during binocular rivalry? Nature, 380: 621–624.

Lustig, C. and Buckner, R.L. (2004) Preserved neural correlates of priming in old age and dementia. Neuron, 42: 865–875.

Maccotta, L. and Buckner, R.L. (2004) Evidence for neural effects of repetition that directly correlate with behavioral priming. J. Cogn. Neurosci., 16: 1625–1632.

Maguire, E.A. (2001) The retrosplenial contribution to human navigation: a review of lesion and neuroimaging findings. Scand. J. Psychol., 42: 225–238.

Malach, R., Levy, I. and Hasson, U. (2002) The topography of high-order human object areas. Trends Cogn. Sci., 6: 176–184.

Mandler, J.M. and Johnson, N.S. (1976) Some of the thousand words a picture is worth. J. Exp. Psychol. [Hum. Learn.], 2: 529–540.

Martin, A., Wiggs, C.L., Ungerleider, L.G. and Haxby, J.V. (1996) Neural correlates of category-specific knowledge. Nature, 379: 649–652.

Mechelli, A., Price, C.J., Friston, K.J. and Ishai, A. (2004) Where bottom-up meets top-down: neuronal interactions during perception and imagery. Cereb. Cortex, 14: 1256–1265.

Merigan, W.H. and Maunsell, J.H. (1993) How parallel are the primate visual pathways? Annu. Rev. Neurosci., 16: 369–402.

Metzger, R.L. and Antes, J.R. (1983) The nature of processing early in picture perception. Psychol. Res., 45: 267–274.

Miller, M.B. and Gazzaniga, M.S. (1998) Creating false memories for visual scenes. Neuropsychologia, 36: 513–520.

Miyashita, Y. and Hayashi, T. (2000) Neural representation of visual objects: encoding and top-down activation. Curr. Opin. Neurobiol., 10: 187–194.

Morcom, A.M., Good, C.D., Frackowiak, R.S. and Rugg, M.D. (2003) Age effects on the neural correlates of successful memory encoding. Brain, 126: 213–229.

Oenguer, D. and Price, J.L. (2000) The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex, 10: 206–219.

Oliva, A. and Torralba, A. (2001) Modeling the shape of a scene: a holistic representation of the spatial envelope. Int. J. Comp. Vis., 42: 145–175.

Palmer, S.E. (1975) The effects of contextual scenes on the identification of objects. Mem. Cogn., 3: 519–526.

Parker, A., Wilding, E. and Akerman, C. (1998) The Von Restorff effect in visual object recognition memory in humans and monkeys. The role of frontal/perirhinal interaction. J. Cogn. Neurosci., 10: 691–703.

Pascual-Leone, A. and Walsh, V. (2001) Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science, 292: 510–512.

Pihlajamaki, M., Tanila, H., Kononen, M., Hanninen, T., Hamalainen, A., Soininen, H. and Aronen, H.J. (2004) Visual presentation of novel objects and new spatial arrangements of objects differentially activates the medial temporal lobe subareas in humans. Eur. J. Neurosci., 19: 1939–1949.

Ranganath, C., DeGutis, J. and D’Esposito, M. (2004a) Cat- egory-specific modulation of inferior temporal activity during working memory encoding and maintenance. Brain Res. Cogn. Brain Res., 20: 37–45.

Ranganath, C., Yonelinas, A.P., Cohen, M.X., Dy, C.J., Tom, S.M. and D’Esposito, M. (2004b) Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia, 42: 2–13.

Rempel-Clower, N.L. and Barbas, H. (2000) The laminar pattern of connections between prefrontal and anterior temporal cortices in the rhesus monkey is related to cortical structure and function. Cereb. Cortex, 10: 851–865.

Schacter, D.L. and Buckner, R.L. (1998) Priming and the brain. Neuron, 20: 185–195.

Schyns, P.G. and Oliva, A. (1994) From blobs to boundary edges: evidence for timeand spatial-dependent scene recognition. Psychol. Sci., 5: 195–200.

Shapley, R. (1990) Visual sensitivity and parallel retinocortical channels. Annu. Rev. Psychol., 41: 635–658.

Simons, J.S., Koutstaal, W., Prince, S., Wagner, A.D. and Schacter, D.L. (2003) Neural mechanisms of visual object priming: evidence for perceptual and semantic distinctions in fusiform cortex. Neuroimage, 19: 613–626.

Sternberg, S. (2001) Separate modifiability, mental modules, and the use of pure and composite measures to reveal them. Acta Psychol. (Amst.), 106: 147–246.

Tanaka, K. (1996) Inferotemporal cortex and object vision. Annu. Rev. Neurosci., 19: 109–139.

Ullman, S. (1995) Sequence seeking and counter streams: A computational model for bidirectional information flow in the visual cortex. Cereb. Cortex, 1: 1–11.

Valenstein, E., Bowers, D., Verfaellie, M., Heilman, K.M., Day, A. and Watson, R.T. (1987) Retrosplenial amnesia. Brain, 110: 1631–1646.

Vogt, B.A., Vogt, L.J., Perl, D.P. and Hof, P.R. (2001) Cytology of human caudomedial cingulate, retrosplenial, and caudal parahippocampal cortices. J. Comp. Neurol., 438: 353–376.

Vuilleumier, P., Henson, R.N., Diver, J. and Dolan, R.J. (2002) Multiple levels of visual object constancy revealed by eventrelated fMRI of repetition priming. Nat. Neurosci., 5: 491–499.

Wagner, A.D., Schacter, D.L., Rotte, M., Koutstaal, W., Maril, A., Dale, A.M., Rosen, B.R. and Buckner, R.L. (1998) Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science, 281: 1188–1191.

Wheeler, M.E. and Buckner, R.L. (2003) Functional dissociation among components of remembering: control, perceived oldness, and content. J. Neurosci., 23: 3869–3880.

21

Wilson, F.A.W., Scalaidhe, S.P.O. and Goldman-Rakic, P.S. (1993) Dissociation of object and spatial processing domains in primate prefrontal cortex. Science, 260: 1955–1958.

Zago, L., Fenske, M.J., Aminoff, E. and Bar, M. (2005) The rise and fall of priming: how visual exposure shapes cortical representations of objects. Cereb. Cortex, 15: 1655–1665.

Introduction

One remarkable aspect of human visual perception is that we are able to understand the meaning of a complex novel scene very quickly even when the image is blurred (Schyns and Oliva, 1994), or presented for only 20 ms (Thorpe et al., 1996). Mary Potter (1975, 1976, see also Potter et al., 2004) demonstrated that during a rapid presentation of a stream of images, observers were able to identify the semantic category of each image as well as a few objects and their attributes. This rapid understanding phenomenon can be experienced while looking at modern movie trailers which utilize many fast cuts between scenes: with a mere

Corresponding author. Tel.: +1-617-452-2492; Fax: +1-617-258-8654; E-mail: oliva@mit.edu

glimpse of each picture, you can identify each shot’s meaning, the actors and the emotion depicted in each scene (Maljkovic and Martini, 2005) even though you will not necessarily remember the details of the trailer. The amount of perceptual and semantic information that observers comprehend within a glance (about 200 ms) refers to the gist of the scene (for a review, Oliva, 2005). In this paper, we discuss two main questions related to rapid visual scene understanding: what visual information is perceived during the course of a glance, and which mechanisms could account for the efficiency of scene gist recognition.

Research in scene understanding has traditionally treated objects as the atoms of recognition. However, behavioral experiments on fast scene perception suggest an alternative view: that we do not need to perceive the objects in a scene to

24

identify its semantic category. The semantic category of most real-world scenes can be inferred from their spatial layout (e.g., an arrangement of basic geometrical forms such as simple Geons clusters, Biederman, 1995; the spatial relationships between regions or blobs of particular size and aspect ratio, Schyns and Oliva, 1994; Sanocki and Epstein, 1997; Oliva and Schyns, 2000). Fig. 1 illustrates the importance of the spatial arrangement of regions for scene and object recognition. When looking at the image on the left, viewers describe the scene as a street with cars, buildings and the sky. Despite the fact that the local information available in the image is insufficient for reliable object recognition, viewers are confident and highly consistent in their descriptions. Indeed, the blurred scene has the spatial layout of a street. When the image is shown in high resolution, new details reveal that the image has been manipulated and that the buildings are in fact pieces of furniture. Almost 30% of the image pixels correspond to an indoor scene. The misinterpretation of the low-resolution image is not a defect of the visual system. Instead, it illustrates the strength of spatial layout information in constraining the identity of the objects in normal conditions, which is especially evident in degraded conditions in which object identities cannot be inferred based only on local information (Schyns and Oliva, 1994).

In this paper, we examine what is the initial representation of a complex, real-world scene image

that allows for its rapid recognition. According to the global precedent hypothesis advocated by Navon (1977) and validated in numerous studies since (for a review see Kimchi, 1992), the processing of the global structure and the spatial relationships between components precede the analysis of local details. The global precedence effect is particularly strong for images constituted of many element patterns (Kimchi, 1998), as it is the case of most realworld scene pictures.

To clarify the terminology we will be using in this article, in the same way that ‘‘red’’ and ‘‘vertical’’ are local feature values of an object (Treisman and Gelade, 1980), a specific configuration of local features defines a global feature value of a scene or an object. For instance, an image composed of vertical contours on the right side and horizontal contours on the left side could be estimated by one global feature-receptive field tuned to respond to that specific ‘‘Horizontal–Vertical’’ configuration. Global feature inputs are estimated by summations of local feature values but they encode holistic properties of the scene as they convey the spatial relationships between components. On the basis of behavioral and computational experiments, we show the relevance of using a low-dimensional code of the spatial layout of a scene, termed global image features, to represent the meaning of a scene. Global features capture the diagnostic structure of the image, giving an impoverished and coarse version of the principal

contours and textures of the image that is still detailed enough to recognize the image’s gist. One of the principal advantages of the global image coding described here lies in its computational efficiency: there is no need to parse the image or group its components in order to represent the spatial configuration of the scene.

In this paper, we examine (1) the possible content of the global structure of a natural scene image, based on experimental results from the scene recognition literature; (2) how the global scene structure can be modeled and (3) how the global features could participate in real-world scene categorization.

The role of global image features on scene perception: experimental evidence

There is considerable evidence that visual input is processed at different spatial scales (from low to high spatial frequency), and psychophysical and computational studies have shown that different spatial scales offer different qualities of information for recognition purpose. On the one hand, the shape of an object is more precisely defined at high spatial frequencies but the object boundaries are interleaved by considerable noise, which requires

25

extensive processing to be filtered out (among others, Marr and Hildreth, 1980; Shashua and Ullman, 1988). On the other hand, low-scale resolution is more contrasted and might be privileged in terms of temporal processing than finer scale (Navon, 1977; Sugase et al., 1999), but this perceptual advantage might be offset by higher uncertainty about the identity of the blobs.

In a series of behavioral experiments, Oliva and Schyns evaluated the role that different spatial frequencies play in fast scene recognition. They created a novel kind of stimuli, termed hybrid images (see Fig. 2), by superimposing two images at two different spatial scales: the low spatial scale is obtained by filtering one image with a low-pass filter (keeping spatial frequencies up to 8 cycles/ image), the high spatial scale is obtained by filtering a second image with a high-pass filter (frequencies above 24 cycles/image). The final hybrid image is composed by adding these two different filtered images (the filters are designed in such a way that there is no overlapping between the two images in the frequency domain). The examples in Fig. 2 show hybrid images combining a beach scene and a street scene.

The experimental results using hybrid stimuli showed that for short presentation time (30 ms,

26

followed by a mask, Schyns and Oliva, 1994), observers used the low spatial frequency part of hybrids (street in Fig. 2B) when solving a scene recognition task, whereas for longer (150 ms) durations of the same image, observers categorized the image on the basis of the high spatial frequencies (e.g., beach in Fig. 2B). In both cases, participants were unaware that the stimuli had two interpretations. It is important to stress that this result is not a evidence for a preference of the low spatial frequencies in the early stages of visual processing: additional experiments (Oliva and Schyns, 1997; Schyns and Oliva, 1999) showed that, in fact, the visual system can select which spatial scale to process depending on task constraints (e.g., if the task is determining the type of emotion of a face, participants will preferentially select the low spatial frequencies, but when the task is determining the gender of the same set of faces, participants used either low or high spatial frequencies). Furthermore, priming studies showed that within a 30-ms exposure, both low and high spatial frequency bands from a hybrid image were registered by the visual system 1 (Parker et al., 1992, 1996; Oliva and Schyns, 1997, Exp.1), but that the requirements of the task determined which scale, coarse or fine, was preferentially selected for covert processing. This suggests that the full range of spatial frequency scales is available with only 30 ms of image exposure, although the resolution at which the local features are analyzed and preattentively combined, when embedded in cluttered natural images, is unknown.

However, hybrid images break one important statistical property of real-world natural images, i.e., the spatial scale contiguity. To the contrary of hybrid images, contours of a natural image are correlated across scale space: a contour existing at low spatial frequency exists also at high spatial frequency. Moreover, statistical analysis of the distributions of orientations in natural images has shown that adjacent contours tend to have similar orientations whereas segments of the same contour that are further apart tend to have more disparate

1A hybrid scene presented for 30 ms and then masked would prime the recognition of a subsequent related scene, matching either the lowor the high-spatial scale of the hybrid (Oliva and Schyns, 1997, Exp. 1).

orientations (Geisler et al., 2001). The visual system could take advantage of spatial and spectral contiguities of contours to rapidly construct a sketch of the image structure. Boundary edges that would persist across the scale space are likely to be important structures of the image (Linderberg, 1993), and would define an initial skeleton of the image, fleshed out later by finer structures existing at higher spatial frequency scales (Watt, 1987; Linderberg, 1993; Yu, 2005). Most of the contours in natural scenes need selective attention to be bound together to form a shape of higher complexity (Treisman and Gelade, 1980; Wolfe and Bennet, 1997; Wolfe et al., 2002), but contours persistent through the scale space might need fewer attentional (or computational) resources to be represented early on. Therefore, one cannot dismiss the possibility that the analysis of fine contours and texture characteristics could be performed at the very early stage of scene perception, either because low spatial frequency luminance boundaries bootstrap the perceptual organization of finer contours (Lindeberg, 1993), or because the sparse detection of a few contours is sufficient to predict the orientation of the neighborhood edges (Geisler et al., 2001), or because selective attention was attending to information at a finer scale (Oliva and Schyns, 1997).

Within this framework, the analysis of visual information for fast scene understanding proceeds in a global to local manner (Navon, 1977; Treisman and Gelade, 1980), but not necessarily from low to high spatial frequencies. In other words, when we say ‘‘global and local’’ we do not mean ‘‘low and high’’ spatial frequencies. All spatial frequencies contribute to an early global analysis of the scene layout information, but organized at a rather coarse layout. Fine image edges, like long contours, are available, but their spatial organization is not encoded in a precise way. In the rest of this section we discuss some of the possible mechanisms used for performing the global analysis of the scene.

A simple and reliable global image feature for scene recognition is obtained by encoding the organization of color blobs in the image (under this representation a view of a landscape corresponds to a blue blob on the top, a green blob on the bottom and a brownish blob in the center, e.g., Lipson et al.,