11.4 Engineering

ing only those loose and abstract perceptual category representations we were hunting throughout the book (figure 2, left side; figure 61).

Even if one had seemingly much understood of the scene with these two glances, there are still big Junks of it, that are not actually recognized. We think that this is the reason why phenomena like change blindness occur (Rensink, 2000; O’Regan et al., 1999): there is just way to much detail in a scene that could be comprehended, even after a long visual search (chapter 1). This point has already been made by others in one or the other way, for example (O’Regan, 1992; Koch, 2004). But what we specifically would like to emphasize is that the scene is understood with perceptual category representations that are loose and abstract, and that this is the major cause for those observed phenomena.

In summary, scene exploration is not so much about the systematic, stepwise recovery of its content, but is a process consisting primarily of making rapid associations based on a small fraction of the structure perceived in a scene. The first rapid association triggers a frame, the gist perception, which in turn is the guidance for the subsequent eye fixation causing another rapid association, and so on.

11.4Engineering

How would one then approach the construction of a scene recognition system? As we have tried to argue in the previous sections, recognizing a scene is like recognizing a basic-level object. Ergo, one has to seek the neural substrate that is able to deal with structural variability. In other words, it boils down to the construction of the basic-level categorization process as we have pursued in the chapters presented in this book. Once such a basic-level categorization process has been constructed for the line-drawings in chapter 5 for example, then one may test it on scenes like in figure 60. Although the ultimate goal is certainly to have a system recognizing objects in gray-scale images, tackling the structural variability in real-world scenes at once may be just too overwhelming. It may therefore be sensible to develop a system that is able to deal with a reduced variability as we did in chapter 5. The room scenes in figure 60 are literally identical with regard to scene content but contain structural variability that can still not be dealt with in an elegant manner by any scene recognition system. Many representational issues can be explored with such simple scenes, without dealing with the full-blown variability existent in real-world scenes. We think that the translation of a line-drawing analyzing system to a real-world (gray-scale) analyzing system could then be systematically carried out.

One may not intend to wait until the categorization machinery has been thoroughly constructed and one may already plan to develop a search system that tries to find and identify simple shapes in scenes.

Figure 60: Two similar room scenes (a and b). There is still no elegant network that can deal with that type of variability.

Since we have a shape-recognition system that is capable of identifying simple shapes (the CPFM system, chapter 10), one may test this system on real scenes and gain possibly valuable experiences. For instance, one could design a saliency process aiming at regions using a crude form of the SAT that locates the center of shapes. The system would determine simple structures by saccading to those center points and by sequentially applying the CPFM system to identify the shapes.

11.5Recapitulation

Scenes have a very regular structure and the representations one has about these scenes incorporate this regularity in some way. It therefore makes sense to treat scenes similar to basic-level objects: there is a common structure amongst scenes of the same category and that common structure - including its characteristic region of course - is stored in some way; and when it is perceived, it triggers the sensation of gist perception. Exploring a scene by saccades is making associations between perceived fragments of the scene; scene exploration does not require detailed reconstruction. We have suggested that one could begin building a crude scene-exploration system using a process to find salient regions, like a modified SAT, and employing the CPFM system to determine the shape of the regions.