Table of Contents

    Topic review

    Perceptual Organization

    View times: 5751


    Gestalt theory has provided perceptual science with a conceptual framework relating to brain mechanisms that determine the way we see the visual world. This is referred to as "Perceptual Organization" and has inspired researchers in Psychology, Neuroscience and Computational Design ever since. The major Gestalt principles, such as the principle of Prägnanz, and more importantly the Gestalt laws of perceptual organization, have been critically important to our understanding of visual information processing, how the brain detects order in what we see, and derives likely perceptual representations from statistically significant structural regularities. The perceptual integration of contrast information across co-linear space for the organization of objects in the 2D image plane into figure and ground convey the most elementary basis to our understanding of the visual world. Gestalt theory continues to generate powerful concepts and insights for perceptual science even today, where it is to be placed in the context of image-base decision making by human minds and machines.

    1. Introduction

    The seminal work by the Gestalt School led to the formulation of Gestalt principles, such as the principle of Prägnanz, and Gestalt laws of perceptual organization that have been critically important to our understanding of visual information processing in the human brain. The perceptual integration of contrast information across co-linear space, and the organization of objects in the 2D image plane into figure and ground, convey the most elementary basis to our understanding of the visual world. Gestalt theory continues to generate powerful concepts and insights for perceptual science even today, when placed within the context of image-base decision making by human minds and machines. The laws and principles which predict how perceptual qualities can be extracted from the most elementary visual signals were discovered by the Gestalt psychologists [1][2][3]. Their seminal work has inspired visual science ever since, and has led to exciting discoveries which have confirmed the Gestalt idea that the human brain would have an astonishing capacity for selecting and combining critical visual signals to generate organized representations for decision making and action. This capacity of selection and integration drives what Gestalt theory refers to as perceptual organization [1][2][3], i.e. the correct estimation of relative positions, trajectories, and distances of objects represented in planar images, the perceptual construction of representations of form from motion, and most of our perception of three-dimensional object properties. The Gestalt laws [1] of perceptual organization, of which the law of good continuation is a particular example, describe the necessary stimulus or input conditions under the premise of a so-called principle of pregnancy (Prägnanzprinzip), which postulates that objects in the visual field will produce the simplest and most complete solution possible for perceptual organization. This entails that the simplest input configurations of contrast and orientation in the visual field will enable human observers to assess which parts of the input is likely to belong together to form a unified visual object or shape, or which parts are likely to be nearer and which further away from the observer if these represented objects (i.e. their image representations) were seen in the real world.

    2.  Gestalt Laws' Design

    The problem relative to how planar image contents will be grouped into perceptual representations of figure and ground [4][5][6][7][8] is one of the study grounds the Gestalt laws have been designed for. Figure-ground representation is a perceptual solution that enables the observer to assess which objects in the image would be likely to be nearer and which objects would be likely to be further away in a real world configuration. Figure-ground solutions are mediated by specific image cues to shape and to relative distance, involving local signals of contrast [9][10][11][12][13][14][15][16][17][18], colour [7][8][19][20][21], and orientation [22][23][24][25][26][27][28][29][30] to fill in specific regions of an image. This enables the perception of surfaces on the basis of a subtle interplay of visual mechanisms driving perceptual grouping and perceptual segregation. The associated perceptual sensations of local contrast enhancement make visual objects in the image appear to stand in front of other objects represented in the same plane. Such sensations are often deemed "illusory" because they have no physical origin, i.e. there is no objective difference in local luminance that would explain their perception. An essential aspect of this process of figure-ground segregation is the perceptual assignment of border ownership. The Gestalt theorist Rubin [3] was among the first to point out that a figure has distinct perceptual qualities that make it stand out against the rest of the visual field, which thereby acquires the perceptual quality of ground (or background). A figure occludes the ground and, therefore, owns the borders which separate it from the latter. Neuroscience experiments have identified neurons predominantly in V2 (but also V1) of the monkey that respond selectively to the location of  borders in the visual field, i.e. to whether the border is seen as belonging to ‘figure’ or to ‘ground’ (border ownership) [31][32]. Selective visual attention to borders assigned to ‘figure’ strengthens the selective neuronal responses [33].

    The Gestalt psychologists correctly presumed that, to recover a representation of holistic figure-ground representations from image parts, the brain must achieve perceptual integration of visual information across co-linear space [1][2]. Contrast integration across co-linear features and borders plays a crucial role in perceptual organization under conditions of stimulus uncertainty and configurative ambiguity [18][26]. Governed by the Gestalt law of good continuation and reflected by psychophysically measurable interactive effects between co-axial stimuli in the visual field [25][26][27][28], co-linear spatial integration results from specific response activities of visual cortical neurons and neuronal interactions “beyond the classic receptive field” [34][35][36][37][38][39][40][41][42]. These reveal the complexity of  hardwired, orientation selective brain mechanisms [43][44] underlying perceptual organization. Co-linear integration is crucial for the detection of alignment, virtual trajectories, and shape borders in a world where most objects are seen incompletely. It enables the human observer to assess the continuity of image fragments under conditions of diminished visibility and heightened stimulus ambiguity.

    However, in complex images, some visible stimulus fragments appear clearly aligned, others do not. Specific phenomenal conditions need to be satisfied to enable collinear interpolation in static 2D scenes, and the process of interpolation constrains the spreading of surfaces across unspecified regions in the image [45][46]. Also, not all of perceptual organization is hardwired [47][48], and the contribution of past experience and perceptual learning to early mechanisms of perceptual integration needs to be taken into account given that specific memory data about objects and their most likely spatial configuration may facilitate or interfere with, depending on conditions, ongoing visual processing of ambiguous image data. The question relative to recovery of ‘veridical’ object properties from image data may be seen as a byproduct of perceptual organization, but was not an issue raised by Gestalt theory. However, the laws of perceptual organization have generated a conceptual framework for addressing this problem space in a bottom-up approach. The ‘veridicality’ of perceptual representations our brain may derive from visual stimuli raises deeper philosophical questions. Perceptual organization as such may be seen as resulting from pressure on the perceptual system to actively construct ordered representations of objects in depth in response to intrinsically ambiguous visual data. The biological role of perceptual organization, placed in the context of evolution, would be to ensure behavioral adaptation and success. Perceptual organization becomes mandatory as soon as the visual signals contained in an image or a scene reach our retina [49]. The major part of the necessary information processing then takes place in the visual brain, involving a whole chain of mechanisms well beyond the retina. Perceptual neuroscience has provided us with a diversified account for the many ways in which visual sensitivity to ordered structure and regularities expresses itself in behavior on the basis of cortical mechanisms. Multiple stages of neural processing transform fragmented signals into visual key representations of 3D scenes that can be used to control effective behavior. Since our survival depends on our ability to pick up order in the physical world, and since we conceive the physical world as an ordered one, our brain must  be sensitive to structural regularities in the physical world. Neural interactions “beyond the classic receptive field” drive the visual processing of texture dissimilarities, boundary completion, surface filling-in, and figure-ground segregation in the brain genesis of “perceptual order”.

    3.  Structural Regularities in Images in Perceptual Organization

    Statistically likely structural regularities in images are important to perceptual organization. Such regularities are required for visual textures to be discriminated from random noise by human observers. Ideal Observer and Order Observer models based on the capture of statistical regularities in textures were shown to closely reflect the sensitivity of the human visual system to structural regularities in images [50]. Symmetry [51][52] may facilitate the perceptual capture of such regularities. Spatiotemporal boundary formation is critical to the perception of shape boundaries in images and their global motion, as shown in experiments on boundary integration from spatially and temporally sparse transformations of texture [53]. The human perceptual system uses positions (x, y, z) and times (t) of element (n) transformations to extract local structural regularities such as edge fragments, which then connect by interpolation processes to produce larger shapes. This process seems to have a spatiotemporal integration constraint of ~80 ms between elements. Structural ambiguity and uncertainty pose a specific problem here. Perceptual organization not only accounts for the ways in which physically defined order is processed by the visual system, but also how such order may be inferred in cases of uncertainty, and how this may give rise to the perception of “illusory” or “phantom” shapes, as shown recently in a new phenomenon [54] in which illusory streaks irradiate from the vertexes of angular contours, connecting pairs of figures, and forming phantom net shapes that are only clearly visible in a specific figural context. The real world conditions or configurative order to which perceptual organization is sensitive also determine our aesthetic perception, as shown in work on aesthetic preferences [51] for structural regularities. A novel variant of perceptual organization, called ‘accentuation for perceptual coupling’ [55] describes new conditions for grouping that cannot be explained in terms of the classic Gestalt laws (similarity, co-linearity, or other). This recent work reveals the ‘supremacy of dissimilarity’ as an important figural complement to similarity in the genesis of aesthetic structure, involving novel dynamics of perceptual coupling.

    The visual brain organizes incoming information early on and interprets a 2D image in terms of the real world, producing configurative order that provides perceptual continuity and enables object-based attention. The Form-And-Color-And-DEpth (FACADE) model and its laminar cortical embodiment and extension by the 3D LAMINART models [56] have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out figure-ground organization to generate 3D percepts from 2D pictures of occluding and occluded objects. The models provide a unified interpretation of groundbreaking neuroscience data [57] about cortical cells and their response properties for perceptual organization, with modulation by attention and temporal input properties. The segregation of image parts into foreground and background requires border ownership assignment by the visual system and recordings from monkey visual cortex show that many neurons, especially in area V2, are selective to border ownership. These neurons are edge selective and have classical receptive fields (CRF). Their responses are modulated (enhanced or suppressed) depending on the location of a “figure” relative to the edge in their receptive field. Each neuron has a fixed preference for location on one side or the other. This selectivity is derived from image context far beyond the CRF [31][32][42]. Recent work in this field suggests that border ownership selectivity reflects the formation of early prototypes of object representations, or “proto-objects” [57]. The evidence includes experiments showing (1) reversal of border ownership signals with change of perceived object structure, (2) border ownership specific enhancement of responses in object-based selective attention, (3) persistence of border ownership signals in accordance with continuity of object perception, and (4) remapping of border ownership signals across saccades and object movements. Grouping, persistence, and remapping are shown to be manifest in low-level visual mechanisms, which challenges any top-down-only model of perceptual organization. Perceptual judgments relative to figure and ground reveal that polarity-invariant as well as polarity-specific object properties are exploited in perceptual organization [18][19][58]. In some conditions, figure-ground is solely determined by the orientation of contrasts, not by their relative sign. Psychophysical data from human observers [18] responding to structurally ambiguous, fragmented 2D objects with varying local contrast sign across their edges reveal this type of polarity-invariant perceptual organization, which was already predicted by the earliest version of the FACADE theory [4].

    In computational science, these processes of perceptual organization carry important meaning for the design and control of visual interfaces and the generation of reliable structural representations for image-based decision making [59][60]. In this context, understanding which among many possible image solutions will produce geometric configurations for optimal perceptual organization is akin to understanding which sentences in complex text will best convey the desired message or meaning. As a consequence, Gestalt theory is as relevant as ever in the context of computational vision and visual interface technology. Image-guided surgery, for example, exploits images taken before and/or during the procedure to help a surgeon navigate, and to augment his/her capacity for optimal decision making and action during the procedure [61]. With the help of augmented reality, visual guidance can be provided directly on a surgeon's view of the patient by mixing real and virtual images [60][61]. Perceptual qualities such as color [62], brightness, and relative feature salience in computationally preprocessed images with a large amount of random variations [63] are essential for making specific regions of interest to a specialist optimally perceptible.


    1. Metzger, W. (1930) Gesetze des Sehens, English trans. L. Spillmann (2009) Laws of Seeing  Cambridge, MA: MIT Press.
    2. Wertheimer, M. (1923) Perceived Motion and Figural Organization, English trans. L. Spillmann, M. Wertheimer, K. W. Watkins, S. Lehar and V. Sarris (2012) Cambridge, MA: MIT Press.
    3. Rubin, E. (1921). Visuell Wahrgenommene Figuren: Studien in psychologischer Analyse.  Kopenhagen: Gyldendalske.
    4. Grossberg, S. (1997). Cortical dynamics of 3-D figure-ground perception of 2-D pictures. Psychological Review, 104, 618-658.
    5. Dresp, B., Durand, S. and Grossberg, S. (2002). Depth perception from pairs of overlapping cues in pictorial displays. Spatial Vision, 15, 255–276.
    6. Craft, E., Schuetze, H., Niebur, E., and von der Heydt, R. (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97, 4310-4326.
    7. Dresp-Langley, B. and Reeves, A. (2012). Simultaneous contrast and apparent depth from true colors on grey: Chevreul revisited.  Seeing & Perceiving, 25(6), 597–618.
    8. Dresp-Langley, B. and Reeves, A. (2014). Effects of saturation and contrast polarity on the figure-ground organization of color on gray. Frontiers in Psychology. doi: 10.3389/fpsyg.2014.01136
    9. Heinemann, E. G. (1955). Simultaneous brightness induction as a function of inducing and test-field luminance. Journal of Experimental Psychology, 50, 89–96.
    10. Hamada, J. (1985). Asymmetric lightness cancellation in Craik-O’Brien patterns of negative and positive contrast. Biological Cybernetics, 52, 117–122.
    11. O’Shea, R. P., Blackburn, S. G. and Ono, H. (1994). Contrast as a depth cue. Vision Research, 34, 1595–1604.
    12. De Weert, C. M. M. and Spillmann, L. (1995) Assimilation: Asymmetry between brightness and darkness. Vision Research, 35, 1413–1419.
    13. Yu, C. and Levi, D. M. (1997). Spatial facilitation predicted with end-stopped spatial filters. Vision Research, 37, 3117–3128.
    14. Yu, C. and Levi, D. M. (2000). Surround modulation in human vision unmasked by masking experiments. Nature Neuroscience, 3, 724–728.
    15. Chen, C.C., Kasamatsu, T., Polat, U., and Norcia, A. M. (2001). Contrast response characteristics of long-range lateral interactions in cat striate cortex. Neuroreport, 12, 655-661.
    16. Chen, C.C., and Tyler, C. W. (2001). Lateral sensitivity modulation explains the flanker effect in contrast discrimination. The Proceedings of the Royal Society (London) Series B, 268, 509-516.
    17. Chen, C.C., and Tyler, C.W. (2008). Excitatory and inhibitory interaction fields of flankers revealed by contrast-masking functions. Journal of Vision, 8,10, 1–14.
    18. Dresp-Langley, B., and Grossberg, S. (2016) Neural Computation of Surface Border Ownership and Relative Surface Depth from Ambiguous Contrast Inputs. Front. Psychol. 7:1102. doi: 10.3389/fpsyg.2016.01102
    19. Dresp, B. and Fischer, S. (2001). Asymmetrical contrast effects induced by luminance and color configurations. Perception & Psychophysics, 63, 1262–1270.
    20. Guibal, C. R. C. and Dresp, B. (2004). Interaction of color and geometric cues in depth perception: When does red mean near, Psychological Research, 10, 167–178.
    21. Dresp-Langley, B., Reeves, A. (2018) Colour for Behavioural Success. I-PERCEPTION, 9 (2), 204166951876717.
    22. Dresp, B., & Bonnet, C. (1991). Psychophysical evidence for low-level processing of illusory contours. Vision Research, 10, 1813–1817.
    23. Kellman, P.J. and Shipley, T.F. (1991). A theory of visual interpolation in object perception. Cognitive Psychology, 23, 141-221.
    24. Field, D.J., Hayes, A., and Hess, R.F. (1993). Contour integration by the human visual system: Evidence for a local
    25. Dresp, B. (1993). Bright lines and edges facilitate the detection of small light targets. Spatial Vision, 7, 213-225.
    26. Dresp, B. (1997). On ‘illusory’ contours and their functional significance. Current  Psychology of Cognition, 16, 489–517.
    27. Tzvetanov, T. and Dresp, B. (2002). Short- and long-range effects in line contrast detection. Vision Research, 42, 2493-2498.
    28. Dresp, B. and Langley, O.K. (2005). Long-range spatial integration across contrast signs: A probabilistic mechanism? Vision Research, 45, 275–284.
    29. Devinck, F., Spillmann, L., and Werner, J. S. (2006). Spatial profile of contours inducing long-range color assimilation. Visual Neuroscience, 23, 573–577.
    30. Huang, P.C., Chen, C.C., and Tyler, C.W. (2012). Collinear facilitation over space and depth. Journal of Vision, 12, 20, 1-9.
    31. Zhang, N.R. and von der Heydt, R. (2010). Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex. Journal of Neuroscience, 30, 6482-6496.
    32. Zhou, H., Friedman, H. S., and von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594–6611.
    33. Qiu, F.T., Sugihar, T., and von der Heydt, R. (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience, 10, 1492-1499.
    34. von der Heydt, R., Peterhans, E., and Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260–1262.
    35. von der Heydt, R. and Peterhans, E. (1989). Mechanisms of contour perception in monkey visual cortex: I. Lines of pattern discontinuity. Journal of Neuroscience 9, 1731–1748.
    36. Peterhans, E. and von der Heydt, R. (1991). Subjective contours-bridging the gap between psychophysics and physiology. Trends in Neurosciences, 14, 112–119.
    37. Kapadia, M.K., Ito, M., Gilbert, C.D., and Westheimer, G. (1995). Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron, 15, 843–856.
    38. Kapadia, M.K., Westheimer, G., and Gilbert, C.D. (2000). Spatial contribution of contextual interactions in primary visual cortex and in visual perception. Journal of Neurophysiology, 84, 2048-2062.
    39. Polat, U. and Sagi, D. (1994). The architecture of perceptual spatial interaction. Vision Research, 34, 73–78.
    40. Polat, U. and Sagi, D. (1993). Lateral interactions between spatial channels: suppression and facilitation revealed by lateral masking experiments. Vision Research, 33, 993-999.
    41. Polat, U. and Norcia, A. M. (1996). Neurophysiological evidence for contrast dependent long-range facilitation and suppression in human visual cortex. Vision Research, 36, 2099-2109.
    42. Spillmann, L, Dresp-Langley, B, & Tseng, CH (2015) Beyond the classic receptive field: The effect of contextual stimuli. Journal of Vision, 15, 7.
    43. Hubel , D.H. and Wiesel, T.N. (1959). Receptive fields of single neurons in the cat’s striate cortex. The Journal of Physiology, 148, 574-591.
    44. Hubel, D. H. and Wiesel, T. N. (1968).Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195, 215-243.
    45. Shipley, T.F. & Kellman, P.J. (Eds.). (2001). From Fragments to Objects:  Segmentation and Grouping in Vision. Amsterdam:  Elsevier Science Press.
    46. Pinna, B. and Reeves, A. (2006). Lighting, backlighting, and the laws of figurality in the watercolor illusion. Spatial Vision, 19, 341–373.
    47. Dresp, B. (1999). The cognitive impenetrability hypothesis: Doomsday for the unity of the cognitive neurosciences? Behavioral and Brain Sciences, 22(3), 375-376.
    48. Kimchi, R. and Hadad, B. S. (2002). Influence of past experience on perceptual grouping. Psychological Science, 13, 41-47.
    49. Dresp-Langley, B., Grossberg S., and Reeves; A (2017) Editorial: Perceptual Grouping—The State of The Art. Front. Psychol. 8:67. doi: 10.3389/fpsyg.2017.00067
    50. Katkov, M., Harris, H., and Sagi, D. (2015) Visual perception of order-disorder transition. Front. Psychol. 6:734. doi: 10.3389/fpsyg.2015.00734
    51. Dresp-Langley, B. (2016) Affine Geometry, Visual Sensation, and Preference for Symmetry of Things in a Thing. Symmetry, 8, 127.
    52. Dresp-Langley, B. (2019) Bilateral Symmetry Strengthens the Perceptual Salience of Figure against Ground. Symmetry, 11, 225.
    53. Erlikhman, G., and Kellman, P.J. (2016) From Flashes to Edges to Objects: Recovery of Local Edge Fragments Initiates Spatiotemporal Boundary Formation. Front. Psychol. 7:910. doi: 10.3389/fpsyg.2016.00910
    54. Roncato, S., Guidi, S., Parlangeli, O., and Battaglini, L. (2016) Illusory Streaks from Corners and Their Perceptual Integration. Front. Psychol. 7:959. doi: 10.3389/fpsyg.2016.00959
    55. Pinna, B., Porcheddu, D., and Deiana, K. (2016) From Grouping to Coupling: A New Perceptual Organization in Vision, Psychology, and Biology. Front. Psychol. 7:1051. doi: 10.3389/fpsyg.2016.01051
    56. Grossberg, S. (2016) Cortical Dynamics of Figure-Ground Separation in Response to 2D Pictures and 3D Scenes: How V2 Combines Border Ownership, Stereoscopic Cues, and Gestalt Grouping Rules. Front. Psychol. 6:2054. doi: 10.3389/fpsyg.2015.02054
    57. von der Heydt, R. (2015) Figure–ground organization and the emergence of proto-objects in the visual cortex. Front. Psychol. 6:1695. doi: 10.3389/fpsyg.2015.01695
    58. Dresp, B., Salvano-Pardieu, V., & Bonnet, C. (1996). Illusory form from inducers with opposite contrast polarity: Evidence for multi-stage integration. Perception & Psychophysics, 58, 111-124.
    59. Batmaz, A.U.; de Mathelin, M.; Dresp-Langley, B. Seeing virtual while acting real: Visual display and strategy effects on the time and precision of eye-hand coordination. PLoS ONE 2017, 12.
    60. Dresp-Langley, B. (2015) Principles of perceptual grouping: implications for image-guided surgery. Front. Psychol. 6:1565. doi: 10.3389/fpsyg.2015.01565
    61. Perrin, D. P., Vasilyev, N. V., Novotny, P., Stoll, J., Howe, R. D., Dupont, P. E., Salog, I. S., and del Nido, P. J. (2009) Image Guided Surgical Interventions. Current Problems in Surgery, 46,730-766.
    62. Silvestri, C.; Motro, R.; Maurin, B.; Dresp-Langley, B. (2010) Visual spatial learning of complex object morphologies through the interaction with virtual and real-world data. Design Studies, 31, 363–381.
    63. Wandeto, J. M., & Dresp-Langley, B. (2019) Ultrafast automatic classification of SEM imagesets showing CD4 + cells with varying extent of HIV virion infection. In 7ièmes Journées de la Fédération de Médecine translationnelle de l’Université de Strasbourg, May 25-26. Strasbourg, France.