1. Introduction

The seminal work by the Gestalt School led to the formulation of Gestalt principles, such as the principle of Prägnanz, and Gestalt laws of perceptual organization that have been critically important to our understanding of visual information processing in the human brain. The perceptual integration of contrast information across co-linear space, and the organization of objects in the 2D image plane into figure and ground, convey the most elementary basis to our understanding of the visual world. Gestalt theory continues to generate powerful concepts and insights for perceptual science even today, when placed within the context of image-base decision making by human minds and machines. The laws and principles which predict how perceptual qualities can be extracted from the most elementary visual signals were discovered by the Gestalt psychologists ^[1][2][3]. Their seminal work has inspired visual science ever since, and has led to exciting discoveries which have confirmed the Gestalt idea that the human brain would have an astonishing capacity for selecting and combining critical visual signals to generate organized representations for decision making and action. This capacity of selection and integration drives what Gestalt theory refers to as perceptual organization ^[1][2][3], i.e. the correct estimation of relative positions, trajectories, and distances of objects represented in planar images, the perceptual construction of representations of form from motion, and most of our perception of three-dimensional object properties. The Gestalt laws ^[1] of perceptual organization, of which the law of good continuation is a particular example, describe the necessary stimulus or input conditions under the premise of a so-called principle of pregnancy (Prägnanzprinzip), which postulates that objects in the visual field will produce the simplest and most complete solution possible for perceptual organization. This entails that the simplest input configurations of contrast and orientation in the visual field will enable human observers to assess which parts of the input is likely to belong together to form a unified visual object or shape, or which parts are likely to be nearer and which further away from the observer if these represented objects (i.e. their image representations) were seen in the real world.

2.  Gestalt Laws' Design

The problem relative to how planar image contents will be grouped into perceptual representations of figure and ground ^{[4][5][6][7][8]} is one of the study grounds the Gestalt laws have been designed for. Figure-ground representation is a perceptual solution that enables the observer to assess which objects in the image would be likely to be nearer and which objects would be likely to be further away in a real world configuration. Figure-ground solutions are mediated by specific image cues to shape and to relative distance, involving local signals of contrast ^{[9][10][11][12][13][14][15][16][17][18]}, colour ^{[7][8][19][20][21]}, and orientation ^{[22][23][24][25][26][27][28][29][30]} to fill in specific regions of an image. This enables the perception of surfaces on the basis of a subtle interplay of visual mechanisms driving perceptual grouping and perceptual segregation. The associated perceptual sensations of local contrast enhancement make visual objects in the image appear to stand in front of other objects represented in the same plane. Such sensations are often deemed "illusory" because they have no physical origin, i.e. there is no objective difference in local luminance that would explain their perception. An essential aspect of this process of figure-ground segregation is the perceptual assignment of border ownership. The Gestalt theorist Rubin ^[3] was among the first to point out that a figure has distinct perceptual qualities that make it stand out against the rest of the visual field, which thereby acquires the perceptual quality of ground (or background). A figure occludes the ground and, therefore, owns the borders which separate it from the latter. Neuroscience experiments have identified neurons predominantly in V2 (but also V1) of the monkey that respond selectively to the location of  borders in the visual field, i.e. to whether the border is seen as belonging to ‘figure’ or to ‘ground’ (border ownership) ^[31][32]. Selective visual attention to borders assigned to ‘figure’ strengthens the selective neuronal responses ^[33].

The Gestalt psychologists correctly presumed that, to recover a representation of holistic figure-ground representations from image parts, the brain must achieve perceptual integration of visual information across co-linear space ^[1][2]. Contrast integration across co-linear features and borders plays a crucial role in perceptual organization under conditions of stimulus uncertainty and configurative ambiguity ^[18][26]. Governed by the Gestalt law of good continuation and reflected by psychophysically measurable interactive effects between co-axial stimuli in the visual field ^{[25][26][27][28]}, co-linear spatial integration results from specific response activities of visual cortical neurons and neuronal interactions “beyond the classic receptive field” ^{[34][35][36][37][38][39][40][41][42]}. These reveal the complexity of  hardwired, orientation selective brain mechanisms ^[43][44] underlying perceptual organization. Co-linear integration is crucial for the detection of alignment, virtual trajectories, and shape borders in a world where most objects are seen incompletely. It enables the human observer to assess the continuity of image fragments under conditions of diminished visibility and heightened stimulus ambiguity.

However, in complex images, some visible stimulus fragments appear clearly aligned, others do not. Specific phenomenal conditions need to be satisfied to enable collinear interpolation in static 2D scenes, and the process of interpolation constrains the spreading of surfaces across unspecified regions in the image ^[45][46]. Also, not all of perceptual organization is hardwired ^[47][48], and the contribution of past experience and perceptual learning to early mechanisms of perceptual integration needs to be taken into account given that specific memory data about objects and their most likely spatial configuration may facilitate or interfere with, depending on conditions, ongoing visual processing of ambiguous image data. The question relative to recovery of ‘veridical’ object properties from image data may be seen as a byproduct of perceptual organization, but was not an issue raised by Gestalt theory. However, the laws of perceptual organization have generated a conceptual framework for addressing this problem space in a bottom-up approach. The ‘veridicality’ of perceptual representations our brain may derive from visual stimuli raises deeper philosophical questions. Perceptual organization as such may be seen as resulting from pressure on the perceptual system to actively construct ordered representations of objects in depth in response to intrinsically ambiguous visual data. The biological role of perceptual organization, placed in the context of evolution, would be to ensure behavioral adaptation and success. Perceptual organization becomes mandatory as soon as the visual signals contained in an image or a scene reach our retina ^[49]. The major part of the necessary information processing then takes place in the visual brain, involving a whole chain of mechanisms well beyond the retina. Perceptual neuroscience has provided us with a diversified account for the many ways in which visual sensitivity to ordered structure and regularities expresses itself in behavior on the basis of cortical mechanisms. Multiple stages of neural processing transform fragmented signals into visual key representations of 3D scenes that can be used to control effective behavior. Since our survival depends on our ability to pick up order in the physical world, and since we conceive the physical world as an ordered one, our brain must  be sensitive to structural regularities in the physical world. Neural interactions “beyond the classic receptive field” drive the visual processing of texture dissimilarities, boundary completion, surface filling-in, and figure-ground segregation in the brain genesis of “perceptual order”.

3.  Structural Regularities in Images in Perceptual Organization

Statistically likely structural regularities in images are important to perceptual organization. Such regularities are required for visual textures to be discriminated from random noise by human observers. Ideal Observer and Order Observer models based on the capture of statistical regularities in textures were shown to closely reflect the sensitivity of the human visual system to structural regularities in images ^[50]. Symmetry ^[51][52] may facilitate the perceptual capture of such regularities. Spatiotemporal boundary formation is critical to the perception of shape boundaries in images and their global motion, as shown in experiments on boundary integration from spatially and temporally sparse transformations of texture ^[53]. The human perceptual system uses positions (x, y, z) and times (t) of element (n) transformations to extract local structural regularities such as edge fragments, which then connect by interpolation processes to produce larger shapes. This process seems to have a spatiotemporal integration constraint of ~80 ms between elements. Structural ambiguity and uncertainty pose a specific problem here. Perceptual organization not only accounts for the ways in which physically defined order is processed by the visual system, but also how such order may be inferred in cases of uncertainty, and how this may give rise to the perception of “illusory” or “phantom” shapes, as shown recently in a new phenomenon ^[54] in which illusory streaks irradiate from the vertexes of angular contours, connecting pairs of figures, and forming phantom net shapes that are only clearly visible in a specific figural context. The real world conditions or configurative order to which perceptual organization is sensitive also determine our aesthetic perception, as shown in work on aesthetic preferences ^[51] for structural regularities. A novel variant of perceptual organization, called ‘accentuation for perceptual coupling’ ^[55] describes new conditions for grouping that cannot be explained in terms of the classic Gestalt laws (similarity, co-linearity, or other). This recent work reveals the ‘supremacy of dissimilarity’ as an important figural complement to similarity in the genesis of aesthetic structure, involving novel dynamics of perceptual coupling.

The visual brain organizes incoming information early on and interprets a 2D image in terms of the real world, producing configurative order that provides perceptual continuity and enables object-based attention. The Form-And-Color-And-DEpth (FACADE) model and its laminar cortical embodiment and extension by the 3D LAMINART models ^[56] have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out figure-ground organization to generate 3D percepts from 2D pictures of occluding and occluded objects. The models provide a unified interpretation of groundbreaking neuroscience data ^[57] about cortical cells and their response properties for perceptual organization, with modulation by attention and temporal input properties. The segregation of image parts into foreground and background requires border ownership assignment by the visual system and recordings from monkey visual cortex show that many neurons, especially in area V2, are selective to border ownership. These neurons are edge selective and have classical receptive fields (CRF). Their responses are modulated (enhanced or suppressed) depending on the location of a “figure” relative to the edge in their receptive field. Each neuron has a fixed preference for location on one side or the other. This selectivity is derived from image context far beyond the CRF ^[31][32][42]. Recent work in this field suggests that border ownership selectivity reflects the formation of early prototypes of object representations, or “proto-objects” ^[57]. The evidence includes experiments showing (1) reversal of border ownership signals with change of perceived object structure, (2) border ownership specific enhancement of responses in object-based selective attention, (3) persistence of border ownership signals in accordance with continuity of object perception, and (4) remapping of border ownership signals across saccades and object movements. Grouping, persistence, and remapping are shown to be manifest in low-level visual mechanisms, which challenges any top-down-only model of perceptual organization. Perceptual judgments relative to figure and ground reveal that polarity-invariant as well as polarity-specific object properties are exploited in perceptual organization ^[18][19][58]. In some conditions, figure-ground is solely determined by the orientation of contrasts, not by their relative sign. Psychophysical data from human observers ^[18] responding to structurally ambiguous, fragmented 2D objects with varying local contrast sign across their edges reveal this type of polarity-invariant perceptual organization, which was already predicted by the earliest version of the FACADE theory ^[4].

In computational science, these processes of perceptual organization carry important meaning for the design and control of visual interfaces and the generation of reliable structural representations for image-based decision making ^[59][60]. In this context, understanding which among many possible image solutions will produce geometric configurations for optimal perceptual organization is akin to understanding which sentences in complex text will best convey the desired message or meaning. As a consequence, Gestalt theory is as relevant as ever in the context of computational vision and visual interface technology. Image-guided surgery, for example, exploits images taken before and/or during the procedure to help a surgeon navigate, and to augment his/her capacity for optimal decision making and action during the procedure ^[61]. With the help of augmented reality, visual guidance can be provided directly on a surgeon's view of the patient by mixing real and virtual images ^[60][61]. Perceptual qualities such as color ^[62], brightness, and relative feature salience in computationally preprocessed images with a large amount of random variations ^[63] are essential for making specific regions of interest to a specialist optimally perceptible.