1. Please check and comment entries here.
Table of Contents

    Topic review


    View times: 201
    Submitted by: Robert Friedman


    Cognition is the acquisition of knowledge by the mechanical process of information flow in a system. In animal cognition, input is received by the various sensory modalities and the output may be a motor or other action. The sensory information is internally transformed to a set of representations which is the basis for cognitive processing. This is in contrast to the traditional definition that is based on mental processes and a metaphysical description.

    1. Introduction

    1.1. A Scientific Definition of Cognition

    A dictionary will often define cognition as the mental process for the acquisition of knowledge. However, this view originated with the assignment of mental processes to the act of thinking. The mental processes are a metaphysical description of cognition which includes the concepts of consciousness and intentionality.[1][2] This also includes the concept that objects in nature are reflections of a true form.

    Instead, a material description of cognition is restrictive to the physical processes of nature. An example is from the studies of primate face recognition where the measurable properties of facial features are the unit of recognition.[3] This perspective also excludes the concept that there is an innate knowledge of objects, so instead cognition forms a representation of an object from its constituent parts.[4][5] Therefore, the physical processes of cognition are probabilistic in nature since a specific object may vary in its parts.

    1.2. Mechanical Perspective of Cognition

    Most work in the sciences accepts a mechanical perspective of the information processes in the brain. However, the traditional perspective, including the descriptions of mental processing, is retained by some academic disciplines. For example, there is a conjecture about the relationship between the human mind and any simulation of it.[6] This idea is based on assumptions about intentionality and the act of thinking. However, a physical process of cognition is instead generated by neuronal cells instead of a dependence on a non-material process.[7]

    Another example concerns the intention to move a body limb, such as in the act of reaching for a cup. Instead, studies have replaced the assignment of intentionality with a material interpretation of this motor action, and additionally showed that the relevant neuronal activity occurs before a perceptual awareness of the motor action.[8]

    Across the sciences, the neural systems are studied at the different biological scales, including from the molecular level up to the higher level which involves the information processing.[9][10] The higher level perspective is of particular interest since there is an analogous system in the neural network models of computer science.[11][12] However, at the lower level, the artificial system is based on an abstract model of neurons and their synapses, so this level is less comparable to an animal brain.

    1.3. Scope of this Definition

    For this description of cognition, the definition is restricted to a set of mechanical processes. The process of cognition is also approached from a broad perspective along with a few examples from the visual system.

    2. Cognitive Processes in the Visual System

    2.1. Probabilistic Processes in Nature

    The visual cortical system occupies about one-half of the cerebral cortex. Along with language processing in humans, vision is a major source of sensory information from the outside world. The complexity of these systems reveals a powerful evolutionary process. This process is observed across all cellular life and has led to numerous novelties. Biological evolution depends on physical processes, such as mutation and population exponentiality, and a geological time scale to build the complex systems in organisms.

    An example of this complexity is studied in the formation of the camera eye. This type of eye is evolved from a simpler organ, such as an eye spot, and this occurrence required a large number of adaptations over time.[13][14]. Also, the camera eye evolved independently in vertebrate animals and cephalopods. This shows that animal evolution is a strong force for change, but may restricted by the genetic code and the phenotypes of an organism, particularly its cellular organization and structure.

    The evolution of cognition is a similar process to the origin of the camera eye. The probabilistic processes that led to complexity in the camera eye will also drive the evolution of cognition and the organization and structure of an animal brain.

    2.2. Abstract Encoding of Sensory Information

    “The biologically plausible proximate mechanism of cognition originates from the receipt of high dimensional information from the outside world. In the case of vision, the sensory data consist of reflected light rays that are absorbed across a two-dimensional surface, the retinal cells of the eye. These light rays range across the electromagnetic spectra, but the retinal cells are specific to a small subset of all possible light rays”.[1]

    Figure 1 shows an abstract view of a sheet of neuronal cells that receives information form the outside world. This information is specifically processed by cell surface receptors and communicated downstream to a neural system. The sensory neurons and their receptors may be abstractly considered as a set of activation values that changes over time, a dynamical process.

    Neurosci 02 00010 g001 550

    Figure 1. An abstract representation of information that is received by a sensory organ, such as the light rays that are absorbed by neuronal cells along the surface of the retina of a camera eye.

    The question is how the downstream processes of cognition work. This includes how knowledge is generalized, also called transfer learning, from the sensory input data.[5][15][16] A part of the problem is solved by segmenting the world and identifying objects with resistance to viewpoint (Figure 2).[17] There is a model from computer science[4] that is designed to overcome much of this problem. This approach includes the sampling of visual data, including the parts of objects, and then encoding the information in an abstract form. This encoding scheme includes a set of discrete representational levels of an unlabeled object, and then employs a consensus-based approach to match these representations to a known object.


    Neurosci 02 00010 g002 550

    Figure 2. The first panel is a visual drawing of the digit nine (9), while the next panel is the same digit but transformed by rotation of the image.

    3. Models of General Cognition

    3.1. Mathematical Description of Cognition

    Experts in the sciences have investigated the question on whether there is an algorithm that describes brain computation.[18] It was concluded that this is an unsolved problem of mathematics, even though every natural process is potentially representable by a model. Further, they identified the brain as a nonlinear dynamical system. The information flow is a complex phenomenon and is analogous to that of the physics of fluid flow. Another expectation is that this system is high dimensional and not represented by a simple set of math equations.[18][19] They further suggested that a more empirical approach to explaining the system is a viable path forward.

    The artificial system, like in the deep learning architecture, has a lot of potential for an empirical understanding of cognition. This is expected since artificial systems are built from parts and interrelationships that are known, whereas in nature the history of the neural system is obscured, and the understanding of its parts require experimentation that is often imprecise and confounded with error.

    3.2. Encoding of Knowledge in Cognitive Systems

    It is possible to hypothesize about a model of object representation in the brain and its artificial analog, a deep learning system. First, these cognitive systems are expected to encode objects by their parts, or elements, a topic that is covered above.[3][4][5] Second, it is expected that this is a stochastic process, and in the artificial system the encoding scheme is in the weight values that are assigned to the interconnections among nodes of a network. It is further expected that the brain functions similarly at this level, given that the systems are based on nonlinear dynamics and distributed representations of objects.[5][18][20][21]

    These encoding schemes are expected to be abstract and not of a deterministic design based on a top-down only process. Since cognition is also considered a nonlinear dynamical system, the encoding of the representations is expected to be highly distributed among the parts of the neural network.[18][21] This is testable in an artificial system and in the brain.

    Further, a physical interpretation of cognition requires the matching of patterns to generalize knowledge of the outside world. This is consistent with a view of the cognitive systems as statistical machines with a reliance on sampling for achieving robustness in its output. With the advances in deep learning methods, such as the invention of the transformer architecture[5][22], it is possible to sample and search for exceedingly complex sequence patterns. Also, the sampling of the world occurs within a sensory modality, such as from visual or language data, and this is complemented by a sampling among the modalities which potentially leads to increased robustness in the output.[23]

    3.3. Future Directions of Study

    3.3.1 Dynamic Properties of Cognition

    One question has been whether animal cognition is as interpretable as a deep learning system. This arose because of the difficulty in disentangling the mechanisms of the animal brain, whereas it is possible to record the changing states in an artificial system since the design is bottom-up. If the artificial system is similar enough, then it is possible to gain insight into the mechanisms of animal cognition.[5][24] However, a problem in this assumption may occur. For example, it is known that the mammalian brain is highly dynamic, such as in the rates of sensory input processing and the downstream activation of the internal representations.[18] These dynamic properties are not feasibly modeled in current artificial systems since there are constraints on hardware design and its efficiency.[18] This is an impediment to design of an artificial system that is approximate of animal cognition. Having an artificial system that includes an overlay architecture with “fast weights” is expected to provide this form of true recursion in processing information from the outside world.[5][18]

    Since the artificial neural network systems continue to scale in capability, it is reasonable to continue to use an empirical approach to explore any sources of error, whether inherent in the method or a result that is not expected. This requires a thorough understanding on how these models work at all levels. One strategy for producing robust output has been to combine the various kinds of sensory information, such as both visual and language data. Another strategy has been to establish unbiased measures of the reliability of output from a model. It should be noted that animal cognition is not immune to error either. In the case of human cognition, there is a bias problem in perception of speech.[25]

    3.3.2 Generalization of Knowledge

    Another area of importance is the property of generalization in the case of a model of cognition. This goal could be approached by processing the particular levels of representation of sensory input, the presumed process that occurs in animals and their ability to generalize knowledge.[4][26][27] In a larger context, this generalizability is based on the premise that information of the outside world is compressible, such as in the repeatability of patterns in the information.

    There is also the question of how to reuse knowledge outside the environment where it is learned, "being able to factorize knowledge into pieces which can easily be recombined in a sequence of computational steps, and being able to manipulate abstract variables, types, and instances".[5] It seems relevant to have a model of cognition that describes high level representations of these "pieces" of the whole, even in the case of an abstract object. However, the dynamic states of internal representations in cognition may contribute to the processes of abstract reasoning.

    3.3.3 Abstract Reasoning

    A model of high level of cognition includes the process of abstract reasoning.[5] This is a pathway or pathways that are expected to learn the high level representations in sensory information, such as visual or auditory, so that novel input generates output that is based on a set of rules. These rule sets are also expected to have generalized applicability. The rule set may include a single rule or multiple rules that occur in a sequence. One method for a solution is to have a deep learning system learn the rule set, such as in the case of a visual puzzle which is solved by use of a logical operation.[28] This is likely similar to one of the major ways that a person masters the game of chess, a memorization of priors for patterns and events of chess pieces on the game board.

    Another kind of visual puzzle is a Rubik's Cube. However, in this case the puzzle has a known final state where each face of the cube shares a unique color. In the general case of visual puzzles, if there is no detectable rule set to solve the puzzle, then a person or a machine system should conclude that no rule set exists. If there is a detectable rule set, then there must be patterns of information, including missing information, that allow detection of the rule set. It is also possible that a particular rule set or those with many steps are not solvable by a person.

    The pathway to a solution should include repeated testing of potential rule sets against an intermediate or final state of the puzzle. This iterative process may be approached by an heuristic search algorithm.[5] However, these puzzles are typically low dimensional as compared to abstract verbal problems, such as in the general process of inductive reasoning. The acquisition of the rule sets for verbal reasoning require a search for patterns in this higher dimensional space. In either of these cases of pattern searching, whether complex or simple, they are dependent on the detection of patterns that represent the rule sets.

    It is simpler to imagine a logical operation as the pattern that provides a solution, but it is expected that a process of inductive reasoning involves higher dimensional representations than an operator that combines boolean values. It is also probable that these representations are dynamic in a person, so that there is a possibility to sample the space of valid representations.

    3.3.4 Phenomenon of Embodiment

    Lastly, there is a question on the dependence of animal cognition on the outside world. This dependence has been characterized as the phenomenon of embodiment, so the cognition is an embodied cognition, even in the case where the outside world is a machine simulation of it.[18][29] This is essentially a property of a robot, where its functioning is dependent on input and output from the outside world. Although a natural system would receive input, produce output, and thus learn from the world along some constrained time scale, a somewhat alternative approach in an artificial system is reinforcement learning,[30] a method that has been used to approximate the sensorimotor capability of animals.


    1. Friedman, R. Cognition as a Mechanical Process. NeuroSci 2021, 2, 141-150.
    2. Vlastos, G. Parmenides’ theory of knowledge. In Transactions and Proceedings of the American Philological Association; The Johns Hopkins University Press: Baltimore, MD, USA, 1946; pp. 66–77.
    3. Chang, L.; Tsao, D.Y. The code for facial identity in the primate brain. Cell 2017, 169, 1013-1028.
    4. Hinton, G. How to represent part-whole hierarchies in a neural network. 2021, arXiv:2102.12627.
    5. Bengio, Y.; LeCun, Y.; Hinton G. Deep Learning for AI. Commun ACM 2021, 64, 58-65.
    6. Searle, J.R.; Willis, S. Intentionality: An essay in the philosophy of mind. Cambridge University Press, Cambridge, UK, 1983.
    7. Huxley, T.H. Evidence as to Man's Place in Nature. Williams and Norgate, London, UK, 1863.
    8. Haggard, P. Sense of agency in the human brain. Nat Rev Neurosci 2017, 18, 196-207.
    9. Ramon, Y.; Cajal, S. Textura del Sistema Nervioso del Hombre y de los Vertebrados trans. Nicolas Moya, Madrid, Spain, 1899.
    10. Kriegeskorte, N.; Kievit, R.A. Representational geometry: integrating cognition, computation, and the brain. Trends Cognit Sci 2013, 17, 401-412.
    11. Hinton, G.E. Connectionist learning procedures. Artif Intell 1989, 40, 185-234.
    12. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw 2015, 61, 85-117.
    13. Paley, W. Natural Theology: or, Evidences of the Existence and Attributes of the Deity, 12th ed., London, UK, 1809.
    14. Darwin, C. On the origin of species. John Murray, London, UK, 1859.
    15. Goyal, A.; Didolkar, A.; Ke, N.R.; Blundell, C.; Beaudoin, P.; Heess, N.; Mozer, M.; Bengio, Y. Neural Production Systems. 2021, arXiv:2103.01937.
    16. Scholkopf, B.; Locatello, F.; Bauer, S.; Ke, N.R.; Kalchbrenner, N.; Goyal, A.; Bengio, Y. Toward Causal Representation Learning. In Proceedings of the IEEE, 2021.
    17. Wallis, G.; Rolls, E.T. Invariant face and object recognition in the visual system. Prog Neurobiol 1997, 51, 167-194.
    18. Rina Panigrahy (Chair), Conceptual Understanding of Deep Learning Workshop. Conference and Panel Discussion at Google Research, May 17, 2021. Panelists: Blum, L; Gallant, J; Hinton, G; Liang, P; Yu, B.
    19. Gibbs, J.W. Elementary Principles in Statistical Mechanics. Charles Scribner's Sons, New York, NY, 1902.
    20. Griffiths, T.L.; Chater, N.; Kemp, C.; Perfors, A; Tenenbaum, J.B. Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences 2010, 14, 357-364.
    21. Hinton, G.E.; McClelland, J.L.; Rumelhart, D.E. Distributed representations. In Parallel distributed processing: explorations in the microstructure of cognition; Rumelhart, D.E., McClelland, J.L., PDP research group, Eds., Bradford Books: Cambridge, Mass, 1986.
    22. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. 2017, arXiv:1706.03762.
    23. Hu, R.; Singh, A. UniT: Multimodal Multitask Learning with a Unified Transformer. 2021, arXiv:2102.10772.
    24. Chaabouni, R.; Kharitonov, E.; Dupoux, E.; Baroni, M. Communicating artificial neural networks develop efficient color-naming systems. Proceedings of the National Academy of Sciences 2021, 118.
    25. Petty, R.E.; Cacioppo, J.T. The elaboration likelihood model of persuasion. In Communication and Persuasion; Springer: New York, NY, 1986, pp. 1-24.
    26. Chase, W.G.; Simon, H.A. Perception in chess. Cognitive psychology 1973, 4, 55-81.
    27. Pang, R.; Lansdell, B.J.; Fairhall, A.L. Dimensionality reduction in neuroscience. Curr Biol 2016, 26, R656-R660.
    28. Barrett, D.; Hill, F.; Santoro, A.; Morcos, A.; Lillicrap, T. Measuring abstract reasoning in neural networks. In International Conference on Machine Learning, PMLR, 2018.
    29. Deng, E.; Mutlu, B.; Mataric, M. Embodiment in socially interactive robots. 2019, arXiv:1912.00312.
    30. Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; Lillicrap, T.; Simonyan, K.; Hassabis, D. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 2018, 362, 1140-1144.