Cognition: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor:

Cognition is the acquisition of knowledge by the mechanical process of information flow in a system. In animal cognition, input is received by the various sensory modalities and the output may be a motor or other action. The sensory information is internally transformed to a set of representations which is the basis for cognitive processing. This is in contrast to the traditional definition that is based on mental processes and a metaphysical description.

  • animal cognition
  • cognitive processes
  • physical processes
  • mental processes

1. Definition of Cognition

1.1. A Scientific Definition of Cognition

A dictionary will often define cognition as the mental process for the acquisition of knowledge. However, this view originated with the assignment of mental processes to the act of thinking. The mental processes are a metaphysical description of cognition which includes the concepts of consciousness and intentionality.[1][2] This also includes the concept that objects in nature are reflections of a true form.

Instead, a material description of cognition is restrictive to the physical processes of nature. An example is from the studies of primate face recognition where the measurable properties of facial features are the unit of recognition.[3] This perspective also excludes the concept that there is an innate knowledge of objects, so instead cognition forms a representation of an object from its constituent parts.[4][5] Therefore, the physical processes of cognition are probabilistic in nature since a specific object may vary in its parts.

1.2. Mechanical Perspective of Cognition

Most work in the sciences accepts a mechanical perspective of the information processes in the brain. However, the traditional perspective, including the descriptions of mental processing, is retained by some academic disciplines. For example, there is a conjecture about the relationship between the human mind and any simulation of it.[6] This idea is based on assumptions about intentionality and the act of thinking. However, a physical process of cognition is instead generated by neuronal cells instead of a dependence on a non-material process.[7]

Another example concerns the intention to move a body limb, such as in the act of reaching for a cup. Instead, studies have replaced the assignment of intentionality with a material interpretation of this motor action, and additionally showed that the relevant neuronal activity occurs before a perceptual awareness of the motor action.[8]

Across the sciences, the neural systems are studied at the different biological scales, including from the molecular level up to the higher level which involves the information processing.[9][10] The higher level perspective is of particular interest since there is an analogous system in the neural network models of computer science.[11][12] However, at the lower level, the artificial system is based on an abstract model of neurons and their synapses, so this level is less comparable to an animal brain.

1.3. Scope of this Definition

For this description of cognition, the definition is restricted to a set of mechanical processes. The process of cognition is also approached from a broad perspective along with a few examples from the visual system.

2. Cognitive Processes in the Visual System

2.1. Probabilistic Processes in Nature

The visual cortical system occupies about one-half of the cerebral cortex. Along with language processing in humans, vision is a major source of sensory information from the outside world. The complexity of these systems reveals a powerful evolutionary process. This process is observed across all cellular life and has led to numerous novelties. Biological evolution depends on physical processes, such as mutation and population exponentiality, and a geological time scale to build the complex systems in organisms.

An example of this complexity is studied in the formation of the camera eye. This type of eye is evolved from a simpler organ, such as an eye spot, and this occurrence required a large number of adaptations over time.[13][14]. Also, the camera eye evolved independently in vertebrate animals and cephalopods. This shows that animal evolution is a strong force for change, but may restricted by the genetic code and the phenotypes of an organism, particularly its cellular organization and structure.

The evolution of cognition is a similar process to the origin of the camera eye. The probabilistic processes that led to complexity in the camera eye will also drive the evolution of cognition and the organization and structure of an animal brain.

2.2. Abstract Encoding of Sensory Information

“The biologically plausible proximate mechanism of cognition originates from the receipt of high dimensional information from the outside world. In the case of vision, the sensory data consist of reflected light rays that are absorbed across a two-dimensional surface, the retinal cells of the eye. These light rays range across the electromagnetic spectra, but the retinal cells are specific to a small subset of all possible light rays”.[1]

Figure 1 shows an abstract view of a sheet of neuronal cells that receives information form the outside world. This information is specifically processed by cell surface receptors and communicated downstream to a neural system. The sensory neurons and their receptors may be abstractly considered as a set of activation values that changes over time, a dynamical process.

Neurosci 02 00010 g001 550

Figure 1. An abstract representation of information that is received by a sensory organ, such as the light rays that are absorbed by neuronal cells along the surface of the retina of a camera eye.

The question is how the downstream processes of cognition work. This includes how knowledge is generalized, also called transfer learning, from the sensory input data.[5][15][16] A part of the problem is solved by segmenting the world and identifying objects with resistance to viewpoint (Figure 2).[17] There is a model from computer science[4] that is designed to overcome much of this problem. This approach includes the sampling of visual data, including the parts of objects, and then encoding the information in an abstract form. This encoding scheme includes a set of discrete representational levels of an unlabeled object, and then employs a consensus-based approach to match these representations to a known object.

 

Neurosci 02 00010 g002 550

Figure 2. The first panel is a visual drawing of the digit nine (9), while the next panel is the same digit but transformed by rotation of the image.

3. Models of General Cognition

3.1. Mathematical Description of Cognition

Experts in the sciences have investigated the question on whether there is an algorithm that describes brain computation.[18] It was concluded that this is an unsolved problem of mathematics, even though every natural process is potentially representable by a model. Further, they identified the brain as a nonlinear dynamical system. The information flow is a complex phenomenon and is analogous to that of the physics of fluid flow. Another expectation is that this system is high dimensional and not represented by a simple set of math equations.[18][19] They further suggested that a more empirical approach to explaining the system is a viable path forward.

The artificial system, like in the deep learning architecture, has a lot of potential for an empirical understanding of cognition. This is expected since artificial systems are built from parts and interrelationships that are known, whereas in nature the history of the neural system is obscured, and the understanding of its parts require experimentation that is often imprecise and confounded with error.

3.2. Encoding of Knowledge in Cognitive Systems

It is possible to hypothesize about a model of object representation in the brain and its artificial analog, a deep learning system. First, these cognitive systems are expected to encode objects by their parts, or elements, a topic that is covered above.[3][4][5] Second, it is expected that this is a stochastic process, and in the artificial system the encoding scheme is in the weight values that are assigned to the interconnections among nodes of the network. It is further expected that the brain functions similarly at this level, given that the systems are based on non-linear dynamics and distributed representations of objects.[5][18][20][21]

These encoding schemes are expected to be abstract and not of a deterministic design based on a top-down only process. Since cognition is also considered a non-linear dynamical system, the encoding of the representations is expected to be highly distributed among the parts of the neural network.[18][21] This is testable in an artificial system and in the brain.

Further, a physical interpretation of cognition requires the matching of patterns to generalize knowledge of the outside world. This is consistent with a view of the cognitive systems as statistical machines with a reliance on sampling for achieving robustness in its output. With the advances in deep learning methods, such as the invention of the transformer architecture[5][22], it is possible to sample and search for exceedingly complex sequence patterns. Also, the sampling of the world occurs within a sensory modality, such as from visual or language data, and this is complemented by a sampling among the modalities which potentially leads to increased robustness in the output.[23]

3.3. Future Directions

3.3.1. Dynamic Properties of Cognition

One question is whether animal cognition is as interpretable as a deep learning system. This question arises because of the difficulty in disentangling the mechanisms of the animal brain, whereas it is possible to record the changing states of an artificial system given the design is bottom-up. If the artificial system is similar enough, then it is possible to gain insight into the mechanisms of animal cognition.[5][24] However, a problem in this assumption may occur. For example, it is known that the mammalian brain is highly dynamic, such as in the rates of sensory input and in the downstream activation of the internal representations.[18] These dynamic properties are not feasibly modeled in current artificial systems since there are constraints on hardware design and efficiency.[18] This is an impediment to design of an artificial system that is approximate of animal cognition. Having an artificial system that includes an overlay architecture with “fast weights” is expected to provide this form of true recursion in processing information from the outside world.[5][18]

Since the artificial neural network systems continue to scale in capability, it is reasonable to continue to use an empirical approach to explore any sources of error, whether inherent in the method or a result that is not expected. This requires a thorough understanding on how these models work at all levels. One strategy for producing robust output is to combine the various kinds of sensory information, such as both visual and language data. Another strategy has been to establish unbiased measures of the reliability of output from a model. It should be noted that animal cognition is not immune to error either. In the case of human cognition, there is a bias problem in perception of speech.[25]

3.3.2. Generalization of Knowledge

Another area of importance is the property of generalization for a model of cognition. This goal could be approached by processing the particular levels of representation from sensory input, the presumed process that occurs in animals and in their ability to generalize and form knowledge.[4][26][27] In a larger context, this generalizability is based on the premise that information of the outside world is compressible, such as in the repeatability of patterns in the information.

There is also the question of how to reuse knowledge outside the environment where it is learned, "being able to factorize knowledge into pieces which can easily be recombined in a sequence of computational steps, and being able to manipulate abstract variables, types, and instances".[5] It seems relevant to have a model of cognition that describes high level representations of these "pieces" of the whole, even in the case of an abstract object. However, the dynamic states of internal representations in cognition may contribute to the processes of abstract reasoning.

3.3.3. Embodiment in Cognition

Lastly, there is a question on the dependence of animal cognition on the outside world. This dependence has been characterized as the phenomenon of embodiment, so the cognition is an embodied cognition, even in the case where the outside world is a machine simulation of it.[18][28][29] This is in essence a property of a robot, where its functioning is dependent on input and output from the outside world. Although a natural system would receive input, produce output, and thus learn from the world in a time scale constrained by the physical world, a somewhat alternative approach in an artificial system is to use reinforcement learning,[29][30][31] a method that has been used to approximate the sensorimotor function of animals.

A paper by Deepmind[29] describes artificial agents in a three-dimensional space where they learn from a world that is undergoing change. The method uses a deep reinforcement learning approach combined with a dynamic generation of environments for building each of the worlds. Each world has artificial agents that learn to undergo tasks and receive a reward for completion of objectives. An agent observes the pixel image of an environment and "receives a text description of their goal".[29] Knowledge of these tasks is sufficiently generalizable that the agents are often adapted for tasks that are not yet known from a previously generated world. This approach reflects an animal that is embodied in a three-dimensional world and learning interactively by performing tasks. It is known that animals navigate and learn from the world around them, so the above approach is a thoughtful experiment for creating similar circumstances to a virtual one.

4. Abstract Reasoning

4.1. Abstract Reasoning as a Cognitive Process

Abstract reasoning is often associated with a process of thinking, but the elements of this process are ideally modeled by physical processes in the brain. This recommendation offers constraints on the emergence of reasoning on abstract concepts, such as the formation of new ideas. Further, a process of abstract reasoning may be contrasted with visual and speech recognition. Both these sensory forms of cognition occur by sensory input to the neural systems of the brain. Without this sensory input, then the layers of the neural systems are not expected to encode a corresponding pathway, such as for recognition of visual objects. It is necessary that these systems are trained on input sources.

Therefore, it follows that abstract reasoning is formed from input sources which are received by the neural systems of the brain. If there is no input of information that resembles the input for a formal pathway of abstract reasoning, then the brain is not expected to substantially encode that pathway. This leads to the hypothesis on whether abstract reasoning is mainly a single pathway or a number of many pathways, and the possible contribution from other pathways, such as from the processes of non-abstract reasoning. It is probable that there is no sharp division between abstract reasoning and the other forms of reasoning, and there are likely many types of abstract reasoning, including the reasoning used for solving a class of related visual puzzles.

Another hypothesis is whether the input sources for abstract reasoning occur mainly from the internal representations of the brain. If true, then a model of abstract reasoning would mainly involve the true forms of abstract objects, in contrast to the recognition of an object by reconstruction from sensory input.

Of greater certainty is that abstract reasoning is dependent on an input source, so there is an expectation that deep learning methods, the modeling of non-linear dynamics of a phenomenon, are sufficient to model one or more pathways of abstract reasoning. This form of reasoning involves recognition of objects that are not necessarily sensory objects, with definable properties and interrelationships. As with the training process to learn of sensory objects and their properties, it is expected that there is a training process to learn about the forms and properties of abstract objects. This class of problem is interesting since the universe of abstract objects is large and their properties and interrelationships are not constrained by the external and physical world.

4.2. Abstract Reasoning in General

A model of high level cognition includes the process of abstract reasoning.[5] This is a pathway or pathways that are expected to learn the high level representations in sensory information, such as visual or auditory, so that novel input generates output that is based on a set of rules. These rule sets are also expected to have generalized applicability. The rule set may include a single rule or multiple rules that occur in a sequence. One method for a solution is to have a deep learning system learn the rule set, such as in the case of a visual puzzle which is solved by use of a logical operation.[32] This is likely similar to one of the major ways that a person masters the game of chess, a memorization of priors for patterns and events of chess pieces on the game board.

Another kind of visual puzzle is a Rubik's Cube. However, in this case the puzzle has a known final state where each face of the cube shares a unique color. In the general case of visual puzzles, if there is no detectable rule set to solve the puzzle, then a person or a machine system should conclude that no rule set exists. If there is a detectable rule set, then there must be patterns of information, including missing information, that allow detection of the rule set. It is also possible that a particular rule set or those with many steps are not solvable by a person.

The pathway to a solution should include the repeated testing of potential rule sets against an intermediate or final state of the puzzle. This iterative process may be approached by an heuristic search algorithm.[5] However, these puzzles are typically low dimensional as compared to abstract verbal problems, such as in the general process of inductive reasoning. The acquisition of the rule sets for verbal reasoning require a search for patterns in this higher dimensional space. In either of these cases of pattern searching, whether complex or simple, they are dependent on the detection of patterns that represent the rule sets.

It is simpler to imagine a logical operation as the pattern that provides a solution, but it is expected that a process of inductive reasoning involves higher dimensional representations than an operator that combines boolean values. It is also probable that these representations are dynamic, so that there is a possibility to sample the space of valid representations.

4.3. Future Directions

4.3.1. Embodiment in a Virtual and Abstract World

While the phenomenon of embodiment refers to our three-dimensional world, this is not necessarily a complete model for reasoning about abstract concepts. However, it is plausible that at least some abstract concepts are modeled in a virtual three-dimensional world, and the paper on this topic by Deepmind[29] shows solutions to visual problems among the generated space of three-dimensional worlds.

The population of tasks and their distribution are also constructed by Deepmind's machine learning approach. They show that learning a large distribution of tasks can provide knowledge to solve tasks that are not necessarily within the confines of the original task distribution.[29][30] This furthers shows a generalizability in solving tasks, along with the promise that increasingly complex worlds will lead to an expanded knowledge of tasks.

However, the problem of abstract concepts extends beyond the conventional sensory representations as formed by the processes of cognition. Examples include visual puzzles with solutions that are abstract and require the association of patterns that extend beyond the visual realm, and the symbolic representations in mathematics.[33]

Combining these two approaches, it is possible to construct a world that is not a reflection of three-dimensional space as inhabited by animals, but instead to construct a virtual world that consists of abstract objects and sets of tasks[30]. The visual and symbolic puzzles, such as in the case of chess and related boardgames[31], are solvable by deep learning approaches, but the machine reasoning is not generalized across a space of abstract environments and objects. The question is whether the abstract patterns to solve chess are also useful in solving other kinds of puzzles. It seems a valid hypothesis that there is at least overlap in the use of abstract reasoning between these visual puzzles and the synthesis of knowledge from other abstract objects and their interactions[29], such as in solving problems by the use of mathematical symbols and their operators. Since we are capable of abstract thought, it is plausible that generation of a distribution of general abstract tasks would lead to a working system that solves abstract problems in general.

If instead of a dynamic generation of three-dimensional worlds and objects, there is a vast and dynamic generation of abstract puzzles, for example, then the deep reinforcement learning approach could train on solving these problems and acquire knowledge of the abstract tasks.[29] The question is whether the distribution of these applicable tasks is generalizable to an unknown set of problems, those unrelated to the original task distribution, and likewise that the task space is compressible.

4.3.2. Reinforcement Learning and Generalizability

Google Research showed that an unmodified reinforcement learning approach is not necessarily robust for acquiring knowledge of tasks outside the trained task distribution.[30] Therefore, they introduced an approach that incorporates a measurement for similarity among worlds that they generated during the reinforcement learning procedure. This similarity measure is estimated by behavioral similarity, corresponding to the salient features by which an agent finds success in any given world. Given these salient features are shared among the worlds, the agents have a path for generalizing knowledge for success in worlds outside their trained experience. Procedurally, the salient features are acquired by a contrastive learning procedure, and embeds these values of behavioral similarity in the neural network itself.[34]

The above reinforcement learning approach is dependent on both a deep learning framework and an input data source. The source of input data is typically a two or three dimensional environment where an agent learns to accomplish tasks within the confines of the worlds and their rules.[29][30] One goal is to represent the salient features of tasks and the worlds in a neural network. As Google Research showed[30], the process required an additional step to extract the salient information for creating better models of the tasks and their worlds. They found that this method is more robust for generalizing tasks. Similarly, in animal cognition, it is expected that the salient features to generalize a task are also stored in a neural network.

Therefore, a naive input of visual data from a two dimensional environment is not an efficient means to code for tasks that robustly generalize across environments. To capture the higher dimensional information in a set of related tasks, Google Research extended the reinforcement learning approach to better capture the task distribution[30], and it may be possible to mimic this approach by similar methods. These task distributions provide structured data for representing the dynamics of tasks among worlds, and therefore generalize and encode the higher dimensional and dynamic features to a lower dimensional form.[29]

It is difficult to imagine the relationship between two different environments. A game of checkers and chess appear as different games and environments. Encoding the dynamics of each of these games in a deep learning framework may show that they relate in an abstract way.[29] This concept was expressed in the above article[30], that short paths in a larger pathway may provide salient and generalizable features. In the case of boardgames, the salient features may not correspond to a naive perception of visual relatedness. Likewise, our natural form of abstract reasoning shows that we capture patterns in these boardgames, and these patterns are not entirely recognized by a single ruleset at the level of our awareness, but instead are likely represented at a higher dimensional level in a neural network.

Lastly, since the advanced cognitive processes in animals involve a widespread use of dynamic representations, typically called mental representations, it is plausible that the tasks are not just generalizable, but may originate in the varied information systems of the brain, such as speech, vision, and memory. Therefore, the tasks may be expressed by different forms, although the lower dimensional representations are more generalizable, providing a better substrate for the recognition of patterns, and essential for any general process of abstract reasoning.

References

  1. Friedman, R. Cognition as a Mechanical Process. NeuroSci 2021, 2, 141-150.
  2. Vlastos, G. Parmenides’ theory of knowledge. In Transactions and Proceedings of the American Philological Association; The Johns Hopkins University Press: Baltimore, MD, USA, 1946; pp. 66–77.
  3. Chang, L.; Tsao, D.Y. The code for facial identity in the primate brain. Cell 2017, 169, 1013-1028.
  4. Hinton, G. How to represent part-whole hierarchies in a neural network. 2021, arXiv:2102.12627.
  5. Bengio, Y.; LeCun, Y.; Hinton G. Deep Learning for AI. Commun ACM 2021, 64, 58-65.
  6. Searle, J.R.; Willis, S. Intentionality: An essay in the philosophy of mind. Cambridge University Press, Cambridge, UK, 1983.
  7. Huxley, T.H. Evidence as to Man's Place in Nature. Williams and Norgate, London, UK, 1863.
  8. Haggard, P. Sense of agency in the human brain. Nat Rev Neurosci 2017, 18, 196-207.
  9. Ramon, Y.; Cajal, S. Textura del Sistema Nervioso del Hombre y de los Vertebrados trans. Nicolas Moya, Madrid, Spain, 1899.
  10. Kriegeskorte, N.; Kievit, R.A. Representational geometry: integrating cognition, computation, and the brain. Trends Cognit Sci 2013, 17, 401-412.
  11. Hinton, G.E. Connectionist learning procedures. Artif Intell 1989, 40, 185-234.
  12. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw 2015, 61, 85-117.
  13. Paley, W. Natural Theology: or, Evidences of the Existence and Attributes of the Deity, 12th ed., London, UK, 1809.
  14. Darwin, C. On the origin of species. John Murray, London, UK, 1859.
  15. Goyal, A.; Didolkar, A.; Ke, N.R.; Blundell, C.; Beaudoin, P.; Heess, N.; Mozer, M.; Bengio, Y. Neural Production Systems. 2021, arXiv:2103.01937.
  16. Scholkopf, B.; Locatello, F.; Bauer, S.; Ke, N.R.; Kalchbrenner, N.; Goyal, A.; Bengio, Y. Toward Causal Representation Learning. In Proceedings of the IEEE, 2021.
  17. Wallis, G.; Rolls, E.T. Invariant face and object recognition in the visual system. Prog Neurobiol 1997, 51, 167-194.
  18. Rina Panigrahy (Chair), Conceptual Understanding of Deep Learning Workshop. Conference and Panel Discussion at Google Research, May 17, 2021. Panelists: Blum, L; Gallant, J; Hinton, G; Liang, P; Yu, B.
  19. Gibbs, J.W. Elementary Principles in Statistical Mechanics. Charles Scribner's Sons, New York, NY, 1902.
  20. Griffiths, T.L.; Chater, N.; Kemp, C.; Perfors, A; Tenenbaum, J.B. Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences 2010, 14, 357-364.
  21. Hinton, G.E.; McClelland, J.L.; Rumelhart, D.E. Distributed representations. In Parallel distributed processing: explorations in the microstructure of cognition; Rumelhart, D.E., McClelland, J.L., PDP research group, Eds., Bradford Books: Cambridge, Mass, 1986.
  22. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. 2017, arXiv:1706.03762.
  23. Hu, R.; Singh, A. UniT: Multimodal Multitask Learning with a Unified Transformer. 2021, arXiv:2102.10772.
  24. Chaabouni, R.; Kharitonov, E.; Dupoux, E.; Baroni, M. Communicating artificial neural networks develop efficient color-naming systems. Proceedings of the National Academy of Sciences 2021, 118.
  25. Petty, R.E.; Cacioppo, J.T. The elaboration likelihood model of persuasion. In Communication and Persuasion; Springer: New York, NY, 1986, pp. 1-24.
  26. Chase, W.G.; Simon, H.A. Perception in chess. Cognitive psychology 1973, 4, 55-81.
  27. Pang, R.; Lansdell, B.J.; Fairhall, A.L. Dimensionality reduction in neuroscience. Curr Biol 2016, 26, R656-R660.
  28. Deng, E.; Mutlu, B.; Mataric, M. Embodiment in socially interactive robots. 2019, arXiv:1912.00312.
  29. Team, E.L., Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M. and McAleese, N. Open-ended learning leads to generally capable agents. 2021, arXiv:2107.12808.
  30. Agarwal, R., Machado, M.C., Castro, P.S. and Bellemare, M.G. Contrastive behavioral similarity embeddings for generalization in reinforcement learning. 2021, arXiv:2101.05265.
  31. Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; Lillicrap, T.; Simonyan, K.; Hassabis, D. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 2018, 362, 1140-1144.
  32. Barrett, D.; Hill, F.; Santoro, A.; Morcos, A.; Lillicrap, T. Measuring abstract reasoning in neural networks. In International Conference on Machine Learning, PMLR, 2018.
  33. Schuster, T., Kalyan, A., Polozov, O. and Kalai, A.T. Programming Puzzles. 2021, arXiv:2106.05784.
  34. Chen, T., Kornblith, S., Norouzi, M. and Hinton, G. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, PMLR, 2020.
More
This entry is offline, you can click here to edit this entry!