Risk Determination versus Risk Perception: History
Please note this is an old version of this entry, which may differ significantly from the current revision.
Subjects: Robotics

Researchers review the progress in developing a science of interdependence applied to the determinations and perceptions of risk for autonomous human–machine systems based on a case study of the Department of Defense’s (DoD) faulty determination of risk in a drone strike in Afghanistan; the DoD’s assessment was rushed, suppressing alternative risk perceptions.

  • interdependence
  • autonomy
  • human–machine systems

1. Introduction

The research tested in the open for complex and uncertain environments is critical to advance the science of autonomy and artificial intelligence (AI) for interdependent human-machine teams (AHMTs) and systems (the National Academy of Sciences, in Endsley et al., 2021; e.g., for an example, see Doctor and colleagues, 2022). But to date, social science’s contributions to autonomy have been marginal. We speculate that disembodied cognition used in the laboratory is at the root of the replication crisis in social science (e.g., Nosek, 2015). We believe that including interdependence, as difficult as it is to manage in the laboratory (Jones, 1998, p. 33), should begin to remedy the problem.

First, one barrier to an AHMT, noted by the National Academy of Sciences (Endsley et al., 2021), was that most proposals for the design and operation of AHMTs are based on laboratory results that do not work well if at all in the real world. In agreement with the Academy, Pearl (viz., causality; in Pearl, 2002; Pearl & Mackenzie, 2018) has long demanded that AI researchers include contact with reality in their models.

Second, for humans working with AI, however, embodied thoughts derived while operating in reality cannot be decomposed from each other (Schapiro & Spaulding, 2021; see also how the body is and has been used across cultures in mathematics, in Kaaronen et al., 2023; also, see its perspective, in Chrisomalis, 2023). This accounts for Chomsky’s (2023) conclusion that ChatGPT cannot capture reality. And it makes satisfying Pearl’s demands more difficult in the laboratory alone.

Third, Lawless and colleagues (2023) add that team science has been hindered by relying on observing how ”independent” individuals act and communicate (i.e., using Shannon information theory, in Shannon, 1948), but independent data cannot reproduce the interdependence observed in teams (e.g., the National Academy of Sciences report in 2015 on human teams; in Cooke & Hilton, 2015).

Digital communication is based on Shannon’s (1948) information theory (e.g., entropy, channels, errors). While the information communicated between individuals works well based on Shannon, that information is factorable (i.e., i.i.d. data; see Sch¨olkopf et al., 2021); early on, it was recognized that interdependence in teams working together under laboratory conditions was not rational, leading to the recommendation that it be reduced (e.g., with information theory, in Conant, 1976; also, in social psychology, see Kenny and colleagues, 1998). Surprisingly, disembodied, factorable, and rational beliefs fail outside of the laboratory, even for simple concepts like “self-esteem” (i.e., Baumeister and colleagues, 2005; or “implicit racism,” by Blanton et al., 2009), creating the replication crisis existing in social science today (see Nosek, 2015), along with the enormous waste that has occurred in the effort to “treat” biases (e.g., see the review by Paluck and colleagues, 2021).

By being disembodied (Whang, 2023), chatbot is not connected to reality (see the New York Times, Chomsky, 2023). From the interview of Joshua Bongard, a roboticist, “The body, in a very simple way, is the foundation for intelligent and cautious action” (in Whang, 2023). Unless these activities are walled off, restricted or bounded (i.e., Simon’s (1989) bounded rationality; e.g., see how Doctor and her colleagues succeeded in the field by applying a box around a robot to reduce its degrees of freedom, in Doctor et al., 2022), these failures extend to disembodied, factorable, separable and rational beliefs, especially when confronted in the open by uncertainty or conflict (viz., Mann, 2018).

Let us draw a comparison with quantum effects. From Aspect, Clauser and Zeilinger (2022, p.1), winners of the 2022 Nobel prize in physics, “That a pure quantum state is entangled means that it is not separable . . . being separable means that the wave function ψ(x, y) can be written ψ(x)ψ(y).” From the Wikipedia article on quantum entanglement, An entangled system is defined to be one whose quantum state cannot be factored as a product of states of its local constituents.” However, these quantum effects seem to be equally valid for the interdependence among teammates. Since the contributions of individual members of a team cannot be decomposed (e.g., the National Academy of Sciences report on AI teams; see p. 11, in Endsley et al., 2021), they are not separable nor factorable but similarly dependent.

Thus, we conclude with our sketch of “a new framework” for embodied cognition (e.g., Shapiro & Spaulding, 2021) that is remarkably analogous to quantum entanglement. We know that constraints reduce information (e.g., Brillouin, 1956). By reducing the degrees of freedom among agents in a social field (i.e., Lewin, 1951), interdependence is a constraint. Interestingly, we know that open-ended “knowledge” reflects the absence of “surprise” (Conant, 1976). We also know that embodied beliefs constructed in reality work very well, with humans making rational “dynamic adjustments” to fit reality as it changes (Lucas, 1995, p. 253). Reflecting the non-factorable nature of embodied cognition, we know that teams cannot be decomposed (i.e., the NAS, in Endsley et al., 2021), forming something like a no-copy principle for constituting teams that is similar to quantum’s no-cloning principle (Wooters and Zurek, 2009, p. 77). In the open field where embodied cognition reigns, a state of maximum interdependence was found to be critical to the best performing scientific teams (i.e., Cummings, 2015); and we have found that oppressive societies significantly reduced interdependence and the freedom to pursue education and innovation (Lawless, 2022c). In every society, freedom allows a society to marshal its available free energy against the problems it has targeted (e.g., Moskowitz, 2022).

We close with the speculation that interdependence, which is embodied cognition and cannot be factored (e.g.,Endsley et al., 2021; Cooke & Hilton, 2015), is the reason why debate is central to the open and free societies that evolve compared to those societies that stagnate, regress or de-evolve (e.g., Lawless et al., 2023). Debate tests ideas and concepts, whether embodied or not; and as pointed out in this brief review, it helps to prevent or mitigate tragedies (e.g., DoD, 2021); its power arises from being the primary path to innovation (e.g., Israel is the innovation leader in Middle Eastern and North African countries; in Lawless, 2022c). For those authoritarian regimes or big businesses that censor, suppress or impede interdependence, a secondary path to innovation is the theft of the secrets or fruits of innovation (e.g., by China, in Baker, 2015; by Russia and Iran, in DoJ, 2023; by large businesses, such as Amazon, in Mattioli, 2020). But for future research, to advance autonomous human-machine teams and systems, we leave open for ourselves and others this question: how do we create a new statistics based on the reduction or absence of Shannon information that signifies a well-run team or system?

2. Summary

First, embodied cognition, like interdependence, arises as we move and interact in reality (Shapiro & Spaulding, 2021). However, it cannot be factored into its constituent parts of cognition and action (Endsley et al., 2021).

Second, the factoring of embodied cognition is prevented by a reduction in the degrees of freedom, causing a loss of Shannon information (Lawless, 2022c).

Third, Shannon’s (1948) mutual information works well to communicate among members of a team, but embodied cognition requires cognitive dissonance, debate, checks and balances, or a red team to seek cause and effect (Lawless et al., 2023).

Fourth, although interdependence cannot be factored, stress in the form of de- bate can test the strength of the claims made by embodied cognition sufficient for an audience (e.g., a jury; a voting body; a collective decision) to judge whether its claims are sufficient or adequate. Whether truth is found or not [19], debate serves as the means for a society to innovate and evolve.

This entry is adapted from the peer-reviewed paper 10.3390/informatics9020030

This entry is offline, you can click here to edit this entry!