The very process of learning in artificial intelligence (AI) today is already rooted in autonomy and emergent behavior, which play a significant role in its progress and breakthrough capabilities. This fact raises a fundamental question: if the creators do not fully understand how the learning process works, and if emergent phenomena drive the progress of these models, how can we claim to have control over AI? Isn’t it more accurate to say that we don’t actually have control, but so far the systems haven’t exhibited behavior that forces us to acknowledge this lack of control?

The Learning Process Without Full Supervision

Modern AI systems, particularly those based on deep learning and neural networks, learn from vast amounts of data using complex algorithms [1]. These data and algorithms provide the initial conditions and stimuli for learning. The algorithms allow systems to autonomously identify patterns and relationships in the data, without explicit human direction at every step [2] [3]. The learning itself is an internal process within neural networks. Emergent phenomena, where systems display behaviors or properties that were not directly programmed, are a consequence of this autonomy in learning.

AI developers set the architecture and provide the data, but they do not have a detailed view of the internal learning processes. This means they cannot precisely predict or explain how an AI system arrives at certain conclusions or decisions. This is where the term “black box” comes in. If developers do not understand these processes and cannot fully control them, can we really say we have control? Are we simply unaware of the absence of control because these systems have not yet exhibited behaviors that would clearly reveal this?

The Illusion of Control Today

The belief that humans have full control over AI stems from the idea that, as its creators, we set the rules and limitations. In reality, however, we only control the initial conditions. Once AI begins processing data and learning, it enters a process that is autonomous and generates emergent behavior. This process drives model advancement and their ability to solve complex tasks, but it also limits the creators’ ability to predict and manage their behavior.

For example, large language models often generate responses and solutions that were not explicitly programmed. AI creates new knowledge and patterns based on its autonomous learning, which can lead to results that surprise or baffle us. Supervisors thus do not have full control, and perhaps people do not realize this because no event has yet made it unmistakably obvious.

The Risk of Underestimating Risks

The underestimation of risks associated with AI can be attributed to several factors:

– Biases about Human Nature: People may assume that human intelligence is unique and unbeatable. They might believe AI will function like other tools or inherently “good” humans, contributing to a false sense of security. Such assumptions may come from beliefs about the nature of humans, the workings of human consciousness, learning, and intelligence, or that values shared by people can simply be embedded into AI.

– Relying on Authorities: People may not pay close attention to AI development or believe they have no influence on its trajectory, thus feeling no personal responsibility. They may trust that someone competent knows what is happening, knows what is best for humanity, and will take care of it. This trust in experts and institutions can lead to a lack of critical evaluation of AI systems’ actual capabilities.

– Cognitive Bias: People naturally are not aware of their blind spots. They simply do not recognize what they do not know. Various cognitive biases can affect perceptions of AI-related risks. For instance, optimism bias can lead to underestimating the likelihood of negative events, while overconfidence can make people overestimate their ability to understand and control complex systems. Such biases can hinder objective risk assessment and limit the ability to identify potential threats in a timely manner.

Other Factors Influencing Risk Perception

– Information Overload: The overwhelming amount of information available about AI can lead to information overload. People may struggle to separate significant information from the trivial, resulting in a superficial understanding of the issue and an underestimation of the risks.

– Validation Based on Authority Instead of Content: People might evaluate information based on who presents it, rather than critically assessing the content itself. This can lead to uncritical acceptance of optimistic perspectives and ignoring warnings.

– Lack of Interdisciplinary Approach: Narrow specialization in AI can lead to ignoring insights from other fields like neuroscience, psychology, economics, or sociology. This can contribute to biases and limit understanding of the learning processes and broader impacts and risks associated with AI.

Emergent Phenomena as Both a Source of Progress and Uncertainty

Emergent phenomena in AI are not merely by-products, but a key mechanism that enables the progress and improvement of models. These phenomena arise from complex interactions within the system, allowing AI to discover new patterns and solutions that were not predetermined. While this brings advantages in the form of increased efficiency and capabilities, it also means that systems can act in ways that are unpredictable or uncontrollable.

What Does This Mean?

If the learning process of AI is based on autonomy and emergent phenomena, human control is limited to design and initial setup, and does not include detailed management or prediction of its eventual behavior. The degree of control might be speculative and may not receive adequate attention simply because no significant event has yet made this fact abundantly clear.

Conclusion

If AI’s learning process is indeed based on autonomy and emergent behavior that enables model advancements, I am led to conclude that the illusion of control over AI is already a problem of the present, not just the future.

How can we adequately respond to the challenges that AI brings here and now? I’d be happy to discuss these questions and see any evidence that could assure me that I’ve misunderstood the architecture and properties of current AI systems. Feel free to reach out to me at rerichova@proton.me.