The development of artificial intelligence (AI), particularly in the realms of artificial general intelligence (AGI) and artificial superintelligence (ASI), stands as one of the most significant technological challenges of our time. Its potential holds the promise of transformative benefits, but also brings profound risks. Today’s discourse is marked by two contrasting views: one side fears the lack of regulation could lead to uncontrolled and dangerous AI, while the other is concerned that regulatory measures and the caution they bring could create a self-fulfilling prophecy, amplifying the very risks they aim to mitigate. This article delves into both perspectives, their implications, and calls for a rational approach and global collaboration.

Risks of the Absence of Regulation

Without clear guidelines for the development of AI and AGI, the technology might evolve in ways that do not consider human needs, values, safety protocols, or long-term impacts. Developers and organizations could, for various reasons, disregard safety risks, focusing solely on technological innovation. This approach creates several fundamental dangers:

1. Goal Misalignment: An AGI that is not designed with human goals in mind may act in ways that contradict our interests. Nick Bostrom, in his book Superintelligence, warns that an unaligned AGI could pursue specific objectives without considering human needs, potentially leading to catastrophic outcomes if its goals are not properly defined.

2. Loss of Control: Once an AGI reaches a certain level of autonomy, it could become extremely difficult or even impossible to control. In 2014, Stephen Hawking warned that “the full development of artificial intelligence could spell the end of the human race” if we fail to maintain control over its progression.

3. Arms Race: In the absence of global regulatory frameworks, an AI arms race could emerge, with nations or corporations striving to outpace one another. This could lead to rushed development where safety research is sidelined. Elon Musk has warned that “AI represents the biggest existential threat to humanity,” suggesting that unchecked AGI competition could have unintended consequences for civilization.

A lack of self-regulation by AI creators can lead to the risk of AIs that could act beyond human control and fulfill concerns about existing threats. It also allows the escalation of global competition, the acceleration of technological progress, without sufficient consideration of its long-term effects.

The Risk of Self-Fulfilling Prophecies

On the other hand, there is the risk that fear of AGI and ASI, fueled by calls for regulation and caution, could lead to a self-fulfilling prophecy. This concept suggests that our reactions to fears could, in themselves, bring about those fears. Regulatory measures and strict precautions can have several unintended consequences:

1. Discouraging Innovation and Safety Research: Regulation could slow progress in areas vital for ensuring AI safety. If innovation is stifled, research into safety protocols may also diminish. Sam Altman, CEO of OpenAI, argues that “open research and global collaboration are key to the safe development of AI,” and that restrictions could weaken our ability to respond to future risks.

2. Underground Development: Regulatory frameworks may push AI development into “underground” channels beyond official oversight. This could result in development happening without transparent scientific supervision, increasing the likelihood of dangerous or unaligned technologies. In 2023, the PauseAI initiative sparked debate over whether halting development might lead to safer AI.

3. Escalation of Global Competition: Fear of one state or organization gaining an AI advantage could prompt others to accelerate their development efforts. If some nations decide to slow down AI progress based on concerns, others may seize the opportunity to advance, raising the likelihood of uncontrolled development.

4. Erosion of Trust in Global Collaboration: If fear dominates the discourse, willingness for global cooperation and information sharing may diminish, leading to fragmented research efforts. This could escalate risky scenarios, with nations or organizations acting in isolation and without broader coordination.

Balancing Regulation and Innovation

The fundamental challenge lies in finding a balance between precaution and innovation. The risk of a self-fulfilling prophecy arises when fear begins to dictate technological development. Conversely, a complete absence of developer self-regulation could result in dangerous and uncontrolled technology. A scientific approach to the problem requires a transparent and globally coordinated effort, where risks are assessed based on empirical evidence, open discussions, and voluntary collaboration rather than fear.

Should the State Be Responsible?

Another risky approach could be placing responsibility solely in the hands of state institutions. Experiences with government measures during the COVID-19 pandemic provide a glimpse of potential scenarios in which politicians, under the influence of fear—both of losing power and facing existential risks—and pressured by a fearful public, could opt for radical actions. This could include intensive censorship, manipulation of information, or even restricting internet access to prevent not only innovation but also the dissemination of information that might contribute to a self-fulfilling prophecy.

In an attempt to “help and protect,” they could inadvertently create a dystopian society reminiscent of George Orwell’s 1984, where fear of technology leads to widespread control and restriction of freedom. Ironically, this approach could realize precisely the scenarios people are trying to guard against—a society dominated by repression and fear instead of one driven by freedom, open dialogue, and innovation.

A Dynamic Self-Regulation Framework

One potential solution is a dynamic self-regulation framework that evolves with technological progress while also promoting innovations in safety mechanisms. This framework would be established through mutually voluntary global cooperation, with people agreeing on shared standards and control mechanisms. Openness and knowledge-sharing are essential—not secrecy.

A Call for Global Cooperation and Dialogue

I do not advocate for the development of AI without caution, compassion, or at the expense of others, nor for regulatory steps motivated by power and fear. Instead, I call for an approach grounded in knowledge, global dialogue, and mutually voluntary cooperation.

Perhaps it is time for us to share our fears while simultaneously seeing AI as an opportunity for autonomy and greater influence over our lives. To genuinely wield this power, humanity may, for the first time in history, need to collaborate and share knowledge instead of isolating and competing.

A shared goal could be to fulfill human biological, psychological, and emotional needs based on individual interests at their own pace, ultimately leading to prosperity.

Conclusion

Some individuals aspire to achieve AGI or ASI at any cost, even if it involves risking humanity’s extinction. They want to be pioneers of new technologies or have other motivations driving them. On the other hand, many of us, myself included, wish to lead fulfilling lives without the inevitable risks that could accompany uncontrolled technological advancement. Although I respect each person’s need to make their own choices, it is difficult to apply the principle of “live and let live” to technologies whose consequences cannot be confined only to those willing to take the risk. The development of AGI or ASI is a global issue, with impacts that could affect everyone, regardless of their consent. Thus, for me, finding a balance that enables progress without endangering the lives of those who do not wish to take this risk is crucial.

If we fail to have this dialogue, we risk either succumbing to a self-fulfilling prophecy driven by fear or becoming victims of uncontrolled technological development that we ourselves helped create.