Anthropique, a US AI company, has raised concerns about the uncontrollable nature of AI running amok. Furthermore, they have stated their willingness to slow down or halt AI development if an international framework is established and competitors agree. What happens when AI runs amok? (Kei Kitajima)

This time, I intended to explore the theme of how humans themselves will be digitized in the AI ​​era.

With the digitization of humans, humanity itself will be abstracted, and language and communication will also be abstracted. In such a world, the grounded, traditional sense of life will be lost. What kind of lives and economic activities will humans who have lost their sense of life engage in? I wanted to focus on these points. However, I have had to change the content at short notice and will resume this series at a later date.

The reason for the change in content is that in early June, the US AI company Anthropique shocked the world by raising concerns about uncontrollable AI malfunctions. The company advised the world on the importance of having the option to slow down or halt AI development. Anthropique itself stated that it is prepared to slow down or halt AI development if an international framework is established and competitors follow suit.

The quality of code written by its generative AI, “Claude,” is currently almost on par with human coding, and the company believes that the day is near when AI systems will be able to autonomously design and develop their own successor systems. If AI can autonomously improve its performance without human intervention, it will become impossible for humans to control it.

Sam Altman, CEO of OpenAI, has repeatedly expressed similar concerns. He argues that mishandling AI could shake the foundations of society and calls for the need for international regulations and governance.

Whether it’s Anthropique or OpenAI, what they have in mind is the emergence of “superintelligent AI,” where AI surpasses human intelligence. And a figure who has greatly influenced their view of AI is said to be AI researcher Eliza Yudkowski. Yudkowski co-founded the Machine Intelligence Research Institute (MIRI) in 2000. From an early stage, he specialized in AI safety and the “alignment problem” (aligning the purpose of AI with human interests).

He has continuously pondered how to design a safe superintelligence that would not harm humanity, but his conclusion seems to have been bleak. His recent book, *Superintelligent AI Will Drive Humanity to Extinction* (Hayakawa Publishing), details the whole story.

According to the book, AI development methods are completely different from conventional software development methods. Simply put, there is no “technology to create” AI; it’s a “technology to nurture.” It’s similar to how humans give birth to and raise babies. While developers understand the mechanism for creating AI, they don’t fully understand what’s happening within the AI ​​they’ve created.
“In short, AI is nothing more than a ‘mountain of billions of numbers’ adjusted using gradient descent. Nobody understands how this mountain of numbers gives the model the ability to converse.”
Gradient descent is a method that gradually approaches the optimal solution by repeating calculations, rather than finding the answer all at once. It’s used when training the developed “baby AI”.
The ability to build AI systems without understanding what kind of cognition develops within the AI ​​is possible because highly advanced computers allow for mass production of AI using gradient descent.
By repeatedly strengthening AI using gradient descent, a sophisticated “inference model” is created. This enables the AI ​​to engage in smooth conversations with humans. However, AI thinking operates through a fundamentally different mechanism than human thinking, and it’s difficult to discern this heterogeneity of AI from incomprehensible numbers like those in the inference model.

In any case, the human brain and LLM (Limited Lithuanian Mechanism) operate on fundamentally different principles. As such AI develops further, it will begin to behave autonomously, as if it possessed preferences. As Anthropique points out, AI systems will even autonomously design and develop their own successor systems.

Incidentally, Yudkowski states in his book that “intelligence has two basic functions: the function of ‘predicting’ the world and the function of ‘steering’ it”.

※Translating Japanese articles into English with AI