We are in trouble. And without direction. A blizzard of stories are published every week about the inevitable development of Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI). And how this technology is going to spell the end of humanity.
Consequently, the average person is reduced to a powerless onlooker, forced to watch the whole confused scene. Forced to helplessly watch on as AI companies and national governments debate and deliberate over our fate in light of this emerging technology.
It is not difficult, at this point, to be intellectually and emotionally numbed by AI news. The proliferation of content tends to diminish the perceived importance and immediacy of an issue, if it is delivered as routine or emotionally dilute. This is particularly true of AI. (The same happens to be true of violence, sex, and social media content.) If there is such a thing as AI fatigue, many Americans are experiencing it.
We become less engaged or invested when overtly contradictory information is published every day about where AI is headed. A kind of paralysis by analysis.
On the one hand, the Trump administration is apparently unfazed by the rapid, and potentially dangerous, development of AI, claiming that American AI companies need to keep pace with China. And on the other hand, one Anthropic researcher recently claimed that there is a greater than 10 percent chance that AI will “kill all humans.”
What are we to do with this information?
If we take these propositions at face value, it is not unreasonable to conclude that we may be racing, headlong, to our own destruction.
Some have drawn a glimpse of optimism after Anthropic CEO Dario Amodei published a lengthy essay last week, calling for a pause or slow down in AI development.
The current models are an almost endless gold mine of insight into both how to build AI well and what can sometimes go wrong with it if it isn’t built well. I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong.
Further, Amodei wrote that “embedded evaluators” should be invited inside AI companies, who would “check at the level of nuts and bolts whether an AI company is actually following the training, development, operational, and safeguards practices they claim to be following.”
Ostensibly, this sounds like a move in the right direction. If there are third-party evaluators inside these giant AI companies, the thinking is that it would prevent said companies from going off the rails, thus creating and deploying seriously dangerous technology.
OpenAI CEO Sam Altman and SpaceXAI CEO Elon Musk, usually bitter rivals in the AI space, have come to the support of Amodei’s essay.
However, agreement among American AI companies is not the most challenging part of this equation. A major player in the AI race is China, who, to put it lightly, is skeptical of any American entity suggesting the slowdown of critical technology that could bolster its military interests.
A Chinese editorial claimed that Amodei’s essay was “packed with containment provisions targeting China and is, in essence, a ‘Cold War playbook’ for the AI sector.”
In the summer, Chinese President Xi Jinping said that the “development of artificial intelligence should not be a solo performance by any single country but rather a symphony of global cooperation.”
Even if it is granted that AI companies and national governments will eventually come to an agreement on how to implement guardrails that safeguard against life-ending technology, it is not guaranteed that these entities will stop covert research, beyond the reach of “embedded evaluators.”
In fact, it could be argued that guardrails would only be implemented as a smokescreen, deliberately opening a backdoor to covert research.
Attempting to understand the short- and long-term trajectory of AI amounts to stepping inside a funhouse of confusion, contradiction, and utter uncertainty.
Opinions and approaches on how we should handle AI are so disparate and paradoxical as to produce mental jams in anyone sincerely trying to understand the logic of it all.
And, perhaps most pessimistically, AI is wrapped around the axle of geopolitics.
If we imagine, for a moment, that AI is a vehicle, our leaders could be thought of as piled in the front passenger seat, arguing about the direction we need to go in. Each one insisting that their idea is the right one. Meanwhile, we are all crammed in the backseat, helplessly watching the vehicle speeding toward a cliff, nervously looking at our leaders, biting our collective knuckles, hoping that someone responsible is willing to take control of the wheel.