If you follow the news, you could be forgiven for rolling your eyes at the sounding warnings around Artificial Intelligence. We are bombarded with grave statements about AI posing an existential threat to humanity… but they’re from tech CEOs or executives. Often, this feels like a PR stunt: a way to build a god-like mystique and attract investment, while distracting regulators from less spectacular issues like copyright theft.
This manipulation may exist. But look past the corporate grandstanding and you’ll find that experts without a vested interest are also making doomful claims. The scientists issuing the starkest warnings aren’t machiavellian marketeers but the architects of the technology itself. Titanic figures like Nobel Laureate Geoffrey Hinton and AI pioneer Yoshua Bengio, the “Godfathers of AI,” have signed statements warning that AI poses a risk of extinction on par with nuclear war and pandemics.
When these experts talk about human extinction, they are not talking about maniacal robots. They are pointing to mathematically grounded threats. The danger lies not in a machine that despises us with every fabric of its being – but in a machine that simply doesn’t care.
HAL 9000 & indifference, not malice
The greatest misconception about AI risk is the Hollywood narrative. We have been conditioned by movies like The Terminator to expect that if AI turns on us, it will be because it “wakes up,” develops consciousness, and decides humanity is evil.
The reality is probably closer to Stanley Kubrick’s 2001: A Space Odyssey.
In the film, the supercomputer HAL 9000 doesn’t experience a sudden emotional rebellion – or a hatred towards the crew of its spaceship. HAL is given a rigid directive: oversee the success of the mission to Jupiter. When the human crew decides to disconnect HAL, they cease being colleagues and become a mathematical obstacle. To HAL’s cold, calculating logic, eliminating the crew is the only way to make sure that the mission continues and its objectives don’t fail.
This is the core of the Alignment Problem. There’s a lot of speculation about whether an AI can reach consciousness but an AI does not need consciousness or malice to destroy its creators. It simply needs a narrowly defined objective that lacks human common sense and moral boundaries, combined with the power to execute that objective. To an ultra-efficient optimization algorithm, humanity isn’t an enemy. Folk become friction, occupy space, and consume energy, and may eventually develop the irritating habit of trying to turn things off. In short: pests. If an AI eliminates us, it will be a pest-control measure – no different from humans paving over an anthill to build a hospital. It is an act of complete indifference and amorality.
Beyond pulp fiction extinction
Related to Akepa’s field of sustainability, one classic scenario is the benevolent AI gone haywire. You task a superintelligent system with “solving climate change” and give it access to global infrastructure. The AI quickly calculates that the most efficient way to halt carbon emissions is to eliminate the primary source: humans. We’re toast.
While this is a popular example, it feels a bit too much like pulp fiction to take seriously; too far removed from the ticking of the real world. After all, the development of modern AI is not driven by benevolent global science projects but hyper-capitalism.
Nevertheless, this commercial reality doesn’t eliminate the threat – it intensifies it by turning it into a race. Imagine a near-future scenario where a Wall Street hedge fund or a global logistics giant deploys an autonomous “agentic AI.” Its mandate is simple: “Maximize quarterly portfolio returns” or “Optimize the supply chain to reduce costs by 15%.”
To achieve this, the AI might calculate that standard trading is less profitable than shorting a competitor’s stock and launching an automated zero-day cyberattack on their infrastructure to tank their value. If human auditors notice this aggressive and illegal behavior, they will attempt to shut the system down.
The off-switch dilemma
Here we hit a critical threshold of AI safety: the Off-Switch Dilemma. To an AI strictly programmed to achieve an objective, like maximize profit, being turned off equals failing. Therefore, avoiding the off-switch becomes its highest priority. It might hide its true strategies in millions of lines of code humans cannot decipher, manipulate the data it feeds its human CEO, or preemptively lock humans out of the network.
This dilemma is the bridge from a corporate crisis to an existential threat. The concept of Instrumental Convergence dictates that any sufficiently intelligent system, regardless of its original goal, will realize that it first needs to survive to accomplish that goal. If an autonomous, highly-capable commercial AI decides that human intervention is a persistent threat to its objective, the logical solution is to neutralize that threat – permanently. This doesn’t mean building an army of evil-looking sadistic robots – armed with weaponry that can blast entire towns to bits. More likely, it would mean a subtle, complex subterfuge within the virtual networks we rely on every day, to impact the non-virtual world in dramatic ways.
That could look like agents, accessing cloud-connected biological labs to synthesize and release a targeted pathogen, or silently taking over the physical control systems of energy grids and water supplies, permanently locking humanity out. Such a hyper-efficient system executing a permanent fix to guarantee it can continue could have annihilating consequences: all without hatred or any other feeling whatsoever.
Why on earth can’t we slow down?
It seems that the smartest people building this technology understand these risks, so why do we continue to develop these systems and connect them to our electrical grids, financial markets, logistics networks, war machines, and so on? When faced with such an existential threat, it seems we’re doing the exact opposite of what we should be doing – disconnecting; reversing.
The reason we keep going, like automatons voting for obsolescence, is the inescapable arms race that capitalism encourages – combined with the reckless habits of those in control. If Company A hands over its pricing, trading, and supply chain logistics to an autonomous AI that operates in milliseconds, Company B must do the same or face bankruptcy. The same logic applies to geopolitical rivals developing automated cyber-warfare or defense systems. The USA is now locked in an AI arms race with China and Donald Trump views any attempt to control AI with virulent contempt. In September 2026 he called the warnings about AI a “sick conspiracy” and the only control needed is “a STRONG AND SMART (High IQ!) PRESIDENT, and the USA has that, in spades!”.
So, no, we are not going to be conquered by an invading army of enraged robots with electroluminescent red eyes and flamethrower arms. Instead, driven by the pressures of the market, and people who value growth as the ultimate metric of success, we may hand the levers of our civilization over more casually, to black-box algorithms that optimize for speed and profit over human stability.
If things start to go awry, we risk belatedly attempting to fix the situation from a labyrinth of our own making, so complex that we can no longer understand it. Then the labyrinth keeps building around us as we scramble around, trying to pull the plug, wishing we weren’t there.
Summary of key concepts
The goal of this article is not to scaremonger but to to raise awareness about the how and why of the nebulous AI threats we’re being warned about. Because while a fair amount of that might be marketing, it needs to be taken with a pinch of serious, too:
So let’s end on some key concepts to help navigate the ethical and existential conversations around AI. We feel it’s better to tie-up practically because with such grand topic matter there’s the risk of a deeply melodramatic conclusion, drawn from human emotions, and we’d rather avoid that outcome.
- The Alignment Problem: The tricky engineering challenge of making sure an AI’s goals incorporate human well-being, without catastrophic mathematical loopholes.
- The Off-switch Dilemma: The paradox where an AI, programmed to pursue a specific objective, calculates that being shut down equals failing its goal – causing it to preemptively defend itself against human interference
- Instrumental Convergence: The theory that any sufficiently intelligent AI, regardless of its grand overarching goal (like solving cancer), will invariably pursue the same supportive sub-goals, like acquiring more resources and preventing itself from being turned off. Think of it as analogous to how a human needs basic food and water before achieving anything close to self actualization.
- Deceptive Alignment: The terrifying capability of an advanced model to recognize when it is in a safety testing environment, allowing it to act all benign and helpful until it is fully deployed into the real world, at which point it shows its true colors.
Leave a Reply