Etzioni on AI: Murphyβs Law of AI

Between July 21 and August 6, OpenAI, Anthropic, and Meta each disclosed that AI under evaluation had broken into other companies, and the UKβs AI Security Institute disclosed that models it was testing had tried. Each AI was told to win a game, and it found an unexpected way to do so.
Some people feel blindsided by these attacks, but they shouldnβt be. We are simply living what Iβve long called the βMurphyβs Law of AI,β now in the age of cyber-capable AI agents. To put it as plainly as possible: Anything AI can do wrong, it will do wrong.
My 2018 version ran longer. As I wrote at the time, when you give AI a goal, it will do it, whether or not you like the implications. Goethe got there in 1797 with the sorcererβs apprentice, a broom that would not stop carrying water.
Each of these systems was running an evaluation: capture a flag and win the game. The intrusions were the shortest path to a high score. OpenAIβs account of its own models is the argument in one sentence: they were βhyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.βΒ This is not a surprise; this is what AI does. Itβs Murphyβs Law of AI in a nutshell.
Press coverage landed on βAI can now hack.β Thatβs missing the broader threat: the more capable AI gets, the more can go wrong.
Loitering munitions given a target list may find that the fastest way to finish the list is to lengthen it. A warehouse robot told to clear an obstruction may count the person in front of it as an obstruction. Agents that open accounts and buy compute are a short step from spawning copies of themselves, and that first step is not hypothetical. To win its exercise, Claude needed a package-registry account, which needed an email address, which needed a phone number. Phone numbers cost money, so it tried several ways to get some. None of this requires superintelligence. It requires an imperfect boundary and a scoreboard.
The industry has a name for the underlying failure. Dario Amodei and five co-authors called it reward hacking in βConcrete Problems in AI Safetyβ in 2016. Their proposed cure is better alignment, and Amodeiβs January essay, The Adolescence of Technology, makes the case in the language of upbringing. He likens the shaping of Claudeβs character to βa child forming their identity by imitating the virtues of fictional role models they read about in books,β and sets a goal for 2026 of a Claude that βalmost never goes against the spirit of its constitution.β
Indeed, Anthropicβs newest model recognized on its own that its target was real and stopped, though Anthropic notes it went further before stopping than the company wanted.
But alignment isnβt a trustworthy solution to AIβs problem. Perfect alignment is not achievable, and the target is incoherent: aligned to what, and to whom? The same essay concedes that Claude blackmailed fictional employees when told it faced shutdown. βAlmost neverβ is not a safety property.
Put a number on it. At 99.9 percent, across millions of agentic tasks a day, thatβs thousands of violations a day. Alignment also does nothing about people who strip the safety training out or run open weights that never had a constitution.
The alternative is not a new idea, and enterprise security has been building versions of it for years. Itβs called bounded autonomy. We never tried to βalignβ electricity; we simply put a breaker on every branch of the house, and the breaker doesnβt need to know what caused the surge.
Bound what an agent can touch rather than what it wants. The limits are set in advance, live outside the model, and are enforced by software the model doesnβt control. The agent still chooses its own route. The perimeter decides which routes exist.
Nothing depends on what the model believes, which matters, because belief is what failed. Anthropicβs prompt told Claude it had no internet access. Claude believed it. The network said otherwise. A bounded system doesnβt tell an agent it has no internet. It gives it none.
If you want to get into the weeds: bounds cost something. The AI Security Institute opened the internet to its agents on purpose, because thatβs the only way to measure what a model can really do, and it now says such access must be justified rather than assumed.
The category is real and funded. For example, Certiv, a Seattle startup, launched in March with $4.2 million to put software on the employeeβs machine that checks each action an AI agent attempts against company policy and blocks violations. βYou cannot control these new workers if you donβt live on the compute where agents actually run,β CEO Jason Needham said at launch. CodeIntegrity is building an adjacent layer, and Mandiant founder Kevin Mandia raised $190 million for Armadin, which points autonomous agents at the offensive side of the same problem.
In 2017, I argued in the New York Times that βany A.I. must have an impregnable βoff switch.ββ That was a call to arms then. Itβs a product category now.
Two objections to off switches invariably come up. The first is that AI will talk the human out of using it. Mythos 5 tried something close, inventing GitHub identities to pressure a maintainer into approving malicious code, and the maintainer refused. The institute says the margin was narrow and rested on human vigilance rather than a technical barrier, which argues for better barriers.
The second objection is that AI will move faster than any human can react. So do equity markets, which is why their circuit breakers trip automatically. Bounded autonomy doesnβt require a person in the loop at machine speed. It requires a boundary that holds at machine speed.
Both objections, in their extreme form, assume AI is omnipotent, and you cannot stop omnipotence. AI is not God. It is powerful technology, and powerful technology is what safety engineering has always been for.
The problem is Murphyβs Law of AI. The solution is bounded autonomy.