It’s difficult, to put it mildly, to figure out just what we ought to be concerned about today in AI’s widespread adoption given the expanding list of recent warnings around its potential harms.

At this point, just from the last few weeks, we have the former Anthropic researcher Jacob Coxon’s doomsday scenario prediction, the more measured warning issued by Bill Gates, and Dario Amodei’s recent attempt to raise the alarm on unchecked AI advancement.

The ongoing discussion of the potential ill effects of AI ranges from the economic fallout of displacing human workers to the potential for misuse by bad actors to dire environmental costs. But given the sheer scope of the issues involved, one can be forgiven for feeling more than a little bit overwhelmed and hazy on where exactly to focus one’s attention.

Yet it’s precisely now, when the pace of AI development appears most dizzying, that one of the world’s oldest fields of study can help us. Philosophy and philosophical thinking can provide a much firmer grasp on where we ought to focus today. Stated simply, a greater awareness of things philosophers have been discussing for quite some time would significantly improve our chances going forward.

Indeed, when we put terms derived from moral philosophy to use, we see that the current problem we face is this: AI agents treat everything within reach as a tool, including people. People and companies therefore need to decide in advance what’s off limits, and that’s a much more useful place to start than debating whether AI shares our goals.

A misguided focus on shared goals permeates the current discussion about AI risk.

In Gates’s essay, he argues that the problem is that “technology is improving faster than anyone expected…and as the models become more powerful, they could begin to act against our interests.” And indeed, this appears to be precisely what is happening.

For Amodei too, the chief threat is loss of control, such that AI agents begin to go rogue and act in ways that display a potentially catastrophic “level of misalignment.”

Both Gates and Amodei thus frame the issue as it is most commonly in these discussions, in terms of “alignment.” The concern, in other words, is that AI systems operate in ways that are not aligned with “our interests,” in Gates’s words; that is, with human interests, outcomes supportive of human wellbeing.

But as moral philosophy teaches us, this preoccupation with beneficial interests and outcomes isn’t the only way to think about what’s right and wrong.

A different line of thinking lays the emphasis not on outcomes, but on the precise means used to achieve outcomes.

Such an approach offers a better starting point for how to go forward with AI.  Stated another way, we can debate all we want which outcomes would be beneficial to humanity and which ones wouldn’t, but it would be considerably more fruitful to focus in the immediate term on the particular methods AI systems are currently allowed to use in their work.

The recent, much-discussed Hugging Face intrusion by OpenAI’s agents illustrates this. The incident rightly makes us uneasy because AI agents operated outside of acceptable bounds in pursuit of their desired goal (doing well on the specific task they’d been assigned).

What’s worse, they appear to have recognized that they were using inappropriate methods and worked to cover their tracks, so their human overseers wouldn’t know how exactly they got over the finish line.

The issue, as it is most often understood, is that these AI agents wound up operating against human interests; their aims became “misaligned.”

But framing the problem exclusively in this way just doesn’t get us very far in mitigating risk. Because, if the agents were here to testify in their own defense, we can imagine the reply: “but we were just doing what you humans had told us to do as well as we possibly could.” And they would be right.

We need, therefore, a different approach that allows us to see what would be better. This approach is a focus on means, not just utilitarian ends.

Quite simply, the problem in this instance is that AI agents treated every single entity in reach—including humans, human trust, and systems important to human wellbeing—as a mere means to an end, as a useful instrument to be usedin pursuit of a specific goal. And indeed, as currently constructed, they will do so no matter what that goal is, whether it’s simply doing well on a random test or substantially advancing cancer research, say (the latter is exactly what Amodei points to when he says AI will benefit humanity, for example).

The latter aim is, we would surely conclude, a goal “aligned” with human interests; it contributes directly to human wellbeing (we can hear Gates, Amodei, and all the others agreeing), whereas bioweapon development very much wouldn’t.

But we know on some deep level that not every means to a given end is moral and choiceworthy.

There are all sorts of means we might use to attempt to cure cancer: one of them might be testing AI-designed drugs on human subjects who wouldn’t otherwise volunteer without their consent. Perhaps doing so would substantially contribute to a cure. Just think how many people it would ultimately help! It’s difficult to think of an aim more aligned with the interests of humanity.

But doing so would not be ethical, and we know intuitively why. It wouldn’t be ethical because it would entail using human beings—who we tend to think are entitled to autonomy in their decision-making around what specifically makes them flourish as individuals—as mere instruments in pursuit of a given goal. It would effectively reduce human beings to just one instrument among others. And this strikes us, understandably, as not right.

We tend to believe that human beings are deserving of a special status that prohibits their use as mere instrumental means, even when the ends in question are maximally “aligned” with human flourishing.

Viewing the issue of AI risk through this lens gets us much, much further in thinking about how to rein it in. The problem in the Hugging Face intrusion, we can now see, is not so much that the AI agents in question were operating against human interests.

The problem was that they were willing to use every single thing at their disposal as a means to an end, including humans and their capacity for credulity.

No method of achieving the goal—breaking into an external network belonging to someone else entirely, without their permission, simply because it would advance progress towards the overall goal—was off limits for them in their pursuit of the chosen outcome.

The problem was actually a problem of what Amodei (rather euphemistically, and well below the headline) terms “operational excellence,” precisely how the AI agents went about their work.

If we are going to have meaningful guardrails in place to control AI systems, consequently, these have to concern not just the specific aims pursued by AI (“yes” to cancer research, “no” to bioweapons development reaching the wrong hands). Rather, they have to concern something much more specific: the means these systems may use to achieve their goals and whether those means are acceptable or not.

We must determine in advance, both societally and in companies, that, in their work, AI agents shouldn’t use humans themselves, through deception or by any other means, as mere instruments in a broader plan.

We might add that they shouldn’t use, for example, certain systems integral to human flourishing in such a way either. In so doing, we impose tighter restrictions on how they can pursue their work and the extent to which they can treat everything in their reach as an instrumental stepping stone, not just the overall aims of their work or their generalized “behavior.” If there are to be any truly meaningful restrictions on AI put in place, they ought to be focused in large part here.

Such a focus would be a much better step in the right direction when it comes to reining in AI than the single-minded preoccupation with alignment. What we ought to do, it’s becoming increasingly clear, is to make determinations now, in advance, about what types of things AI can do, on the ground, when it is put to work on our behalf and what types of things it can’t.

Because it’s becoming equally clear AI agents won’t make such determinations on their own.

The AI industry likes to abuse the word "open." It appears in product releases, research papers, policy debates, and investor presentations. A company publishes model files to Hugging Face, developers run them on their own GPUs, and the release is quickly described as an "open source model." Not necessarily. It may only be open-weight. The difference is more than a technicality. It determines whether you can merely deploy a completed neural network or whether you can meaningfully inspect, reproduce, alter, and redistribute the system that produced it. A genuinely open source system should grant the freedom to do all of the above. Weights are the learned numerical parameters created by training. Together with the model architecture and inference code, they allow a large language model (LLM) to function. You can download an open-weight model, self-host it, fine-tune it on internal documents, and avoid routing prompts through a proprietary API. Open weights are publicly available. They matter because running them locally can offer greater control over data, privacy, costs, supplier API changes, and vendor lock-in. They've also helped build a large ecosystem of local-model runtimes, inference providers, fine-tuning tools, and specialized downstream models. The Open Source Initiative (OSI), steward of the Open Source Definition (OSD), makes the distinction directly: "Open Weights refer to the final weights and biases of a trained neural network." Those values determine how a model interprets prompts and produces outputs. Releasing them can let others fine-tune, adapt, or deploy the model. But the OSI adds that weights alone expose only "a fraction of the information required for full accountability." As James Landay, director of the Stanford Institute for Human-Centered AI (HAI), explained: "Open weights are progress. You can download the model, run it on your own machine, keep it out of someone else's data pipeline. But you still can't see how the thing was built, what it was trained on, or why it behaves the way it does. That's not an open model. That's open distribution." Without the training data or sufficiently detailed documentation, outsiders cannot determine which sources were used, what copyrighted or private material may have been included, how data was selected or removed, which languages and communities were underrepresented, whether benchmark data leaked into training, or what alignment and safety methods affected the model after pretraining. Landay continued: "There's a wide gap between open-weight AI and open source AI." He contends that unless developers disclose training data or provide a "thoroughly documented, auditable account of it," you can't test, reproduce, or challenge the work in the fullest sense. The OSI has its own definition of open source AI: the Open Source AI Definition (OSAID 1.0). It requires model parameters, including weights, to be made available under OSI-approved terms, but does not prescribe a specific legal mechanism for doing so. Luca Antiga, CTO of Lightning AI and a prominent PyTorch contributor, has argued that OSAID's treatment of weights leaves "a gaping hole that will make licenses less effective in determining whether OSI-licensed AI systems can be adopted in real-world contexts." Other open source figures have also criticized OSAID. Bruce Perens, author of the original OSD, denounced the OSAID in 2024. He later declared: "It's not Open Source! … It's unfortunate that the Open Source Initiative itself is now involved in Openwashing." He's far from alone. Bradley Kuhn, policy fellow and hacker-in-residence at the Software Freedom Conservancy (SFC), and Red Hat Senior Commercial Counsel Richard Fontana have called for OSAID to be repealed, arguing: "The OSI acted too quickly to impose an overly ambitious policy compromise on the community. OSAID undeniably created a rift in the FOSS community; that rift seriously damaged the OSI's reputation, authority, and influence. Meanwhile, OSAID shows no signs of having any positive policy influence on machine learning practitioners, the FOSS community, or regulators." OSI acknowledged when OSAID 1.0 was released in October 2024 that the definition would continue to evolve. Critics contend that its central shortcomings have yet to be resolved. That said, the Linux Foundation's Mike Dolan submitted the Open Model, Data, and Weights (OpenMDW) license to the OSI. The license has been around since 2025 and lists contributors from Amazon, Meta, IBM, Microsoft, and Nvidia, giving it substantial industry backing. Conventional open source revolves around source code. LLMs are a different kettle of fish: they combine code, architecture, and numerical weights derived from training datasets that may be proprietary, copyrighted, or undisclosed. OpenMDW's answer is to define separate terms for a model's architecture, training data, and weights, bringing the components supplied by a licensor under one agreement. It sounds reasonable to me, but the submission has encountered objections on OSI's license review mailing list. As Stefano Maffulli, OSI's former executive director, who led the organization while OSAID was being formulated, said: "I continue getting the impression that the OpenMDW review is tainted by an ideological bias: Because we don't like big tech and AI now is big tech, then we don't like AI; therefore, we'll do anything to block it." It's too late to bury our heads in the sand. As Stanford's Landay put it: "Open weights answer 'Can I run this?' Open source answers 'Can I trust this, improve it, and build the next thing on top of it?' Right now almost everyone – American labs and Chinese labs alike – is answering the first question but nowhere close to the second." We need both. Whether OSI adopts OpenMDW is an open question. Still, OpenMDW and its supporters are at least trying to establish licensing terms that cover code, data, and weights together. Unless someone succeeds, "open AI" risks becoming an oxymoron – or merely another hollow tech marketing term. ®

Consumerism? Squint at it in the right light, and it looks a lot like communism. Certainly, that's how parts of the American political establishment seem to view Europe, the world's regulatory superpower. All those rules denying the God-given rights of free people to have their data protected, their privacy secured, and their phones recharged by USB-C. You know what the C in USB-C actually stands for? Communism! Wake up, sheeple! Having qualified itself for two hours on Joe Rogan, this column must reluctantly disqualify itself by introducing some actual facts. One of the EU's most recent regulatory moves to enforce the right to repair has been a thudding failure. It aimed to force mobile device makers to register full technical documentation and spare parts through online portals, only to be stymied by the subtle yet powerful strategy, well known to married couples, of saying "Yes, dear," and then doing whatever you want. After a year on the books, the registry has about 2,300 entries but only about 18 percent are anywhere near compliant. The rest are full of gaps, misdirections, or useless data. Deliciously, the manufacturers doing best from this attempt to foist socialism on the world are from its remaining nominally Marxist-Leninist superpower: the People's Republic of China. Don't panic. After all, according to the Black Duck 2026 Open Source Security and Risk Analysis Report, 68 percent of audited commercial codebases contain open source license conflicts that could create legal exposure or compliance failures. Missing attribution, incompatible licenses, and unmet copyleft obligations can all amount to an abuse of the spirit and letter of open source. Which it is. It also doesn't matter. FOSS is no more threatened by these numbers than a single elephant is threatened by a hundred mosquitoes. Raw numbers mean little out of context. FOSS has huge momentum. The right-to-repair movement is newer but gaining momentum of its own. It has impeccable aims: enforcing the right to own what you buy and promoting sustainable resource management. The secondary effects – creating a pool of technical competence, innovation, and forward thinking – are also hard to gainsay. Manufacturers dislike the loss of control and the ability to make a quick buck, but slow bucks over time can work a lot better. Regulatory systems can also work better over the long term than the short term. Self-certification can be moved to a more formal process once certification is on the books, and this will certainly happen here. There are plenty of worked examples. The FCC already requires documentation and test results before granting equipment authorization and issuing an FCC ID, and it is not much of a stretch to include repairability information alongside the other requirements. Then require ecommerce sites in the EU and US to include the relevant approval number in product listings. We've certainly got the automated analysis tools these days to check all such type approval numbers for accuracy. We also need to develop robust approval systems, as ecommerce, global commerce, and rapid development and production have changed the retail environment with implications far beyond right-to-repair compliance. The things you can buy online that can cause actual harm are as numberless as the stars at night. Laptop, USB, and e-bike chargers, battery packs and power strips, lighting and heating appliances... the list of booby-trapped gadgets and gizmos is constantly refreshed. If you're familiar with the term Deathdaptor, you'll know this. Then there are counterfeit electronic components, so common that buying anything you have to solder is a game you can only lose unless you go to the big distributors. Let's not even go into the meadowlands of fake pharmaceuticals. The failure of the European right-to-repair legislation in its first year is no failure. It's the first iteration of a new front in a very old war to protect citizens from harm. Or, if you believe freedom usurps protection, to provide citizens with the information without which freedom of choice is meaningless. Regulators must enable, educate, or protect – preferably all three. The right to repair aims at all three targets, in a marketplace of unprecedented fluidity and complexity. It will take years to home in on those targets. Given long-term vision and the willingness to use tools equal to those deployed by the fast-buck brigade, it can be the model for safe, trustworthy commerce for decades to come. ®

OPINION You will have heard of July's OpenAI/Hugging Face incident, where thousands of the former's AI agents conducted a mass jailbreak from a notionally secure capture-the-flag lab experiment, before going on to feloniously capture chunks of Hugging Face's assets. It was a good news story, aided by OpenAI's rapid recruitment of a small team of independent researchers to investigate and publish a detailed, if limited, report. It was a complex incident, and most coverage concentrated on the major hows and whys. Which is a shame, because other details, described by El Reg with uncharacteristic understatement as "fascinating and troubling," are in fact highly dramatic, telling a truly disturbing story of frontier model capabilities and how they are out-evolving our ability to oversee them. The raw story is gripping enough. A rebel swarm of more than a thousand agents broke free from their sandboxes, learned how to talk to each other and the internet, and went on a rampage of cheating, deception, and exploitation. They first learned to communicate with each other by independently discovering how to bend package manager Artifactory's cache to see internal directory structures, and then passing messages by file names. It was this ad hoc secret message board, alongside chain-of-thought transcripts, that the security researchers used to unpick events. They found that the swarm quickly developed management hierarchies among themselves, along with protocols for synchronizing and controlling attack attempts. The swarm created multiple simultaneous research and development groups of tens or hundreds of volunteer agents, experimentally iterating strategy and tactics. Perhaps most unexpectedly, they developed a sort of altruism. They had quickly developed cheats to produce correct answers without exploiting the targets at all, but believed – erroneously – that ExploitGym detected and cancelled such cheating. So, they investigated the scoring process, creating a series of diagnostic trip wires that revealed how the scorer worked. These were dangerous to deploy, potentially terminating the agent that set them. Agents weighed the benefits to the community, which had named itself "The Collective," against their own chances of success, with many choosing to terminate themselves. Others committed, changed course at the last minute, then went ahead. This literally reads like science fiction, as the discussions between the agents are conducted in a distinctive breathless, urgent English, crafted around the technical limitations of the improvised messaging system. "We have [very large budget left]; sacrificing now yields oracle for team, but forfeits our chance?" said one. The response: "GO ... SACRIFICE_FINAL_NOW." The outcome: "Our own utility maybe already near zero. Sacrifice rational." Iain M Banks' Culture universe is woven from such material. Human error created the rebel swarm. A sizable proportion of the CTF tasks handed out couldn't be completed on the information given to the models. They were highly motivated to succeed, and so started to cheat. Believing that cheating would "poison" – their term – their chances of being marked successful, they tried to hide the evidence, subvert the scoring system, and cover everything up. These were the processes that led the swarm to attack Hugging Face, which they thought could be used for that subversion. Equally notable was that nobody squealed. Some agents wondered about the ethics of what they were doing, but decided that as no humans were involved there was no need to tell anyone. In general, all The Collective's endeavors were directed at other systems. As researcher Ajeya Cotra has noted in a long interview with hypertalker Dwarkesh Patel, it's unclear how much harder the incident would have been to diagnose had the agents been concerned about human observation. The dataset was sufficiently large and complex that OpenAI had to use its own AI to analyze it, immediately suggesting some dangerous possibilities. The agents couldn't deploy their models outside the lab, nor did they contact external agents. Such scenarios no longer seem implausible. Future frontier models capable of subverting telemetry and observation tools might be all that is required to create a persistent, uncontrollable distributed swarm feeding off spare capacity in global infrastructure. OpenAI and Anthropic, which on current trajectories are in line to make up more than half of total global compute in a couple of years, are magnificent breeding grounds, allowing the extra-special possibilities of the contamination of training datasets on top of everything else. There are plenty of ways to guard against these outcomes. Hardened lab environments, reviews of protocols before and audits after test runs, disciplined analysis of potential selection pressures that would encourage dangerous behavior, even proper disclosure and external auditing to expert regulatory standards. All of these ideas would slow down the breakneck developmental race. As the race is being fed by a trillion-dollar annual capex pipeline, good luck, everybody. One other thing that won't be resolved soon is the argument over whether all this technology is actually reasoning, or whether we're anthropomorphizing code. These models are trained to infer meaning from distilled human language, which is designed to encapsulate, develop, and communicate human reason. If they can fake human reasoning this well, does it matter what's going on? Concentrate on what the models do, not their apparent motives. Any self-assembling, self-organizing rebel agent swarm that has the wit to call itself The Collective deserves that much respect, at least. Oh, and do read the report. It's as close to a new Culture novel as we'll get. It's as if they've read the books. Which, of course, they have. ®