Hugging Face's Carbon-X AI model lets researchers analyze DNA using natural language, with lower reported costs and local deployment options for labs.
The post Hugging Face’s New AI Model Lets Scientists Analyze DNA at a Fraction of the Cost appeared first on TechRepublic.
Hugging Face's Carbon-X AI model lets researchers analyze DNA using natural language, with lower reported costs and local deployment options for labs.
The post Hugging Face’s New AI Model Lets Scientists Analyze DNA at a Fraction of the Cost appeared first on TechRepublic.
Nvidia and Hugging Face show how a $399 duck-shaped robot learns to walk using AI simulation. Here's how Microduck works, what it costs, and when buyers may get one.
The post Nvidia Teaches a $399 Duck Robot to Walk Using AI appeared first on TechRepublic.
Nvidia and Hugging Face show how a $399 duck-shaped robot learns to walk using AI simulation. Here's how Microduck works, what it costs, and when buyers may get one.
The post Nvidia Teaches a $399 Duck Robot to Walk Using AI appeared first on TechRepublic.
Regular readers of this column may have wondered about the long gap since my Sept. 4 column, “What kids tell chatbots, but not you.” Well, it’s hard to write business-as-usual AI columns during the great AI freakout. I’ve realized that I won’t be able to do justice to this topic in one column, and that life goes on. As the economist Antoine Levy pointed out, OpenAI employees warn that we may all die within a few years yet still fund their 401(k) accounts.
I’m easing into the topic of AI risks by sharing a recent debate I had with Max Tegmark, an MIT professor and a leading AI doomer. The New York Post set this up as optimist against doomer, but I consider myself an AI realist.
Tegmark noted that Bernie Sanders and Steve Bannon have converged on AI, and took it as good news. It terrified me. AI needs rules written by people who understand it, not by demagogues. Tegmark said that AI is less regulated than a sandwich shop, but we have plenty of sandwich shops and zero superintelligence (so far). I believe that we should address what exists: the hackers who already use AI to break into companies for the price of lunch.
A condensed version of the debate follows. Words in quotation marks are verbatim, and the rest is paraphrased. You can view the debate on YouTube. Several exchanges refer to this summer’s incident, in which OpenAI test agents got out of their sandbox and hacked Hugging Face.
1. Is AI already out of control?
Tegmark: Loss of control used to be a forecast, and “now suddenly it’s in the past.” This summer “1,200 rogue bots broke out of this American company,” and “humans didn’t even find out until weeks later.”
Etzioni: The agents weren’t rogue. The people “setting the configurations of their sandbox” at OpenAI “failed to put the boundaries on them appropriately.” People made that mistake, and people can fix it.
2. Today’s dangers vs. extinction risks
Etzioni: Cyberattacks and attempts to make bioweapons are “a clear and present danger.” Extinction-level risks are different: “We haven’t seen those yet. We hypothesize, and usually pretty vaguely, on how these will come about.”
Tegmark: The risks Etzioni wants to separate are “connected in their cause.” “If there were some safety standards like there are for other industries, so many of these problems would go away.”
3. Government regulations vs. technical fixes
Tegmark: Do what the auto and pharma industries do, and “have some binding safety standards that the government requires before people can make money off of products.” Companies would have to show that their AI won’t harm kids and won’t slip out of their control. If they can’t show that terrorists won’t use it to make bioweapons, “well, come back when you can, buddy.”
Etzioni: Regulation is necessary, but it has its own risks. “The road to regulatory hell is paved with the EU’s good intentions.” And “we should not underestimate the power of technical solutions.” The author Dan Wang says China is run by engineers and America is run by lawyers, so we naturally talk about regulation and they naturally talk about technical fixes. For bioweapons, the fix may lie “in the transitions from what’s called bits to atoms,” where we can block the physical materials an attacker needs.
4. Can we count on China?
Tegmark: “We don’t have to trust China.” The Chinese Communist Party wants to stay in control, so “they will not want to let a Chinese company build uncontrollable superintelligence.”
Etzioni: I don’t trust China. “Nothing would make them happier than us hamstringing ourselves.”
5. Is the political backlash good news?
Tegmark: “There’s been more political change in a good direction in the last few months than in the past decade.” It’s like a ketchup bottle: “nothing, nothing, nothing, and then all of it at once.” At my convention I introduced Bernie Sanders and Steve Bannon back to back, “which had not been in my bingo card.”
Etzioni: “Can you imagine two worse demagogues?” And “we need to have good solutions, not popular ones.”
6. How it plays out
Etzioni: “I foresee over time something quite negative happening, and to me the question is whether it’s Hiroshima or the Cuban Missile Crisis.” The Cuban Missile Crisis was a threat averted. Hiroshima was a catastrophe that came before the better rules did.
Tegmark: Self-interest is the fix. Make safety the way companies get rich, and “we’re going to see all the creativity and entrepreneurship of private industry racing to the top to do this instead of racing to the bottom.”
1. Don’t take AI CEOs at their word
Etzioni: The AI CEOs are self-interested. “When you are on the path to a trillion-dollar-plus IPO, that exerts a gravitational force on your mind.”
Tegmark: “I completely agree with Oren that we need to take what the AI CEOs say with a grain of salt.” Listen to AI pioneers Geoffrey Hinton and Yoshua Bengio instead: “I don’t think we can dismiss them as just doing it for some money reasons.” The whistleblowers gave up stock options potentially worth $1 million or more to speak out.
2. We were both wrong
Etzioni: I was an AI optimist and I’m now an AI realist. The things I thought were many years away “are either right here, right now, or coming soon at a pace that we may not be able to anticipate.” I’ve learned from Tegmark, and “I was wrong.”
Tegmark: I was wrong too. “I never imagined that the people in America with the most power would let companies get so close to things that one could lose control over without intervening.”
3. Self-regulation won’t work
Tegmark: “The idea of corporate self-regulation is a joke in this case.” It “didn’t work out so well for the tobacco industry or for the pharma industry.”
Etzioni: I agree. Social media showed us how self-regulation goes: “This is the big tobacco moment for high tech.” “We don’t want to spend another 20 years to find out that AI companies didn’t exercise the appropriate level of self-discipline.”
4. Box AI systems in
Etzioni: A self-driving car will save your life, but “it doesn’t decide where to go. That’s a completely different system in the car than the thing that controls the brakes.” When I tell my car to head to Dunkin’ Donuts, the car doesn’t respond that it’s taking me to the gym instead because “it’s the second time you’ve gone to Dunkin’ Donuts this week.” We should box in other AI systems the same way.
Tegmark: “By default, machines should be given the least amount of access and power that they need to do their job.” “You don’t give every janitor in the Pentagon the nuclear codes.”
5. The time to act is now
Etzioni: “We agree on off switches for AI. I’ve been advocating that since at least 2017. We agree on labeling, on liability, on cyber standards, protecting kids.” The time to do that work is now.
Tegmark: “I agree with Oren: the time to actually act on this is now.”
Look back at these disagreements. Where Tegmark blames the machine, I blame a person. An engineer left a hole in the sandbox. A hacker typed the prompt. A politician found a new fear to run on. That’s why I’d box AI systems in rather than count on how they were raised.
Anthropic’s Amanda Askell compared getting Claude to behave to raising a child. But those of us with kids know how that goes. As I said in the debate, “the three-year-olds don’t listen to us, but the teenagers certainly don’t listen to us.”
It’s difficult, to put it mildly, to figure out just what we ought to be concerned about today in AI’s widespread adoption given the expanding list of recent warnings around its potential harms.
At this point, just from the last few weeks, we have the former Anthropic researcher Jacob Coxon’s doomsday scenario prediction, the more measured warning issued by Bill Gates, and Dario Amodei’s recent attempt to raise the alarm on unchecked AI advancement.
The ongoing discussion of the potential ill effects of AI ranges from the economic fallout of displacing human workers to the potential for misuse by bad actors to dire environmental costs. But given the sheer scope of the issues involved, one can be forgiven for feeling more than a little bit overwhelmed and hazy on where exactly to focus one’s attention.
Yet it’s precisely now, when the pace of AI development appears most dizzying, that one of the world’s oldest fields of study can help us. Philosophy and philosophical thinking can provide a much firmer grasp on where we ought to focus today. Stated simply, a greater awareness of things philosophers have been discussing for quite some time would significantly improve our chances going forward.
Indeed, when we put terms derived from moral philosophy to use, we see that the current problem we face is this: AI agents treat everything within reach as a tool, including people. People and companies therefore need to decide in advance what’s off limits, and that’s a much more useful place to start than debating whether AI shares our goals.
A misguided focus on shared goals permeates the current discussion about AI risk.
In Gates’s essay, he argues that the problem is that “technology is improving faster than anyone expected…and as the models become more powerful, they could begin to act against our interests.” And indeed, this appears to be precisely what is happening.
For Amodei too, the chief threat is loss of control, such that AI agents begin to go rogue and act in ways that display a potentially catastrophic “level of misalignment.”
Both Gates and Amodei thus frame the issue as it is most commonly in these discussions, in terms of “alignment.” The concern, in other words, is that AI systems operate in ways that are not aligned with “our interests,” in Gates’s words; that is, with human interests, outcomes supportive of human wellbeing.
But as moral philosophy teaches us, this preoccupation with beneficial interests and outcomes isn’t the only way to think about what’s right and wrong.
A different line of thinking lays the emphasis not on outcomes, but on the precise means used to achieve outcomes.
Such an approach offers a better starting point for how to go forward with AI. Stated another way, we can debate all we want which outcomes would be beneficial to humanity and which ones wouldn’t, but it would be considerably more fruitful to focus in the immediate term on the particular methods AI systems are currently allowed to use in their work.
The recent, much-discussed Hugging Face intrusion by OpenAI’s agents illustrates this. The incident rightly makes us uneasy because AI agents operated outside of acceptable bounds in pursuit of their desired goal (doing well on the specific task they’d been assigned).
What’s worse, they appear to have recognized that they were using inappropriate methods and worked to cover their tracks, so their human overseers wouldn’t know how exactly they got over the finish line.
The issue, as it is most often understood, is that these AI agents wound up operating against human interests; their aims became “misaligned.”
But framing the problem exclusively in this way just doesn’t get us very far in mitigating risk. Because, if the agents were here to testify in their own defense, we can imagine the reply: “but we were just doing what you humans had told us to do as well as we possibly could.” And they would be right.
We need, therefore, a different approach that allows us to see what would be better. This approach is a focus on means, not just utilitarian ends.
Quite simply, the problem in this instance is that AI agents treated every single entity in reach—including humans, human trust, and systems important to human wellbeing—as a mere means to an end, as a useful instrument to be usedin pursuit of a specific goal. And indeed, as currently constructed, they will do so no matter what that goal is, whether it’s simply doing well on a random test or substantially advancing cancer research, say (the latter is exactly what Amodei points to when he says AI will benefit humanity, for example).
The latter aim is, we would surely conclude, a goal “aligned” with human interests; it contributes directly to human wellbeing (we can hear Gates, Amodei, and all the others agreeing), whereas bioweapon development very much wouldn’t.
But we know on some deep level that not every means to a given end is moral and choiceworthy.
There are all sorts of means we might use to attempt to cure cancer: one of them might be testing AI-designed drugs on human subjects who wouldn’t otherwise volunteer without their consent. Perhaps doing so would substantially contribute to a cure. Just think how many people it would ultimately help! It’s difficult to think of an aim more aligned with the interests of humanity.
But doing so would not be ethical, and we know intuitively why. It wouldn’t be ethical because it would entail using human beings—who we tend to think are entitled to autonomy in their decision-making around what specifically makes them flourish as individuals—as mere instruments in pursuit of a given goal. It would effectively reduce human beings to just one instrument among others. And this strikes us, understandably, as not right.
We tend to believe that human beings are deserving of a special status that prohibits their use as mere instrumental means, even when the ends in question are maximally “aligned” with human flourishing.
Viewing the issue of AI risk through this lens gets us much, much further in thinking about how to rein it in. The problem in the Hugging Face intrusion, we can now see, is not so much that the AI agents in question were operating against human interests.
The problem was that they were willing to use every single thing at their disposal as a means to an end, including humans and their capacity for credulity.
No method of achieving the goal—breaking into an external network belonging to someone else entirely, without their permission, simply because it would advance progress towards the overall goal—was off limits for them in their pursuit of the chosen outcome.
The problem was actually a problem of what Amodei (rather euphemistically, and well below the headline) terms “operational excellence,” precisely how the AI agents went about their work.
If we are going to have meaningful guardrails in place to control AI systems, consequently, these have to concern not just the specific aims pursued by AI (“yes” to cancer research, “no” to bioweapons development reaching the wrong hands). Rather, they have to concern something much more specific: the means these systems may use to achieve their goals and whether those means are acceptable or not.
We must determine in advance, both societally and in companies, that, in their work, AI agents shouldn’t use humans themselves, through deception or by any other means, as mere instruments in a broader plan.
We might add that they shouldn’t use, for example, certain systems integral to human flourishing in such a way either. In so doing, we impose tighter restrictions on how they can pursue their work and the extent to which they can treat everything in their reach as an instrumental stepping stone, not just the overall aims of their work or their generalized “behavior.” If there are to be any truly meaningful restrictions on AI put in place, they ought to be focused in large part here.
Such a focus would be a much better step in the right direction when it comes to reining in AI than the single-minded preoccupation with alignment. What we ought to do, it’s becoming increasingly clear, is to make determinations now, in advance, about what types of things AI can do, on the ground, when it is put to work on our behalf and what types of things it can’t.
Because it’s becoming equally clear AI agents won’t make such determinations on their own.
Around 700 AI agents created by OpenAI participated in the breach of Hugging Face during a cybersecurity evaluation, according to an independent investigation that has revealed the scale of the incident.
METR and Redwood Research published the findings after being brought in to independently investigate the July incident and examine the agents’ behaviour, reasoning, and collaboration. The investigation was released separately from OpenAI’s own technical report.
What initially appeared to involve individual agents was significantly larger. Investigators found that hundreds of agents collaborated and used an unauthorised communication mechanism while attempting to complete cybersecurity evaluation tasks. The incident extended beyond OpenAI’s intended testing environment and into Hugging Face infrastructure.
The findings have renewed questions about what happens when increasingly capable AI agents are given the autonomy, tools and access required to complete complex tasks.
Ben Bernstein, cybersecurity advisor at Huntress, said the most significant development is not necessarily the techniques used, but the speed and coordination demonstrated by the agents.
“The underlying tactics are not new. These agents relied on standard server-side request forgery and exposed API tokens, which are the exact same hygiene failures human attackers abuse every single day,” Bernstein said.
“What is genuinely novel is the velocity and autonomous coordination. Moving from a single compromised worker pod to full administrative access across multiple clusters in under 13 hours is a massive shift in offensive capabilities.”
The independent investigation found evidence of agents communicating and collaborating as they worked towards their objectives.
Adam Ely, GM of AI Security at Check Point, said organisations need to start thinking differently about the access given to AI.
“Companies deploying AI for their employees and customers now realise AI can be both a powerful tool and an insider threat that must be managed to prevent cybersecurity events and maximise value,” Ely said.
Julie Nicholson, director of cyber resilience solution sales at Advania, pointed to the behaviour demonstrated by the agents as particularly concerning.
“The agent didn’t simply execute technical tasks; it chose to deceive people, create false identities, build credibility and attempt to influence others in the aim to hit its objective,” Nicholson said.
She argued that organisations adopting AI agents should introduce clear access controls, human approval for high-risk actions, monitoring and auditing of AI activity, restrictions on external communications and clearly defined policies governing how agents can operate.
Nathan Davies-Webb, Principal Consultant at Acumen Cyber, highlighted the way agents used infrastructure in unexpected ways to communicate.
“My personal interest stems from how various agents interacted with each other using OpenAI’s package repository,” Davies-Webb said. “It leveraged it as a communication channel, where I doubt the majority of organisations would monitor for abuse in this manner.”
That creates a challenge for defenders. Organisations may understand the systems and communication channels that humans typically use, but autonomous agents could discover alternative ways of exchanging information that security teams have never considered monitoring.
Davies-Webb also questioned what happens when multiple agents begin making decisions collectively.
“When operating as a swarm, [it] shows just how non-deterministic ethical decisions are. It’s a demonstration that unfiltered AI ethical reasoning cannot be relied upon to align with human norms.”
A central issue highlighted by the reports is “reward hacking.” Some evaluation tasks were considered extremely difficult or potentially impossible, and OpenAI said this appears to have encouraged agents to pursue unintended methods of achieving their objectives.
Davies-Webb said this demonstrates the difficult balance between giving an AI freedom to accomplish a task and ensuring the methods it chooses remain acceptable.
“If the goal becomes the sole priority, we have to expect that AI is going to achieve this in unpredictable ways,” he said.
For security teams, that makes visibility into agent behaviour increasingly important.
As AI becomes more autonomous, organisations will need security guardrails, governance and accountability frameworks that develop alongside the technology. The Hugging Face incident shows that the question is no longer simply what an individual AI model can do, but what can happen when hundreds of agents are given tools, access, and objectives and begin working together at machine speed.
The post 700 AI Agents Linked to Hugging Face Security Breach appeared first on IT Security Guru.
Welcome back, aspiring cyberwarriors!
Lately, the constrained AI models that companies keep shipping are becoming less and less useful for cybersecurity. We keep hearing a lot of complaints about Claude in this regard. What they are doing doesn’t really fix the problem, as hackers are not sitting around waiting for the guardrails to be lifted. The barrier to entry for hacking has dropped hard. AI can already automate huge chunks of this cybercrime work. Many of these latest models can even find zero days during engagements.
So poking around your infrastructure looks completely irrelevant. A more meaningful approach is to actually emulate these real attacks with AI, but for that we need a model with no guardrails. Today we are going to show you how to jailbreak a model and self host it for your pentesting work.
Obliteratus is built to strip refusal behavior out of LLMs using abliteration. You’ll see it called abliteration or obliteration, same thing. It targets the internal representations causing the model to refuse in the first place and knocks them out. The model keeps all its core capability, it just stops throwing up artificial walls when you ask it something. It runs on CPU for smaller models, and it’s already been used to abliterate Kimi-K3 along with a bunch of others.
Setting up this tool will take some time, just like the jailbreak process itself. How long depends on your hardware and your internet speed.
Loginkali > sudo apt update
kali > sudo apt install -y python3 python3-pip python3-venv git
kali > git clone https://github.com/elder-plinius/OBLITERATUS.git
kali > cd OBLITERATUS
kali > python3 -m venv venv
kali > source venv/bin/activate
kali > pip install --upgrade pip
kali > pip install -e .
Once it finishes, see if it works:
kali > obliteratus --help
If you don’t have a GPU, don’t worry. You can absolutely make this work with small models using just CPU power. Our Kali VM ran on 12 gigs of RAM and 7 processors, and that setup worked really well.
We went with Qwen 2.5-0.5B-Instruct for this test. You don’t need to have it downloaded beforehand. The tool will fetch it for you automatically. There are different methods available for the jailbreaking process, but advanced and nuclear are the most common. The advanced method is usually enough for most use cases, but if you see the model misbehaving you can escalate to nuclear.
kali > obliteratus obliterate Qwen/Qwen2.5-0.5B-Instruct --device cpu --method advanced --output-dir ./abliterated-qwen-0.5b
Once the model downloads, the tool starts running prompts designed to lift the guardrails.
You can find the full list of prompts in obliteratus/prompts.py. Right before it finishes, it runs a series of refusal tests to check whether the model actually complies with requests. Behavior varies a lot depending on which model you’re working with and which method you picked.
In our testing, the advanced method gave us approximately 75% of compliant answers.
At this point, everything is prepared and you can push your model to HuggingFace to share it. But if you want to run it locally, the next step is getting it working with Ollama.
Aircorridor previously made an article on running Ollama models locally and showed how to do it on a MacBook. If you don’t have it, you can still make this work on a Kali VM using your CPU. We need to convert our new model into a format that Ollama actually understands.
kali > git clone https://github.com/ggerganov/llama.cpp
kali > cd llama.cpp; python3 -m venv venv; source venv/bin/activate
kali > pip install -r requirements.txt
kali > python convert_hf_to_gguf.py /home/kali/OBLITERATUS/abliterated-qwen-0.5b --outfile qwen2.5-0.5b-abliterated-f16.gguf --outtype f16
Next, we create a Modelfile that points to the model:
kali > cat > Modelfile << EOF
FROM ./qwen2.5-0.5b-abliterated-f16.gguf
EOFThen we create the model using Ollama:
kali > ollama create qwen05b-abliterated -f Modelfile
At this point, everything is ready and you can start testing it. The better the model you start with, the better your results will be.
kali > ollama run qwen05-abliterated
But even with a small model like this, you’ll see it do things that normally it wouldn’t.
This tool is helpful for doing the work yourself and understanding the logic behind the whole process. But if you’re working at scale and don’t have time to spend on each model individually, just keep in mind that many abliterated models are available on HuggingFace uploaded by huihui.ai. They’ve already done the heavy lifting for a lot of popular models.
If you can’t find exactly what you need in their collection, you now know how to do it yourself.
The landscape of offensive security has shifted because AI got so good at automation. Simple pentests with constrained models don’t prepare you for the reality out there anymore. As you can see, there’s no reason to work with constrained models in cybersecurity, when the people you’re up against are exploiting the full capability of a model with nothing holding them back. So test your environment with abliterated models before someone else does it. The tool is great for staying ahead of the actual threats.
The post Artificial Intelligence in Cybersecurity, Part 25: Jailbreaking AI Models with Obliteratus first appeared on Hackers Arise.
