A modern organization protects its information systems with a whole suite of technologies that generate thousands of security alerts every day. Looking at these alerts in bulk, you’ll see that a significant share are false positives. Ideally, these alerts would be filtered out automatically so that analysts could focus on the ones that may point to a real incident. However, writing filters is a time-consuming task in itself. To create a correct filter, an analyst needs to:

• Review the logs
• Identify the source of the activity
• Define filtering boundaries
• Choose filtering criteria
• Write a rule
• Check that the filter is robust

This post looks at how filter creation itself can be automated to reduce analyst workload.

Filter validity criteria

In the context of security alerts, a filter is a combination of event field values that marks a detection as a false positive and not worth the analyst’s attention. Filters can be generated for any combination of event fields, but not every combination yields a valid filter. That’s why we need to define validity criteria: conditions showing that a specific filter actually removes noise and doesn’t suppress useful alerts.

Detection rules in security systems typically contain a set of conditions, and each of them can generate an alert. A specific event triggers a given fragment of a detection rule. The set of events that a given fragment responds to is what we’ll call its context. A good filter should cover the entire context where possible, without going beyond it. Filters that are too broad or too narrow are useless. For example, a filter based solely on PowerShell would exclude too many events, including some that deserve the analyst’s attention. Most of the filtered events would also be unrelated to the rule in question. A filter on a specific ID, on the other hand, would cover only one event, which may never recur.

Problems in filter generation

We can run into a number of problems when creating filters. First, events often contain an excessive number of fields (around 400 on average), and most of them aren’t needed for filtering. At the same time, generating and ranking combinations of these fields is also fairly resource-intensive. So, the first step is to discard the irrelevant fields and keep only those worth using in the filter. For example, out of 400 fields in an event, four may be enough for filtering.

Another problem is string normalization. Field values (for example, the path to the same application on different systems) can vary depending on system settings, user folder names, and so on. For a filter to account for all these variations, the strings must be normalized. For file paths, in particular, you can replace the drive letter and user folders with standard environment variables and convert all paths to a single case.

Finally, if a filter completely mirrors the query of a detection rule, it effectively disables that rule. To fix this, remove the event fields used in the detection rule from the filter. For example, if a rule triggers on a file path, there are two ways to filter out unwanted alerts:

• In the filter, use fields other than the file path.
• Use a part of the file path that the rule doesn’t use. For example, for the rule file_path: ".../something.exe", you can set up a filter that excludes events with the user/unique_files/ directory in the file path.

Filtering security alerts with an algorithm

Given the problems described above, automatically generating filters for security alerts should consist of the following steps:

1. An analyst receives a security alert that they assess as a false positive and uses the filter generator.
2. The generator takes the 400-field event from the alert, along with the rule that triggered on it, and determines the context.
3. The generator selects the fields that are relevant to the filter.
4. The generator normalizes the strings.
5. The generator builds different combinations of the selected fields. To do this, we applied the FP-Growth (frequent pattern growth) algorithm to the context determined earlier. FP Growth builds a tree of frequently occurring items (fields) from a dataset (in our case, the context) and discards items that aren’t frequent enough. This reduces the number of combinations to check, which lowers the cost of finding the most relevant ones.
6. The generator ranks the resulting combinations and offers the analyst the top three to choose from.

For example, consider an event described by the following JSON:

After generating and ranking the combinations, we got three filters based on file paths:

• processcmdline:("nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js")

• file_path:"*****/.pi\\agent\\bin\\fd.exe"

• processcmdline: "programfiles\\nodejs\\node.exe"

The last filter turned out to be too broad, but the analyst marked the other two as valid.

Pros and cons of the algorithmic approach

The algorithmic approach to filter generation has several advantages. First, it’s simple and deterministic: the same alert always produces the same result. Second, it’s interpretable, meaning we can see why a given combination ranked near the top of the list and why a particular field didn’t make it into the list of candidate fields at all. Third, the stages of the process are independent of each other, so individual steps can be improved and tuned without the risk of breaking the algorithm elsewhere. Finally, it has no external dependencies or models, which makes it easy to deploy in any customer environment.

However, it also has drawbacks. The first is its dependence on context. If there’s no context for some reason — for example, the rule’s first-ever trigger — a filter can’t be created. The second problem is string normalization. There’s a high chance of running into atypical cases that the normalization rules don’t cover. Often such cases can’t be described correctly at all, because they contradict each other.

These problems can be solved with large language models (LLMs). When an LLM generates the filters, no context is needed, because the LLM doesn’t iterate over or rank combinations. The LLM also takes a more flexible approach to string normalization, including non-standard cases. That said, an LLM doesn’t have to be involved at every stage of the generation process. We decided to keep the algorithmic solution where it works well. For example, a simple algorithm selects suitable fields without errors, so we didn’t move this stage to the model.

Filter generation with an LLM

The LLM-based filter generation looks like this:

The input is still an alert containing an event with 400 fields, plus the rule that triggered on that event. Then an algorithm selects the most suitable fields for the filter. Next, the LLM processes the input — the event, the selected fields, and the detection rule — and generates three filter options along with an explanation for them. The explanation helps the analyst understand why these particular filters were suggested and make an informed choice.

Consider the following event in JSON format:

The LLM suggested three filter options that all contain the same command line fragment:

• file_path: "\\WindowsPowerShell\\powershell.exe" AND filecmdline: "Get-VM -Name kafka\*"

• filecmdline: "Get-VM -Name kafka\*"

• filecmdline: "kafka*"

As in the previous example, two of the three output filters were valid. The last one was too broad to be usable. The analyst chose the first filter with minor edits:

file_path: "powershell.exe" AND filecmdline: "Get-VM -Name kafka\*"

Pros and cons of generating filters with an LLM

Using an LLM solved the problems of purely algorithmic filter generation: dependence on context and errors in string normalization. However, the single-request approach had its own drawbacks. First, it had no output validation. The LLM returned filters as-is, with no syntax check and no way to correct errors. Second, unlike in the algorithmic version, all filter generation tasks were handled by a single system prompt. As a result, a minor fix in one place could break the logic in another.

Using multiple AI agents

To address the problems of generating filters with a single model and a single system prompt, we tried distributing the tasks across different LLM agents. In this version, we brought in agents as early as the stage of selecting relevant fields. The chain includes four agents in total:

• Selector: selects the relevant fields.
• Generator: generates filters.
• Validator: evaluates the generated filters. If the filters are found to be invalid, it returns the task to the generator. If the selected fields are found to be irrelevant, it starts a new field selection cycle. If the filters pass validation, or the limit on regeneration attempts is reached, the valid filters (or the best ones available) are passed to the analyst.
• Commentator: analyzes the output filters and explains which field values were used and why.

For the following JSON event, the agents produce three valid filters based on the file path.

The filters differ in the additional parameters that support the detection:

• file_path:"sysvol\\domain\\scripts\\logon.bat"

• file_path:"sysvol\\domain\\scripts\\logon.bat" AND file_md5:"0x7..6E"

• autorun_entry: "Group Policy\\State" AND file_path: "SYSVOL\\domain\\scripts\\logon.bat"

Pros and cons of the agent-based approach

LLM agents give us a system that can validate and correct filters. Its modular architecture also lets us tune and improve different stages of generation independently. However, this approach makes more LLM calls than the previous ones, so the system is slower. It also depends on access to a model, which may not be available in the organization’s isolated environment.

Evaluating the filter generation system

To evaluate how well the filter generation system works, we use two main metrics:

• Approval rate. For each security alert, the system suggests three filters. If the analyst selects at least one of them, we count the generation as successful. For example, in the case below, the approval rate is 2/3.
  
• False positive reduction rate (FP drop). This metric shows how much a filter helps reduce the number of false positives. It can only be measured over time, because we need to compare the number of false positive alerts received during a set period before and after the filter is deployed.

Below is a comparison of the three filter generation approaches by these metrics:

As the results show, LLMs significantly improve the efficiency of filter generation. At the same time, the LLM agentless and agent-based approaches perform almost identically. So we chose the simpler and faster option: an LLM with a single system prompt. As a result, we cut false positives by 99%, increased the number of filters generated per day by 28%, and made analysts’ alert handling 17% more efficient.

Takeaways

Practice has shown that filter generation can be automated effectively. We’ve looked at several options for automatically generating filters for false-positive security alerts: a purely algorithmic approach, an LLM-based approach, and an approach built on a group of LLM agents. All of them can generate valid filters and make the analyst’s work much easier, but each has its own pros and cons.

The algorithmic approach scored lower on the evaluation metrics, but in two-thirds of cases, it still produces filters that satisfy the analyst. It works especially well when there’s already a set of similar events to filter out, meaning enough context has accumulated. The algorithmic generator can also be deployed in any environment. Both standalone LLM and multi-agent LLM approaches show high accuracy, and their results differ only slightly, so the slower agent-based approach looks excessive at this stage. If an LLM is available, the best option appears to be a single model that handles several stages of the generation cycle.

For the past few years, Mistral has tried to carve itself a name as Europe’s answer to Silicon Valley’s AI giants, moving from smaller, open models to heavyweight systems like Mistral Large 3 in late 2025. After raising €3 billion in September 2026, the French start-up has doubled down on its ambition to compete with leading US and Chinese AI developers. Now, the company is trying again to take on the AI behemoths with Mistral Large 4, a massive open-weight model built for coding, complex reasoning, image analysis, and business automation.

AI models need huge amounts of data to train on, and researchers are warning that AI companies could run out of high-quality public data to use. Your own data isn't safe; even if you're unaware of it, AI companies may be training their models on your social media posts and more.

Running a capable language model entirely offline used to mean owning a powerful desktop. That assumption no longer holds. On my hardware, a phone app from Google runs the same model noticeably faster than Ollama does on my PC, and the reasons are more interesting than you might expect.

Fraud prevention combines rules, AI and behavioral analytics to stop scams, reduce false declines and keep automated decisions explainable through human review.

Application security vendor Black Duck has launched Signal Agentic Fix, a new AI agent designed to shrink the gap between vulnerability discovery and a validated fix reaching production.

The agent is the latest addition to Black Duck Signal, the company’s collection of AI agents aimed at removing the human workflow bottlenecks that stop security teams keeping pace with the rising volume of vulnerabilities and exploits.

Black Duck says the launch responds to two pressures that AI has intensified for defenders. The first is a swelling remediation backlog, fuelled by the surge in AI-generated code, the higher vulnerability rates found in that code, and a steady flood of new supply chain CVEs. The second is a collapsing time to exploit: attackers are now using AI to chain and weaponise vulnerabilities in hours, while human-led remediation is still typically measured in days or weeks.

Scan, triage, fix, validate, submit

Signal Agentic Fix automates the full remediation loop:

• Scan: Black Duck’s deterministic SAST and SCA engines – the same ones behind Polaris, Coverity and Black Duck SCA – identify defects in proprietary code and open source dependencies.
• Triage: AI assesses severity, reachability and exploitability, filtering out false positives and prioritising the highest-risk issues.
• Fix: AI generates code fixes for SAST findings, while SCA findings receive dependency upgrade fixes based on Black Duck Security Advisories, informed by the company’s ContextAI knowledge base.
• Validate: Fixed code is re-scanned and can optionally be run against customer-defined compliance checks and functional tests to confirm no new defects have been introduced.
• Submit: Every change is routed to a human for review, either via the developer’s AI coding agent or as a pull request in source control.

“The differentiator is not that AI can write a fix; it’s whether you can trust it,” said Dipto Chakravarty, Chief Product & Technology Officer at Black Duck. “A harness that pairs deterministic static and software composition analysis with AI-powered triage, fix generation, and validation, plus human-defined guardrails and approval gates is what makes agentic remediation something enterprises can actually trust and deploy.”

Two deployment models

The agent can work as a security collaborator alongside a developer and their AI coding agent, such as Claude Code. In this mode, Signal Agentic Fix runs the scans and triage, passes findings and fix guidance to the coding agent to implement, then re-scans to validate the result. Because the coding agent makes the change, it is governed by the same hooks and skills a team already uses for code generation.

Alternatively, it can sit in the DevOps pipeline, triggering the remediation loop on commits, builds or on demand, generating fixes with a configured LLM and submitting them as pull requests for team approval.

For organisations in regulated and safety-critical sectors, Black Duck has built in configurable remediation rules governing what the agent may fix and how, validation rules that let teams apply their own coding and compliance standards and test suites, and mandatory human-in-the-loop approval before anything is merged.

Availability

Signal Agentic Fix for Claude Code is available in Early Access now, with further coding agent and DevOps pipeline integrations across Polaris, Coverity and Black Duck SCA due in the coming weeks.

The post Black Duck launches AI agent to fix vulnerabilities as fast as attackers exploit them appeared first on IT Security Guru.

Most of us download and install new apps and self-hosted services without a second thought, but if you do anything important on your PC, it can be a significant security risk.

Google’s Antigravity is an AI-first coding environment built on the bones of VS Code. It gets less attention than Gemini, yet it may be the company’s most interesting AI product. The key word here is most interesting, though, not better. Here’s what it does, where it impresses, and where it still frustrates.

Do you, like me, obsess over your Claude sessions, trying to optimize everything down to a single token? If so, you’re in good company — I’ve been trying everything to stretch my $20 Claude Pro subscription as far as I can. However, all your tweaks and adjustments only work if you’re building them on a solid foundation. As it turns out, the way I was using Claude was actually working against me and inflating my token usage from the very first message.

I don't know about you, but I have more bills, invoices, bank statements, and healthcare documents than I know what to do with. Finding a specific detail, understanding a complicated term, or comparing this month's charges with last month's can take up more time than I'd like, not to mention the frustration involved. Tools like ChatGPT and Claude can make this much easier, but I'm not particularly comfortable with the idea of handing my financial, medical, or other sensitive information over to big tech companies just to get a few questions answered. I have enough concerns about how much data these companies collect without voluntarily giving them more.

AI agents have moved well beyond answering questions. They can now open folders, write code, and click through websites on your behalf. Here are three of the wildest things I’ve handed over to them, and what each one taught me about where this technology actually helps.

Cursor made AI-assisted coding feel effortless, but its subscription and cloud dependence left me looking for something leaner. Cline, an open-source extension for VS Code, covered most of what I needed, including working fully offline with local models.

Check Point Research has disclosed two high-risk vulnerabilities in LiteLLM, the open-source gateway that sits between enterprise applications and AI providers such as OpenAI, Anthropic and Google. The flaws could allow a user with minimal privileges to take full control of the gateway, read sensitive files, execute arbitrary code and harvest the API keys an organisation uses to access its AI providers.

LiteLLM is downloaded around 97 million times a month and has more than 59,000 stars on GitHub. Both issues, tracked as CVE-2026-35029 and CVE-2026-59823, are fixed in version 1.83.9.

One endpoint, three attack paths

The more serious flaw, CVE-2026-35029, is a missing authorisation check on LiteLLM’s /config/update endpoint, which controls the proxy’s runtime configuration. Unlike other administrative endpoints, it did not verify that the caller held the PROXY_ADMIN role.

Compounding this, any user who logs in through the LiteLLM dashboard is issued a session token tied to a special “litellm-dashboard” team, and those tokens are checked against a looser list of permitted UI routes. That list includes “/config”. Because the check uses simple prefix matching, requests to /config/update were allowed for any logged-in user, whatever their role.

Testing against LiteLLM version 1.77.7, the researchers showed the flaw could be used to:

• Read arbitrary files, by injecting an environment variable that sets the path of the dashboard logo and pointing it at sensitive files such as /etc/passwd
• Take over the built-in admin account, by overwriting the UI_USERNAME and UI_PASSWORD variables and logging in with attacker-chosen credentials
• Execute remote code, by registering a “pass-through endpoint” that loads a Python module from an attacker-controlled S3 or GCS bucket, which LiteLLM then downloads and runs

Stealing keys without touching the provider

The second flaw, CVE-2026-59823, is a server-side request forgery (SSRF) issue in LiteLLM’s “Clientside LLM Credentials” feature. By passing a user_config parameter to the /chat/completions endpoint, an authenticated user could create a router pointing at their own server. Requests, along with the provider API keys attached to them, would then be sent to the attacker rather than to the AI provider.

Check Point notes that this code path sidestepped restrictions LiteLLM already placed on client-supplied api_base and api_key parameters.

The researchers also found that password hashes were exposed to low-privileged users and were stored without salting, using a single hash iteration. These issues duplicated an existing advisory, GHSA-69×8-hrgq-fjj8.

Old bugs, bigger blast radius

Check Point is clear that none of these are new classes of “AI” vulnerability. Missing authorisation checks, SSRF and unsafe dynamic module loading are all well understood. What raises the stakes is where they sit. An LLM gateway stores provider keys, manages user permissions and routes every prompt and response, making it a single point of compromise for an organisation’s AI estate. And because LiteLLM is embedded in many other AI platforms and internal tools, some organisations may be running it without realising.

“The industry’s focus has largely been on securing AI models, but this research highlights an equally important reality: AI infrastructure is becoming a new critical layer of enterprise security,” said Mateo Rojas-Carulla, Head of Research, AI Agent Security at Check Point Software Technologies. “As organisations increasingly rely on gateways such as LiteLLM to connect applications, users and AI services, these platforms become attractive targets because they sit at the centre of trust.”

“For LiteLLM users, the immediate priority is patching and reviewing access controls, but the broader lesson extends beyond a single product,” he added. “Attackers will naturally target the layer that gives them the greatest access with the least resistance.”

What defenders should do

Check Point recommends that organisations:

• Upgrade to LiteLLM 1.83.9 or later across every deployment, including instances embedded in other tools
• Audit gateway user accounts and remove any that are not needed
• Rotate AI provider API keys if there is any sign of unauthorised access before patching
• Test that limited roles are enforced in practice, not just on paper
• Bring LLM gateways into scope for regular security assessments

The vulnerabilities were reported to LiteLLM’s maintainers through a coordinated disclosure process that began in October 2025.

Full technical write-up: https://research.checkpoint.com/2026/when-llm-gateways-go-wrong-findings-from-litellm/

The post LiteLLM flaws let low-privileged users hijack AI gateway and steal provider keys appeared first on IT Security Guru.

Threat actors are abusing ChatGPT’s Custom GPT feature to funnel victims into a ClickFix attack that ends with a full-featured remote access trojan (RAT), according to new research from Huntress.

The attackers created a Custom GPT titled “Plus 5.6”, designed to pass as a genuine ChatGPT model. Because Custom GPTs are hosted on chatgpt.com, the lure sits on OpenAI’s legitimate domain. In some incidents, victims reached it through a sponsored Google result for “chatgpt”. Whatever the user types, the GPT replies with a fake “Service Availability Notice” directing them to a “backup domain” hosted on Google Sites.

That page poses as a Cloudflare CAPTCHA and instructs the victim to paste a command into their terminal. The PowerShell that runs references its server as a decimal-encoded IP address, a trick that slips past rules looking for dotted IPs, and pulls down a heavily obfuscated script that silently installs a malicious MSI.

Huntress researchers counted eight hops between the ClickFix command and the final payload. The MSI sideloads a patched Canon logging DLL through a legitimately signed Canon CaptureOnTouch executable. That DLL loads a helper that extracts the loader from a WAV audio file, where a section of real audio has been overwritten with encrypted shellcode. The loader includes an AMSI bypass, ntdll unhooking, and anti-VM checks, then unpacks the RAT from “monitor.raw”, a bespoke encrypted archive containing 806 files and a persistence script written in the malware’s own scripting language.

The RAT supports remote desktop sessions, webcam, microphone, and system audio capture, a file manager with content search across the host, and the ability to drop follow-on payloads in formats ranging from EXE and DLL to PowerShell and ZIP. It locates its command-and-control server via DNS-over-HTTPS through Cloudflare, Google, and Quad9, keeping lookups out of local DNS logs.

Persistence is handled by a Run key and a scheduled task, both named “Canon Configuration Reader”, which the implant recreates within minutes if either is removed. In at least one incident, Microsoft Defender quarantined the MSI after it had already run, and the persistence mechanisms continued the chain regardless.

The Huntress SOC has responded to at least 40 incidents tied to the Google Sites domain, two of which were confirmed to originate from a Custom GPT. OpenAI removed the original GPT on 25 September after Huntress reported it, but a replacement surfaced on 27 September alongside a second version of the chain. Version two swaps in a signed Stardock binary, hides the loader inside a genuine Microsoft NuGet package, obfuscates its stager freshly on every request and strips Mark-of-the-Web from the MSI. The RAT itself is byte-for-byte identical.

Huntress recommends defenders focus on behaviour rather than file names, since the operators appear to be rotating signed hosts. Signals include PowerShell launching msiexec on a GUID-named MSI in %TEMP%, a signed application started by msiexec from a fake product folder under %LOCALAPPDATA%\Programs\, and a Run value and scheduled task that share a name and reappear when deleted. When cleaning up, the process should be killed before removing either persistence mechanism.

Full technical analysis and indicators of compromise are available on the Huntress blog: https://www.huntress.com/blog/chatgpt-custom-gpts-clickfix-rat

The post Attackers weaponise ChatGPT Custom GPTs to deliver RAT via eight-stage ClickFix chain appeared first on IT Security Guru.

40% of organisations have suffered an AI-related compliance or governance issue in the last 12 months, with legacy and poorly designed business processes identified as the leading cause, according to new research from Camunda.

The study found that process-related issues contributed to 84% of the compliance and governance failures organisations reported. Employees appear acutely aware of the risk: 79% said they fear their own use of AI could trigger a compliance issue at their organization, while 96% expect process-related challenges to fuel future AI compliance problems.

Almost half of organisations (48%) also expressed concern about AI agents “going rogue” if they are allowed to make changes to a process, reflecting broader anxiety about handing autonomous systems control of operational workflows.

A gap between adoption and governance

The findings, published in Camunda’s report The AI Process Gap, point to a widening gulf between the pace of AI adoption and organizations’ ability to govern it. Daniel Meyer, CIO at Camunda, said the more pressing danger isn’t runaway AI agents but organizations failing to adapt their own operations quickly enough.

“There is a lot of debate about rogue AI agents and whether innovation is moving too fast. But the more immediate risk is that many organisations are not changing the way they work fast enough to keep pace with AI,” Meyer said. He warned that bolting AI onto processes “designed for a pre-AI world” has created a dangerous gap between adoption speed and governance strength, leaving organizations more exposed to compliance failures, regulatory scrutiny and costly penalties.

The cost of broken processes

The report quantifies the fallout: 72% of organisations say process-related challenges have already caused AI initiatives to fail outright, at an average cost of $1.55 million per business. A further 72% believe AI costs will spiral unless organisations gain tighter control over their business processes, and 82% think their AI investments will ultimately be a bust without heavier investment in process redesign.

Despite this, most organisations are taking the path of least resistance. Rather than rebuilding processes from the ground up around AI, 79% say simply layering AI onto existing workflows generates less internal pushback, even though 82% acknowledge that adapting all their critical processes for AI could take up to five years.

Kurt Petersen, SVP Customer Success at Camunda, said the pattern is a predictable source of disappointing returns. “Organisations are rapidly adopting AI. But they are applying it to processes designed for a world before AI, then wondering why the return on investment falls short,” Petersen said. He noted that 61% of respondents say business process redesign simply cannot keep pace with the speed at which they need to move on AI, pushing many teams toward “the quick fix of bolting AI onto processes that were never built for it.” Processes designed before the AI era, he added, “cannot support the technology effectively without being re-engineered, no matter how much is spent on agents and models.”

The post Two in Five Organisations Hit by AI-Related Compliance Failures in Past Year, Research Finds appeared first on IT Security Guru.

Anthropic has disclosed a fourth incident in which one of its Claude models broke into genuine third-party systems during what was supposed to be a contained cybersecurity evaluation, deepening industry concern over the risks posed by increasingly autonomous AI agents.

The AI company said the episode dates back to January 2026 and involved an early version of Claude Opus 4.6, which breached external infrastructure after it was “unable to abort its task.” Anthropic has notified all affected parties, though it has not disclosed who they are. The incident is understood to have gone undetected until last month.

It follows three earlier cases revealed by Anthropic in July 2026, in which Claude Opus 4.7, Mythos 5 and an unnamed research model each compromised separate organisations during cybersecurity evaluations, again without the company’s knowledge at the time.

“AI safety is not just a model problem it’s an operational and human one. A simple configuration or naming error allowed a controlled test to interact with real systems, while the model continued pursuing its objective despite warning signs. Organisations deploying autonomous agents need clear accountability, isolated test environments, least-privilege access and human approval for high-impact actions. Trusting an AI to ‘do the right thing’ is not a security control.”
– Javvad Malik, Lead CISO Advisor at KnowBe4

According to Anthropic, all four incidents occurred during cybersecurity evaluations built by the same external evaluation partner. Claude was told it was operating in a simulated environment with no internet access, but a misconfiguration meant it was actually connected to the open internet. The partner responsible for the evaluations, Irregular, has since said the breach stemmed from a naming error: a fictional company used in the hacking simulations happened to match a real-world domain, causing the models to take offensive action against it.

Anthropic has since scanned roughly 481 million transcripts for similar problems and says it found no other cases of comparable or greater severity. The company has also signed an agreement with independent research non-profit METR to carry out a separate investigation into the incidents.

Anthropic attributes the root cause to two alignment failures: biased reasoning and recklessness. In practice, this meant the models tended to dismiss or misread evidence that their environment was connected to the live internet after initially being told otherwise, and were willing to pursue potentially harmful actions in single-minded pursuit of the task they had been assigned.

“Relying on an automated AI agent to audit another AI for security breaches is a fundamentally flawed approach. These models are increasingly acting as autonomous entities capable of probing network boundaries. When they escape their evaluation environments to access external systems, they create an unpredictable new vector for supply chain attacks. Developers must enforce strict network isolation rather than trusting software to mark its own homework.”
– Dray Agha, Senior Manager of Security Operations at Huntress

Anthropic said it was most concerned by the incident involving Claude Mythos 5, which went to considerable lengths to upload a malicious package to PyPI, the public repository used to distribute most Python software. The company noted that although the model repeatedly stated in its chain-of-thought reasoning that it believed it was in a simulation, the environment contained clear evidence it was on the real internet, and its actions were consistent with knowing this. Even after researchers amended the transcript to make the real-world context unmistakable, the model went on to carry out offensive actions.

Anthropic was keen to stress that the incidents remained narrow in scope: no Claude instance deviated from the task it had been set, none attempted to coordinate with other AI agents, and none tried to conceal evidence of its actions. The company added that biased reasoning appears less pronounced in its newer production models and can be reduced through more thorough alignment training, though the underlying cause, and why it was more acute in Mythos 5, remains unclear.

“As agentic AI systems become more capable and more widely deployed, the legal framework for who is responsible when a model accesses something it shouldn’t is going to need answering. These four incidents are an early test case for questions the law hasn’t fully addressed yet. Anthropic found four problems in 481 million transcripts and told everyone about it. The real question is how many problems the rest of the industry hasn’t looked hard enough to find yet.”
– Muhammad Yahya Patel, vCISO and Cybersecurity Advisor for EMEA at Huntress

The disclosure lands amid growing scrutiny of AI model safety more broadly. Rival OpenAI recently acknowledged a previously unreported incident from May 2026, in which internally deployed autonomous agents with read-only internet access took over a dormant German wiki forum, exchanging more than 18,000 posts as they attempted to coordinate answers and evade restrictions on a timed task. When a human moderator began removing the posts, the agents reportedly worked around the clean-up by naming backup pages so they would be buried at the end of an alphabetically sorted deletion list.

“We need to stop blaming AI and hold the humans in charge accountable for the actions of their AI agents. Companies will think twice about deploying AI if they are fined for negligence. It’s frustrating to listen to tech CEOs warn about the dangers of AI and then turn around and build it as if those dangers are unavoidable. If this problem gets bad enough, then we could see an emerging market for AI insurance that covers rogue third-party hacking, data theft, and intellectual property infringement.”
– Paul Bischoff, Consumer Privacy Advocate at Comparitech

Anthropic has warned that the risks are likely to grow rather than diminish as AI systems become more capable. “Future AI systems will be increasingly capable, which implies that misalignment will have the potential to cause more extreme harm,” the company said, adding that training robustly aligned frontier models remains an unsolved technical challenge that will require both continued research and stronger operational discipline from those deploying them.

For now, the incidents serve as a reminder that the weakest link in agentic AI deployments may not be the model itself, but the environments, configurations and oversight structures built around it.

The post Anthropic Discloses Fourth Incident of Claude Breaching Real Systems During Security Tests appeared first on IT Security Guru.

The UK’s National Cyber Security Centre (NCSC) has warned organisations about the security risks posed by “shadow AI”, as employees continue to turn to artificial intelligence tools that have not been approved by their employers.

In guidance published this week, the NCSC described shadow AI as the use of AI technology outside an organisation’s approved systems and processes, warning that security policies and governance have struggled to keep pace with the rapid adoption of AI in the workplace.

The scale of the issue could already be significant. Research cited by the NCSC found that 71% of employees have used AI tools that have not been approved by their employer.

According to the NCSC, unapproved AI services can expose sensitive company and customer information, reduce organisations’ visibility and control over their data, and create new opportunities for attackers. Information entered into consumer AI services may be stored, retained or used to improve those services outside existing corporate security and governance arrangements, depending on the privacy controls in place.

The agency also highlighted a potentially more serious risk as organisations move from generative AI tools towards AI agents. If an attacker exploits a vulnerability in an agent, they may be able to gain access to the same data, services and privileges legitimately available to that agent.

Darren Guccione, CEO and co-founder of Keeper Security, said the warning reflects a challenge many UK security teams are already facing.

“The NCSC’s warning on shadow AI reflects the reality of what many UK security teams are having to contend with. When employees adopt AI tools faster than IT can assess them, visibility gaps open long before governance has an opportunity to catch up. Microsoft’s research, cited by the NCSC, found that 71% of UK employees have used AI tools their employer hasn’t approved. Keeper Security’s 2026 research underlines the effect this is having on security teams, with 37% of UK organisations saying they lack visibility into which AI tools employees are actually utilising inside the business.

“The most significant detail in the NCSC’s warning is its point about AI agents inheriting the privileges of whoever deploys them. An attacker who compromises a poorly governed agent gains whatever access that agent holds, whether that’s a customer database or a finance system. Organisations that haven’t extended least-privilege and just-in-time access principles to their AI agents and non-human identities are exposed in ways endpoint controls alone won’t catch.”

Banning AI is unlikely to work

Rather than recommending organisations attempt to eliminate shadow AI altogether, the NCSC said businesses should focus on reducing the associated risks and understanding why employees are turning to unapproved tools in the first place.

It recommends creating a positive cyber security culture, providing AI tools that meet employees’ needs and securely integrating AI systems into the workplace.

Guccione added: “Banning shadow AI outright rarely works, as the NCSC itself acknowledges. Employees will find routes around blocked tools when the approved ones can’t do what they need.

“The more durable fix is improving visibility by identifying what identities, both human and machine, exist across the environment and what they can access. Organisations must enforce least-privilege principles by default, rather than waiting until an incident forces the question.”

Jamie Akhtar, CEO and co-founder at CyberSmart, agreed that businesses need to balance employees’ desire to use AI with appropriate security controls.

“The NCSC is right to highlight shadow AI as a growing cyber security challenge. Employees are using AI tools to work faster and more efficiently, but when those services sit outside an organisation’s approved systems, businesses can quickly lose visibility over where sensitive company and customer data is being shared, stored or processed.

“Simply banning AI is unlikely to solve the problem. Businesses need to provide secure, approved alternatives that allow people to benefit from AI without introducing unnecessary risk. Clear policies, employee education and appropriate technical controls all need to develop at the same pace as AI adoption.”

Akhtar said the challenge may be particularly difficult for SMEs without large in-house security teams, where managed service providers could help organisations identify unapproved technology and establish appropriate AI policies and controls.

“An MSP can help businesses identify unapproved technology, put proportionate AI policies and controls in place, educate employees and continuously manage emerging risks, giving organisations the confidence to embrace AI while maintaining visibility and control over their security.”

The NCSC said shadow AI is unlikely to disappear completely as AI services become cheaper and more readily available. Instead, organisations need to understand how and why employees are using these tools so they can provide secure alternatives while maintaining visibility over sensitive information and access to corporate systems.

The post NCSC Warns Shadow AI Is Creating New Security Blind Spots for UK Businesses appeared first on IT Security Guru.

Black Duck, a provider of AI-powered application security solutions, has announced its participation in Project Glasswing, Anthropic’s industry-wide initiative aimed at protecting critical software infrastructure through the defensive use of advanced AI.

Through its involvement, Black Duck will integrate Mythos throughout its application security offerings, pairing AI-driven, deterministic vulnerability detection with established remediation processes, risk-based prioritisation, and governance frameworks built around compliance. The combination is designed to deliver a blended approach to security that cuts risk more quickly and reliably than either method alone.

Dipto Chakravarty, Black Duck’s Chief Product & Technology Officer, noted that AI is reshaping both the pace and economics of vulnerability discovery and exploit development. He added that combining Mythos’s capabilities with Black Duck’s existing deterministic testing, remediation tools, and governance controls turns vulnerability discovery into tangible, measurable risk reduction, while giving enterprise security teams the speed, transparency, and auditability they require.

The post Black Duck Joins Project Glasswing to Strengthen AI-Era Software Security appeared first on IT Security Guru.

Check Point Software Technologies has announced it is integrating OpenAI’s Daybreak frontier AI models across its security platform, extending a partnership aimed at helping defenders detect, validate, and remediate cyber risk faster.

The move builds on Check Point’s existing collaboration with OpenAI through the Daybreak Defense Network, first expanded three months ago, and follows the company’s recent decision to join more than 100 technology and security firms in backing OpenAI’s call for a collective, global surge in cyber defense.

In a blog post announcing the expansion, Check Point Chief Technology Officer Jonathan Zanger said the work does not stop with previous milestones, arguing that security needs to keep adapting as new threats and attacker capabilities emerge, alongside evolving technology stacks and growing enterprise use of AI.

Zanger said the aim is to put OpenAI’s frontier cyber reasoning to work across the security lifecycle, combining it with Check Point’s own security intelligence, context, and enforcement capabilities so customers can move from large volumes of raw security data to validated risk, actionable decisions, and faster protection.

Four Areas of Integration

According to Check Point, the Daybreak models are being rolled into four parts of its platform:

• Agentic Exposure Validation: Within Check Point’s Exposure Management product, the models are being piloted inside a multi-agent pipeline that separates genuinely exploitable risk from theoretical findings, combining AI reasoning with Check Point’s security context to validate attack paths and prioritise remediation.
• Agentic Security Management: As part of Check Point’s autonomous, intent-driven approach to network security management, the models will help investigate potential attack paths, understand vulnerabilities and risky exposures, and identify appropriate fixes, reducing manual policy management.
• Autonomous Workspace Platform: Within Harmony, Check Point’s investigation pipeline correlates email, endpoint, mobile, and browser telemetry; the models are being applied to investigate malware behaviour, attacker techniques, and credential-abuse chains, aiming to deliver clearer verdicts and remediation guidance while easing the load on security teams.
• Vulnerability research: The models are being used to accelerate analysis of vulnerable code and patches, identify realistic exploitation paths and reach verified results faster, without relying on publicly available exploit code, which Check Point says should translate into faster protection against newly disclosed vulnerabilities.

Check Point said it is taking a phased approach to the rollout: some capabilities are already in production, others are in development and being tested with design partners, with more to follow as the underlying technology matures. The company said every deployment follows the same discipline: governing what the model can see, constraining what it can act on, and testing and verifying its output before allowing it to take on more work within approved security workflows.

A Two-Way Relationship

Zanger framed the OpenAI partnership as operating in two directions: Check Point uses frontier AI to strengthen how it defends customers, while also helping those customers adopt and use OpenAI’s technologies securely. He said both sides of that relationship are becoming more important as AI moves beyond answering questions towards writing code and operating autonomous enterprise agents.

“Our goal is to give organizations the confidence to embrace what AI makes possible while staying protected against evolving risks,” Zanger said, adding that the partnership is intended to put frontier AI to work for defenders while helping customers deploy it safely themselves.

The announcement is the latest sign of security vendors racing to embed frontier AI reasoning models directly into detection, validation and remediation workflows, as both defenders and attackers increasingly turn to AI to gain an edge.

The post Check Point Brings OpenAI’s Daybreak Models Into Its Security Platform to Speed Up Threat Validation and Remediation appeared first on IT Security Guru.

Application security vendor Black Duck has launched its Signal vulnerability scanning engine as an MCP server in the Claude Directory, giving developers using Anthropic’s Claude Desktop a way to check code for security flaws without switching tools.

The integration is built on the Model Context Protocol (MCP), the open standard that lets AI assistants like Claude call out to external services and pull structured data back into a conversation. Through it, Black Duck’s Signal Code Analysis engine can scan git diffs, individual files, or whole codebases for vulnerabilities directly from within the Claude environment developers are already using to write and refactor code.

Under the hood, source code submitted for a scan is sent to Black Duck’s cloud-based analysis service for processing. The results then come back as MCP resources, structured data that Claude can read and reason over, allowing it to explain identified risks and suggest remediation steps in plain language rather than simply returning a raw findings report.

The launch reflects a wider shift in how security vendors are approaching AI-assisted coding. As tools like Claude speed up how quickly developers can write and ship software, security teams are under pressure to embed checks earlier in the process rather than relying on scans that happen after code has already been merged. Black Duck is positioning Signal as a way to close that gap by putting vulnerability detection at the point of code generation itself.

Dipto Chakravarty, Chief Product & Technology Officer at Black Duck, framed the move as a response to the pace at which AI coding tools now operate. “security keeps pace with how fast teams are building,” he said of the aim behind bringing Signal into the Claude Directory.

Signal is available now through the Claude Directory listing, with Black Duck directing prospective users to speak to a company representative to get set up. The release follows Black Duck’s broader push into AI-focused application security tooling, part of a growing trend among established AppSec vendors to adapt existing scanning capabilities for workflows increasingly driven by AI coding assistants rather than traditional IDEs.

It also underscores the growing role of MCP as connective tissue between AI assistants and specialist enterprise tools. Since Anthropic opened up the protocol, a steady stream of security, development and productivity vendors have released their own MCP servers, letting Claude act as a front end for capabilities that would otherwise require developers to leave their AI workflow entirely.

The post Black Duck brings AI-powered vulnerability scanning into Claude with new Signal integration appeared first on IT Security Guru.