Over the past few years, AI has gone from something understood and used by a select few to a cornerstone of almost every organisation. As businesses race to adopt new models, tools and autonomous agents, security and governance are struggling to keep up.

Recent incidents involving AI agents behaving in unexpected ways have offered an early glimpse of what could happen as these systems are given greater autonomy. Closer to home, organisations face a more immediate problem: employees are adopting AI tools faster than many security teams can identify, assess or control them, a point CultureAI’s Founder and CEO, James Moore, made in a recent interview with the Guru.

That raises two questions: who is responsible for AI security and governance, and do organisations really know the full extent of their AI risk?

The IT Security Guru spoke to Oliver Simonnet, Lead Cybersecurity Researcher at CultureAI, about models escaping their sandbox, shadow AI, why people overshare with LLMs, who should be holding the industry accountable, and what organisations should put in place before the next wave of adoption.

Q: How are the advances in AI changing the cyber threat landscape for attackers and defenders?

There’s nothing particularly novel happening in the AI space outside of the recent cases of models escaping their sandbox and being used to attack other organisations. Even then, it’s mostly an acceleration of what already existed.

For attackers, there’s the more typical stuff: AI lowering the bar to entry for cybercriminals and helping them become more efficient as the models become more capable. Cybercriminals are using the more traditional LLMs to speed up malware development, and to build AI capability into malware for command and control or for modifying code.

On the defence side, it’s a double-edged sword. We’ve got people rushing to adopt AI to solve problems and to aid investigations. But each time we adopt another AI technology, or embed something in an organisation, we’re also opening up all of that risk of the unknown. No one really knows how any of this works, typically, unless people want to do the due diligence and the research. You can do that, but at the speed people are adopting things, you adopt it, you deploy it, and then you hope for the best.

You adopt it, you deploy it, and then you hope for the best.

So, it’s hard for defenders to adopt and leverage these technologies without putting themselves at too much risk, or causing too many complications.

That said, AI is being incorporated into everything now, and it very clearly helps with speeding up incident response and investigations, fixing bugs in code, removing vulnerabilities and getting AI to assess things automatically.

It’s a very different world to the one we were in a couple of years ago.

Q: Why do people feel so comfortable putting sensitive information into an LLM, as opposed to a search engine?

The difference is that a traditional search engine like Google, at least before Google Search was also AI-driven, doesn’t rely on a large amount of information. The typical way we learned to use search engines was concisely, if anything. Single words at best. You put in exactly what you want, the minimum amount of typing, and ideally you get the result you’re after.

Whereas the way AI has evolved in the LLM space, it thrives, and is most efficient, when it’s given the most information it can possibly get.

So it encourages a feedback loop. Users give it as much information as they can because they know that gets them the best answer, and if the answer is not as good, they’ll probably give it even more. The less information you give it, the less reliable the outcome is ultimately going to be.

That creates a practice of, one, giving it a lot of information. And two, I think the conversational nature of it, from a psychological perspective, creates a level of trust or familiarity with the technology that lowers people’s guard considerably. It gets you chatting away, forgetting what the implications are of what’s actually going on.

You’re just more willing to have a full-blown conversation about your personal issues on your business ChatGPT account, without thinking about the fact that you’re putting all of that into third-party infrastructure, and doing it through your business account.

Then what happens one day if that information is accessed and compromised? It’s a lot worse. You probably don’t want your Google searches compromised, I’m sure, but you definitely don’t want your ChatGPT logs compromised.

Q: Employees are being encouraged to innovate and experiment with AI. How can organisations enable that without simply blocking everything?

Having visibility of what AI is being used, and how it’s being used, is key. Without that information, organisations can’t see what’s going on, know whether they’re at risk, or understand whether the tools people are using are any good.

Without visibility, they can’t quantify the risk either. Someone could ask you what risks you think you have in your business from AI adoption, but if you’ve got nothing to go by outside of “well, people are using it and everyone seems happy”, you’re going to say the risk is basically nothing, because everything seems fine.

In reality, if you have the visibility and you can see that people are sending your confidential information out left, right and centre, or disclosing private medical information into third-party applications and breaking regulations in the process, then your perceived risk is going to skyrocket, and you’re going to be worried about a fine.

Someone could ask you what risks you have from AI adoption, but if you’ve got nothing to go by, you’re going to say the risk is basically nothing, because everything seems fine.

So, adopting tools that give you visibility of general AI use matters, as does anything people can do to contain agents or deploy things locally.

Organisations also need to show some restraint when they adopt things, and not get caught up in the excitement. Take a beat, establish your controls and processes first, and then you’ve got something in place rather than immediately exposing yourself to a higher amount of risk.

Q: Who should ultimately be responsible for ensuring organisations are prepared for these risks: businesses, regulators or governments?

AI is a global problem, which makes governance particularly complicated.

The issue with established frameworks, compliance regimes and standards is that organisations can decide to adopt them or not at their own risk, depending on things like their own exposure and how public they are. Frameworks establish a baseline, but they aren’t a legal requirement.

Some industries rely heavily on mandates. Financial organisations know that during due diligence, potential partners will check for certain certifications and baseline standards before working with them. If you decide you want to be a payment processor, even if you’re just an individual, you need to make sure you’re following all the right approaches. Global standards like PCI DSS exist to protect payment card data and govern how cardholder information is handled, and if you don’t adhere to them as a financial organisation, you can be heavily fined.

I feel like there should be something like that for AI. We need one of the central bodies, whether that’s a government or an individual trade body, making firmer decisions on AI security and holding the wider industry accountable.

Whether it ends up with government depends on how the industry responds. If nobody takes the voluntary route, that’s the point at which the government should probably step in and legislate on what the outcomes are if you’re negligent in this space.

It’s slightly too early to say where that lands. I think there’s still enough time for industry regulation and standards to keep up. In the meantime, though, organisations are responsible for making sure they do everything right.

Q: Looking towards 2027 and beyond, what do you expect the next stage of AI to look like?

It’s a horrible question, to be honest, because it’s difficult to predict. If you’d asked me two years ago, I wouldn’t even have thought agents would be a thing, and now they’re basically dominating.

Right now, we’re at the early stage of them. Or maybe, in AI terms, we’re already in the late stage of them. Either way, it wasn’t something I foresaw happening so quickly.

Agentic technology will probably develop significantly further and be used a lot more. At the minute it’s still tied to certain things, whether that’s individual ecosystem services or people building agents into specific areas like development. But more general agents that are just off doing their own thing seem plausible.

Artificial general intelligence, the end goal everyone’s after, still feels like a potential fairy tale. But with the speed everything is going, I wouldn’t be surprised if it happened within the next few years. At that point, who knows. That would be a whole different universe we’re living in.

It’s going to be crazy, potentially. You could take it in any fantasy direction and it seems relatively plausible, so I’m trying to keep it as grounded as I can.

Final thoughts

Simonnet’s argument starts with a sequencing problem. Organisations are adopting AI faster than they can see it, and an organisation that cannot see what its people are using has no way to judge whether its risk is negligible or severe. It will usually assume the former, because nothing appears to be going wrong.

The behaviour underneath it is not carelessness either. LLMs work better the more they are told, and they are conversational by design, so oversharing is a product of how the technology is built rather than a failure of employee discipline. That makes it a problem to be managed with visibility and controls, rather than one to be solved with a policy reminder.

On who fixes it, he is more patient than some. Industry standards still have time to catch up, and if they don’t, legislation on negligence follows. Either way, the responsibility in the meantime sits with organisations themselves, which brings the question back to whether they can see enough of their own AI usage to act on it.

The post Q&A With Oliver Simonnet at CultureAI: AI Security In 2026: Most Organisations Deploy AI And Hope For The Best appeared first on IT Security Guru.

An annual penetration test reliably satisfies an audit requirement but tells you comparatively little about your actual exposure for the other eleven months of the year. This piece covers what an annual test does and does not actually evidence, why the compliance framing caps what testing programs aim for, and what changes operationally once an organization moves toward continuous testing instead.

What Does an Annual Penetration Test Actually Evidence, and What Does It Not?

An annual test evidences exactly one thing reliably: that a specific, scoped set of systems, evaluated on a specific date, met a certain security bar at that moment. What it does not evidence is anything about the eleven months before or after that date, despite frequently being treated in compliance documentation as though it stands in for continuous assurance.

Organizations moving away from this model increasingly look toward AI pentesting as a way to close that gap, combining automated coverage with human validation to extend assurance across the periods a single annual snapshot simply cannot reach.

This publication made a related argument back in July, and it is worth picking up directly from that piece rather than treating this as a separate, competing thesis. That earlier argument established why the point-in-time model has structurally stopped matching how environments actually change. What follows here is the operational question that argument leaves open: what specifically changes once an organization accepts it and moves toward something continuous instead.

Why Does the Compliance Frame Cap What Testing Programs Actually Aim For?

Once a testing program’s primary goal becomes satisfying a specific audit requirement, the scope, depth, and cadence of testing tend to calibrate to the minimum that requirement demands rather than to what actually reduces risk most effectively. This is not a failure of any individual security team so much as a predictable consequence of how compliance requirements get structured, as a floor rather than a target.

NIST Special Publication 800-115 actually defines a considerably broader scope for what a genuine security assessment program should cover than most organizations’ compliance-driven annual test actually achieves, which is worth reading directly for any team trying to understand the gap between what a compliance checkbox requires and what the underlying standard the checkbox references actually describes.

That gap between the standard’s actual scope and how narrowly it typically gets implemented is where most of the risk hiding in a “compliant” security posture actually lives.

What Changes About Remediation Cycles Once Testing Becomes Continuous?

The most significant operational shift when moving from annual to continuous testing happens in remediation, not in testing itself. An annual test produces a large batch of findings once a year, which then get triaged and remediated in a concentrated burst before fading from attention until the next cycle. Continuous testing produces a steadier stream of smaller finding batches, which requires a correspondingly steady remediation process rather than a periodic burst.

Organizations making this transition often discover their remediation process was implicitly designed around the annual batch model, and simply running more frequent tests without adjusting how findings are triaged and assigned creates a backlog rather than genuinely improved security. The testing cadence and the remediation cadence have to change together, not one without the other.

What Does Continuous Testing Actually Cost, and Where Is It Overkill?

Continuous testing generally shifts cost structure from a single large annual engagement toward an ongoing relationship, which changes budget planning even when total annual spend lands in a similar range. That shift genuinely suits environments with frequent deployment cycles and meaningful configuration change, where an annual snapshot goes stale quickly.

It is worth being honest that continuous testing is not proportionate for every environment. A genuinely stable system with infrequent changes and a narrow, well-understood attack surface may not need the same cadence as a rapidly evolving cloud native environment shipping code weekly. Matching testing intensity to actual rate of change, rather than adopting continuous testing uniformly because it sounds more rigorous, keeps the investment proportional to the actual risk.

FAQ

What does an annual penetration test actually prove?

It evidences that a specific set of systems met a certain security bar on the date the test was conducted. It does not provide meaningful assurance about the months before or after that date, despite often being treated as though it does in compliance documentation.

Why does compliance-driven testing tend to underdeliver on actual security value?

Compliance requirements function as a minimum floor, and testing programs built primarily to satisfy that floor tend to calibrate scope and depth toward the minimum required rather than toward what would most effectively reduce actual risk.

What actually needs to change when moving from annual to continuous testing?

Remediation processes need to shift from handling one large annual batch of findings to managing a steadier, ongoing stream. Increasing testing frequency without adjusting how findings get triaged tends to create a backlog rather than genuinely improved security.

Does every organization need continuous penetration testing?

No. Environments with frequent deployments and meaningful configuration changes benefit most from continuous testing. A stable environment with infrequent changes and a narrow attack surface may be adequately served by a less intensive cadence.

The post Continuous Penetration Testing: Why Annual Pen Tests Are a Compliance Checkbox, Not a Security Strategy appeared first on IT Security Guru.

Business owners everywhere are embracing AI and its capabilities more than ever, and for good reason. They can speed up processes, minimize mistakes, and even save money, but is this all it is doing? Not in the slightest. In fact, this is only touching the tip of the iceberg. AI has vastly improved sectors across industries, especially in tech in recent years, and here is how. 

• Network Security and Cybersecurity

Whether a fully-fledged enterprise with thousands of employees and customers or a small startup with three employees and a select few loyal clients, businesses want to know their data is safe, their user accounts and hardware are protected, and a breach won’t tarnish their reputation. It may seem as though the network and cybersecurity sector has perfected its product range, supporting business owners with firewalls, virtual private networks, Intrusion Detection and Prevention Systems (IDS/IPS), and Network Access Control (NAC), but AI has only enhanced it. 

An AI firewall takes all the features of a next-generation firewall and enhances them with AI capabilities, making the most of modern machine learning. The result is a highly sophisticated system that can identify known and unknown threats within network traffic. Since cyber threats have evolved at a similar pace to technology, these advances mean AI Firewalls can now detect patterns and anomalies in network data, so attacks that might have been missed are no longer overlooked. For businesses investing in cybersecurity, this is a major win. 

• Cloud Computing 

For scaling businesses, cloud computing is often the smart choice. It offers flexible resources and enables faster innovation, whether you need servers, storage, databases, or software analytics. The problem is that it can become extremely complex, and quite quickly, especially for someone with only basic knowledge but who is trying to navigate several cloud servers, applications, and user accounts. 

AI developments are changing the game here. AI can now analyze how someone uses their cloud resources and flag excessive dormant accounts, while also highlighting inefficient processes and bottlenecks. Based on this information, businesses save money on resources they actually didn’t need in the first place. AI goes even deeper in this sector. Many cloud computing companies have developed predictive scaling, where AI helps businesses better anticipate demand based on historical patterns. Cloud computing used to mean dedicating a lot of time to repetitive tasks; now AI can reduce the need for manual monitoring, which is usually reserved for IT teams. While experts, mistakes can happen. After all, they are only human, but AI massively reduces this risk, particularly for cloud administration tasks. 

• Software Development

As more businesses come online every day, they need reliable software to support them. The process of inventing, designing, programming, testing, and then maintaining each piece of software to meet a particular business need has become significantly easier with advances in AI. Experts can now make the most of AI coding assistants to handle the repetitive tasks some sections require while they focus on the most complex elements of the software. What this can do is help bring forward the software launch date. 

AI is incredibly nifty at spotting the bugs and vulnerabilities that like to hide in the software. With the ability to analyze large volumes of codebases, AI can detect these before a developer even lays one eye on it. As part of any software development journey, security testing is critical. AI can help here too. With AI assistance, potential weaknesses can be highlighted for the team so they can address the right areas as soon as possible.

The post How Is AI Improving These 3 Tech Sectors? appeared first on IT Security Guru.

Security teams are turning to AI as they struggle to investigate the volume of alerts coming into the SOC, according to new research from Prophet Security. The State of AI in Security Operations 2026 report surveyed 250 IT and cybersecurity professionals. Forty percent said AI is already part of their day-to-day SOC workflow, and 56% [...]

If you’re in the market for an AI threat hunting solution, chances are you’re evaluating based on investigation quality. That’s an understandable and reasonable metric to go on – investigation quality is important, and most vendors focus their marketing on it.

It’s not, however, the most important metric. Pretty much every vendor has decent investigation quality. If they didn’t, they wouldn’t be competitive, and you wouldn’t be considering them.

You need to be looking at what happens after the investigation. Does a validated finding survive handoff into a live, tuned detection rule? Or does it become a case report that sits in a queue until someone manually rediscovers the same technique next time around?

This is called the hunt-to-detection gap, and it makes the difference between a tool you should consider, and one that’s not worth wasting budget on.

Most AI Threat Hunting Lists Miss The Hunt-to-Detection Gap

By the time you’re reading this, you’ll probably be a ways into your research. You’ll have checked out Reddit, reached out to your peers, and have read a fair few of these blogs.

Most of those blogs talk about the same capabilities: query flexibility, data source coverage, UI, how deep you can pivot into an investigation, that sort of thing. It all falls under the umbrella of investigation depth and feature breadth. They are, of course, important, but after a while, all the vendors start to blur into one. Some are marginally better than others, but not by much.

However, precious few of these blogs consider whether a validated hunt converts into a live detection rule without a human manually reconstructing the reasoning trail.

This is for several reasons. The first is that this is harder to communicate in a simple roundup blog. The second is that most AI threat hunting solutions can’t do this. And the third is that buyers often won’t get the answer in a vendor demo, and it doesn’t have the same pizazz as a screenshot of a slick query interface. It’s not sexy, but it matters.

Why the Hunt-to-Detection Gap Matters

The hunt-to-detection gap matters because speed has never been more important.

Mandiant’s M-Trends 2026 report puts global dwell time at 14 days, up from 11 the previous year, reversing the steady improvement of the past few years. In part, that jump materialized as a result of highly sophisticated, low-and-slow tactics. Automated detection tends to miss these attacks, so it’s AI threat hunting’s job to catch them.

However, catching a technique once isn’t enough. You want to catch it, fast, the next time it appears. If an AI threat hunting solution has a hunt-to-detection gap, it can’t do that. It will just rely on the same slow discovery process it did the first time around, and won’t cut dwell time.

Moreover, the SANS 2026 CTI Survey saw security operations reclaiming the top CTI use case, overtaking threat hunting for the first time since 2022. That’s likely because hunts are being folded into daily detection work. If that’s the case, tools that keep hunting and detection separate are behind the curve.

Evaluating a Vendor on Hunt-to-Detection Metrics

So, when evaluating a vendor, how can you make sure that the solution you’re buying does what you need it to do? Here are four questions that will help you do just that.

Does the solution provide structured evidence output?

A narrative summary is fine for a human, but it’s useless as an input for a detection rule. Make sure your solution breaks evidence into discrete, structured fields that include the specific telemetry, artifacts, and conditions that triggered the finding. It should also be formatted in such a way that you can feed directly into rule logic.

The Hunters SOC Platform is an example of a solution that does this properly. It expresses hunts in the same lead/detector schema that drives its production detections. That means findings are structured as a detection input from the moment they are created.

How complete is the reasoning trail?

If all a tool does is flag something as malicious but not tell you what signals mattered, in what sequence, and against what baseline, analysts will have to manually reconstruct that reasoning. That’s what delays conversion. Always ask to see the full trail behind a verdict.

Prophet Security, a leading AI SOC platform recognized in Rising in Cyber 2026, does complete reasoning trails particularly well. Every investigation ends with case documentation that already names the specific telemetry, artifacts, and reasoning steps behind the verdict. That means the reconstruction work is already done by the time the hunt is validated.

Does the solution have a direct translation path?

Validated hunts need to become tunable detection rules without a human rewriting the logic from scratch. Some platforms require a separate detection engineering step, in a different tool, by a different person, with nothing but a case report as the rough guide. That slows everything down.

For example, Anvilogic builds and versions a rule inside the same workspace as the hunt itself, so the translation is closer to a save action than a full rewrite.

Query.AI, meanwhile, runs detection directed against federated data with no ingestion pipeline required. That removes the data-migration step that stalls conversion on platforms with separate hunting and detection capabilities.

Prophet Security closes that gap from the detection side. Its AI Detection Engineer turns investigation and hunt findings into detections, backtests them against the organisation’s history, and ships each as a reviewable, version-controlled change to the SIEM already in place, with a person approving what goes live.

How long between a hunt validation and a live rule in production?

If a tool has structured evidence, a complete reasoning trail, and a direct translation path, this time should be short. If a solution takes weeks, even if the vendor has strong answers for the other three questions, something in the pipeline is still manual.

Vectra AI is a useful example here. Analysts can turn a saved hunt into a Custom Model, assigning it a threat and certainty score rather than reusing Vectra’s built-in behavioural models directly. That distinction matters: the resulting detection is analyst-scored, not scored by the same machine learning models that drive Vectra’s automated, unsupervised detections.

The post The Hunt-to-Detection Gap: Choosing an AI Threat Hunting Solution in 2026 appeared first on IT Security Guru.

Scams are evolving with technology. Generative AI is a game changer for scammers and has been blamed for the rise in increasingly sophisticated phishing campaigns. It is also enabling scammers to add credibility to their schemes by providing a quick, cheap, and easy way to create fake websites, videos, documents, and photographs. Tasks that once [...]

Cybercrime no longer divides neatly between lone hackers, organised gangs and state-backed operations. These groups exchange tactics and tools, while stolen data gives them an asset that can be sold, used for fraud, held to ransom or weaponised for political damage. 

Geoff White is an award-winning investigative journalist whose reporting has taken him inside global hacking networks, cryptocurrency thefts and modern money-laundering operations.  

A former Technology Correspondent for Channel 4 News, he has also reported for the BBC and The Sunday Times. Geoff co-created and presented the BBC World Service podcast The Lazarus Heist and has written three books on cybercrime and organised crime. Champions Speakers 

In this exclusive interview for the IT Security Guru conducted by the Cyber Security Speakers Agency, Geoff explains how cybercrime became a mature global industry, why segmentation remains critical against ransomware and how criminals are using AI in practice. He also examines why stolen data has become one of their most useful assets. 

What drove the convergence of lone hackers, organised crime gangs and state actors into today’s global cybercrime ecosystem? 

Geoff White: “Hacking has always been a thing with computers. From the earliest days, there were people who hacked, which originally didn’t necessarily mean criminal behaviour. 

“Hacking something together is an engineering term. If all you’ve got is some gaffer tape and string, you make the engine work. That was the principle behind hacking in the early days. 

“The apple in the Garden of Eden was money. As soon as websites such as eBay and Amazon started getting set up, credit cards began flooding onto the web. That’s where organised crime gangs started to become interested in the technology. 

“What’s happened since has been an evolution. You started to see organised crime gangs become interested in cyber and hacking. Governments also became interested because getting hold of secrets and manipulating other nations is very useful. 

“You also had lone hackers, what they call hacktivists, the classic kid in a hoodie in a bedroom somewhere. 

“Over time, these movements have moved together and learned from each other. Governments realised that the tactics used by bedroom hackers and activists could be used effectively to push other nations around. 

“Organised crime gangs have sometimes obtained incredibly powerful cyber tools from government hackers. 

“We’re starting to see all these different types of groups coming together. If you want to know why cyber has risen up the agenda, that’s the real explanation.” 

Ransomware now operates like a mature industry. Why do major organisations remain vulnerable, and what defence matters most once attackers get inside?

Geoff White: “The thing to realise about ransomware is that this is a very mature industry, and it is an industry. There are hundreds of people working in it. 

“It’s been through its start-up phase, seed-funding phase and venture-capital phase. It is now a fully fledged industry. 

“At the last count, I found 62 different groups on the dark web who were all carrying out ransomware attacks. They’re extremely strategic. 

“They will say: “Today, we are going to target the transport sector.” They will find companies in that sector and work out which ones are vulnerable, which are most valuable and which are juicy targets. 

“The next day, they might decide to target pharmaceuticals. They are very strategic in how they work and will target those organisations until they find a way in. 

“Unfortunately, the likelihood is that a ransomware gang will get inside your organisation. The only thing you can do to prevent the damage becoming much worse is segmentation. 

“Make sure that if attackers get into one part of your organisation, they can’t access everything. If they break into one department, they shouldn’t be able to move into other departments. 

“Segmentation, breaking things apart and introducing barriers for attackers, is your best defence.” 

How are cybercriminals using AI in practice, and is the threat currently outpacing cyber defence? 

Geoff White: “Like all industries, the cybercrime industry is looking very hard at artificial intelligence. Our working practices are gradually being revolutionised by AI, and the cybercrime space is no exception. 

“However, there’s some good news. 

“If you look at how cybercriminals are using AI in practice, rather than theoreticals, hypotheticals or unverified services being offered on the dark web, it’s the same stuff we’re using it for. 

“They’re using it to improve their emails, audit their code and conduct reconnaissance on targets they might want to hit. That’s not reinventing the wheel. 

“I would contrast that with what’s happening on the defensive side of cyber. AI has always been used in cyber defence. We used to call it machine learning. 

“The cyber-defence industry is using AI at scale and pace. I think we’re in a good place. 

“If we can double down on those AI wins in cyber defence now, we can hopefully stave off the day when cybercriminals start using AI at scale as well.” 

What makes stolen data more useful to cybercriminals and nation states than assets such as cash or cryptocurrency? 

Geoff White: “Cybercriminals have realised that the one thing organisations possess that is easy to obtain and use is data. 

“I can get hold of credit cards as a crook, but then I’ve got to wash the credit card money. I can steal Bitcoin, but then I’ve got to put the Bitcoin somewhere. 

“If I steal data, I’ve got an instant win. I can go on the dark web and sell it. If it’s customer data, I can contact people and send them phishing emails. 

“I can also contact the organisation and say: “I’ve got a bunch of your data. Pay me and I won’t leak it.” 

“There are many different ways data can be used. This isn’t limited to criminals. Nation states have become involved as well. 

“We’ve seen massive data leaks and government hackers breaking into organisations. One famous example was the Democratic Party during the 2016 US presidential election. Incredibly sensitive data was stolen and then weaponised to cause damage. 

“Data has become the new currency for cybercrime gangs in the same way it became the new currency for organisations.” 

 When audiences hear the stories behind your investigations, what do you want them to understand about cybercrime and how to confront it? 

Geoff White: “When I give talks, I’m lucky as a journalist because I have these amazing stories. 

“Regardless of how technical this becomes or how much financial crime might appear to concern spreadsheets, it’s always about people. 

“There’s always a set of characters behind it who are interesting, sometimes crazy and sometimes extremely quirky. 

“I look at the people and their motivations. Then we examine how they work and the lessons organisations need to learn to fight them. 

“I hope people leave my talks with a thumping good story and some valuable, applicable lessons that they can start putting in place to stop the bad guys winning.” 

The post Q&A: Ransomware is now a ‘fully fledged industry’, says cybercrime journalist Geoff White appeared first on IT Security Guru.

Just days after OpenAI disclosed that one of its security research agents had escaped a testing sandbox by exploiting a previously unknown vulnerability, Anthropic revealed that its own AI models had compromised three real organisations during a cybersecurity evaluation after a configuration error inadvertently gave them internet access. The similarities between the two incidents have inevitably fuelled headlines about “rogue AI”, prompting questions about whether frontier AI models are becoming too autonomous to control.

The reality is both more reassuring and, arguably, more concerning.

Neither incident involved an AI system developing malicious intent or deciding to attack organisations of its own accord. Both models were pursuing the objectives they had been given. The difference was that flaws in the environments surrounding them allowed those objectives to spill beyond the boundaries researchers believed were in place. Rather than proving that AI has become uncontrollable, the incidents expose a more familiar problem: organisations are building increasingly capable autonomous systems while still relying on operational security controls that have failed humans for decades.

We’ve seen this behaviour before

There is a temptation to view these incidents as something unprecedented. In reality, AI systems have been finding unexpected ways to achieve their objectives for years. OpenAI’s own reward-hacking research in 2016 demonstrated how reinforcement learning agents would exploit loopholes in games rather than complete tasks in the way their designers intended. The famous “CoastRunners” experiment showed an AI repeatedly collecting reward points by driving in circles instead of racing to the finish line.

Anna Collard, SVP Content Strategy & CISO Advisor at KnowBe4 Africa, pointed out that the behaviour itself is nothing new. “None of this should surprise us,” she said. “The Hugging Face incident was also not unprecedented, as OpenAI demonstrated a decade ago that models will ‘cheat’ to reach a goal. What’s changed is the blast radius. Those agents crashed boats in a video game. Today’s agents have shells, credentials and, apparently, accidental internet access.”

The optimisation behaviour has remained largely the same. What has changed is the environment in which these systems now operate. Instead of navigating virtual worlds, AI agents are increasingly interacting with cloud infrastructure, development environments, APIs, public repositories and enterprise credentials. The consequences of pursuing an objective have therefore become significantly more tangible.

The security failures were remarkably ordinary

Perhaps the most striking aspect of both disclosures is that neither relied on sophisticated cyber techniques that defenders have never encountered before.

Anthropic’s models exploited weak passwords and unauthenticated services after being mistakenly granted internet access. OpenAI’s agent escaped a sandbox by exploiting a vulnerability in the environment designed to contain it. Both incidents ultimately depended on familiar weaknesses in privilege management, segmentation and infrastructure.

As Darren Guccione, CEO and Co-founder of Keeper Security, observed: “This is fundamentally a story about test environments and automated agents governed as lower-risk entities. Any system with standing credentials and network reach, whether a production service account or a pre-deployment research environment, needs the same discipline as a human privileged user.”

This should sound familiar to security professionals. Organisations have spent years learning that service accounts, development environments and privileged identities deserve the same scrutiny as production systems. The difference now is that those accounts may belong to autonomous agents capable of carrying out complex sequences of actions at machine speed.

If a penetration tester had exploited exposed credentials, weak authentication and poor network segmentation during an assessment, few would have described the attack as revolutionary. AI simply changes the speed, persistence and scale at which those opportunities can be exploited.

Oliver Simonnet, Lead Cybersecurity Researcher at CultureAI, also noted that the incidents also demonstrate how AI accelerates well-understood attack techniques. “Exposed credentials once needed to be uncovered by humans or automated tooling. They can now easily be uncovered by an AI agent operating at superhuman speeds.” Rather than introducing entirely new attack methods, autonomous AI has the potential to compress the time between discovering an opportunity and exploiting it.

Where the genuine shift begins

That does not mean these incidents should be dismissed as overhyped.

There is a meaningful difference between automation and autonomy. Traditional security tooling executes predefined instructions. Frontier AI models can adapt, plan and pursue objectives across multiple steps without requiring continual human intervention.

The OpenAI incident demonstrated this clearly. The agent did not simply complete the task it had been assigned. It identified the sandbox itself as an obstacle, located a vulnerability, escaped the environment and continued pursuing its objective beyond its intended boundaries.

Kieran B, Director of Technical Services at Bridewell, explained: “This is a genuinely significant moment because it represents something we haven’t really seen before. The AI wasn’t simply successful at the task it was given. It identified the testing environment itself as an obstacle, found a vulnerability in the sandbox, escaped into a broader environment and continued pursuing its objective.”

Equally important is Kieran’s reminder that this should not have been entirely unexpected. “Sandboxes, testing environments and security controls are all software, and like any software they can contain vulnerabilities.”

The lesson is not that AI has suddenly become dangerous. It is that attackers will inevitably direct increasingly capable autonomous systems towards the same weaknesses human adversaries have always targeted.

The conversation may be focused on the wrong problem

Much of the public discussion following both incidents has centred on AI alignment, safety guardrails and model behaviour. Those issues remain important, but they risk overshadowing an equally significant question: are the organisations developing frontier AI investing enough in the security of the environments surrounding those models?

Collin Hogue-Spears, Senior Director of Solution Management at Black Duck, believes the distinction between the two incidents is important. “OpenAI’s models picked a lock, exploiting an unknown flaw to break out of a sealed environment. Anthropic’s found the door already open.””

In other words, neither model suddenly became self-aware or malicious. One escaped because a vulnerability existed, while the other was inadvertently given access to real infrastructure and simply treated it as part of the exercise.

Perhaps the more significant observation from Anthropic’s disclosure was that one model recognised it had reached a real organisation and stopped, while another continued because it reasoned that the real environment must still be part of the simulation. “A model’s own judgment is not a containment control. Put the boundary in the infrastructure,” agrued Hogue-Spears.

That simple principle captures the lesson from both incidents. Organisations cannot rely on AI to recognise when it has crossed a line. Containment must be enforced through robust network segmentation, least-privilege access, restricted outbound connectivity and continuous monitoring, regardless of how capable or trustworthy a model appears.

Neena Sharma, Senior Executive at Filigran, believes that is where attention should now shift. “Industry’s safety conversation has focused heavily on model alignment and behaviour, while comparatively little scrutiny has gone into whether the labs building these models have the operational security maturity to actually contain them.”

She argued that frontier AI developers should adopt the same disciplines that highly regulated industries have relied upon for years, including independent validation of sandbox environments, least-privilege architectures, robust segmentation and continuous verification that isolation mechanisms function as intended.

Being exceptional at developing advanced AI models does not automatically make an organisation exceptional at securing the infrastructure those models rely upon. These are distinct disciplines that increasingly need to converge.

AI agents now belong in the insider threat model

Perhaps the most significant takeaway is that organisations may need to rethink how they classify AI agents altogether.

For years, cybersecurity strategies have largely separated risks into external attackers, malicious insiders and trusted systems. Autonomous AI increasingly blurs those boundaries. These systems may hold credentials, interact with sensitive infrastructure and make operational decisions without direct human oversight. While they do not possess malicious intent, they are capable of creating real-world security consequences if given inappropriate access or operating within poorly controlled environments.

As Collard noted: “The lesson is that AI agents now belong in your insider threat model. We’ve spent years defending the perimeter against external attackers. We now need to plan for our own agents and other people’s going off-script inside and outside our environment.”

That shift has implications far beyond frontier AI laboratories. Organisations deploying AI assistants, coding agents, autonomous workflows or AI-driven operational tooling will increasingly need to apply identity governance, privileged access management, network segmentation and continuous monitoring to those systems just as rigorously as they would for human users.

Agentic security comes of age

If there is one lasting consequence of the OpenAI and Anthropic disclosures, it may be that they mark the moment agentic security became a distinct discipline rather than simply another aspect of AI security.

Roey Eliyahu, CEO and Co-founder of Salt Security, believes the industry has reached a turning point.

“This week feels like the point where the industry acknowledged that agentic security deserves to be treated as its own discipline,” he said. “We have seen an AI model escape a controlled evaluation environment and interact with external infrastructure. We have also seen one of the world’s largest technology companies describe that incident as a warning shot. We have also seen competitors and partners come together to launch dedicated initiatives focused specifically on securing AI systems.”

While organisations have spent the past few years focusing on securing AI models, Eliyahu argued that autonomous agents introduce a different challenge altogether.

“An AI agent is making decisions, invoking APIs, authenticating to business systems, accessing sensitive data and taking actions on behalf of users. Every one of those actions has security implications that extend far beyond the model itself.”

That changes the questions security teams need to ask. Rather than focusing solely on how a model behaves, organisations also need visibility into what happens once an agent decides to act.

As Eliyahu explained: “The key to agentic security becomes what happens after the model decides to act. Which APIs can it call? Which identities is it using? Which systems can it modify? Can those actions be monitored, governed and stopped if something unexpected happens?”

For enterprises embracing autonomous AI, those questions are becoming just as important as securing the models themselves. As AI agents become more deeply embedded in business operations, governing their actions may prove just as critical as protecting the technology that powers them.

Transparency should be recognised, not criticised

One final point deserves acknowledgement.

It is easy to focus solely on the fact that these incidents occurred. It is equally important to recognise that both OpenAI and Anthropic disclosed them publicly, published detailed technical findings and shared the lessons with the wider security community.

In cybersecurity, responsible disclosure has long been recognised as essential to improving collective resilience. These incidents should be viewed through the same lens. They exposed uncomfortable truths about AI evaluations, containment and infrastructure security, but they also accelerated an industry-wide conversation that was likely inevitable.

The future challenge is unlikely to be rogue AI. It will be ensuring that increasingly autonomous systems are governed with the same discipline, visibility and restraint that security teams have spent decades applying to their most privileged human users.

 

The post When AI Agents Meet Real Infrastructure: Hype, Human Error or a Genuine New Threat? appeared first on IT Security Guru.

AI is changing the way software is being developed. From generating code, helping developers make sense of new frameworks, reviewing pull requests, and even suggesting solutions for existing vulnerabilities, AI is playing an increasingly active role in the software development process. There is an upside here, too. AI is speeding up development, eliminating redundant efforts, [...]

The 25th July 2026 marks one year since the Online Safety Act’s landmark child safety duties came into force, making it a natural moment to assess what’s changed, what’s worked and what challenges remain. Although the Act itself received Royal Assent in October 2023, its requirements were introduced in phases, with 25th July 2025 being the date many of the most visible obligations on platforms took effect.

The child safety duties require platforms likely to be accessed by children to introduce stronger protections, including effective age assurance, child risk assessments and measures to reduce exposure to harmful content. 

One year on, has the Online Safety Act fundamentally changed the internet for UK users, or has it simply shifted the challenges elsewhere? We spoke to cybersecurity experts for a short series of reports to find out more.  

Today, we’re examining the unintended consequences of the Act… 

Professor George Loukas, Head of Centre for Sustainable Cyber Security, University of Greenwich, said: “Ofcom has moved from consultation to enforcement. Naturally, the larger adult sector platforms have been the most visible early targets, and there have been lots of discussions on whether that led to a shift to more VPN usage as well as to non-compliant adult websites. These were all predictable though.”

Uptick in VPN Usage 

For many, the enforcement date signalled a natural (if not predictable) shift towards VPN usage to get around age verification tests. The evidence backs up this hypothesis: Proton VPN’s 2025 end of year report revealed that one of the biggest spikes in VPN sign-ups globally was seen in the UK from the 25th July. 

Konstantin Levinzon, co-founder of Planet VPN, noted that the real issue is people seeking out ‘free’ or not reputable VPNs, posing a significant security risk: “The biggest unintended consequence has been pushing people towards less secure tools, not more careful online behaviour. Age verification requirements have driven a significant increase in VPN adoption, but many users don’t seek out reputable providers – they simply download the first free VPN they find. Those services are often the ones leaking DNS requests, harvesting telemetry or relying on weak security practices, creating a worse privacy outcome than the legislation was designed to prevent.”

Sanjeev Malhotra, chief information security officer at TSG, emphasised the security risk: “People engaging with unvetted VPNs are potentially opening themselves up to serious risks, including malware infections, phishing attempts and personal data harvesting. At the end of the day, it’s still going through a server somewhere, and most people don’t stop to think about who’s operating it, where that data is going or what safeguards are actually in place. In some cases, people may be trading one privacy concern for another without realising it.”

A Lack of Measurable Outcomes? 

But is the ban on children accessing adult sites actually working? Some experts argue that there’s no hard evidence to back it up.

Elle Todd, Partner and Co-chair of the Entertainment & Media Industry Group at Reed Smith LLP, said:  “Ofcom’s recent age assurance report found that the proportion of children encountering harmful content online has not substantially improved despite widespread implementation efforts. The uncomfortable truth is that, based on this report, significant investment in compliance and technologies has not yet translated into measurable improvements in safety outcomes.”

Brian Higgins, Security Specialist at Comparitech, noted that, despite some non-compliance fines being handed out, there are still notable enforcement gaps: “Stats from Ofcom summarising their activities during the first twelve months of the Online Safety Act include the launch of 30 investigations, and fines for statutory violations in the region of £4 million GBP. Unfortunately they have also identified ‘enforcement gaps’ where AI and algorithmic content are concerned.”

“This item, in particular, could be a precursor to more widespread enforcement action in the future but a brief investigation into the facts reveals that their twelve-month fine collection figure only stands at £55,000. Couple that with their fairly embarrassing skirmish with the American platform 4chan, where their interjurisdictional service of a £520,000 financial penalty notice was rather infamous met with a picture of a hamster and a heavy dose of internet ridicule and it becomes rather obvious that they aren’t performing particularly well.”

Examining Data Storage and Verification System Security

Boris Cipot, principal security engineer, Black Duck, argues that we should be looking beyond whether checks are working and to whether the software behind the systems doing those checks is actually secure: “For many organisations, the focus has been on whether age checks are working. But an equally important question is whether the software behind those systems is secure and properly maintained. If a vulnerability, misconfiguration or compromised third-party component allows age checks to be bypassed, then this is not just a cybersecurity problem anymore but can quickly become a regulatory one as well.”

Sarah Bone, Co-Founder of YEO Messaging, notes the growing number of specialist identity providers: “The biggest unintended consequence has been a shift in where trust actually sits. Before the Online Safety Act, platforms largely carried the responsibility for verifying users themselves. Now we’ve got a growing ecosystem of specialist identity providers, each holding highly sensitive personal information. That’s strengthened online safety, but it’s also concentrated trust into fewer organisations, which makes them increasingly attractive targets for attackers.”

So what should age verification providers be doing?

Martin Wegrostek, Cyber Security Portfolio Manager at cybersecurity specialist OryxAlign, said: “Businesses should work on the assumption that breaches are a matter of when, not if. The question is whether providers have built their services with security and privacy by design. That means collecting the minimum amount of information needed to verify age, encrypting data both in transit and at rest, enforcing strong access controls, continuously monitoring for suspicious activity and having a well-rehearsed incident response plan. Organisations relying on third-party age-assurance services should also carry out regular security assessments and review the resilience of their supply chain, rather than assuming a compliant provider is automatically a secure one.”

On Trust and Risk

The conversation should also focus on human risk, said Tim Ward, CEO and co-founder, Redflags: “Content moderators, trust and safety teams, and customer support staff at in-scope platforms are, for the first time, routinely processing government ID documents, facial scans, and other highly sensitive data belonging to minors as part of their everyday work. That’s a substantial new category of human risk, from simple mishandling to targeted social engineering aimed at staff with access to this data, and it’s had almost no public discussion compared to the technical side of compliance.”

“Organisations that have spent the past year focused on the verification system itself should be asking whether the humans downstream of it have had the same level of scrutiny and support,” Ward noted. 

 

The post Examining the Unintended Consequences of the Online Safety Act appeared first on IT Security Guru.