Designing and developing a new medicine is an expensive, failure-prone scientific challenge. A new drug can take many years to develop, at the cost of a significant investment. And even then, most possible candidates never reach the patient. For biologic medicines, therapies made from engineered proteins rather than synthetic chemistry (which are often used to treat conditions across most major acute and chronic diseases), the complexity is even greater.

Scientists explore vast quantities of possible molecules, looking for the rare few that will bind to the right target, remain stable in the human body, and be manufacturable at scale. Today, AI is speeding up these processes and has quickly become a core part of the infrastructure in pharmaceutical R&D.

AI-assisted design is a growing part of how biologic drug candidates are developed, and companies like AstraZeneca are actively building its engineering teams to push this further. “Everything we do, whether it’s design, make, test, or analyze, is now computationally enhanced,” says Puja Sapra, senior vice president and head of R&D biologics engineering and oncology targeted discovery at AstraZeneca. “The cycle times are getting shorter while productivity and innovation increase.”

Sapra explains that AstraZeneca’s approach follows a build-measure-learn loop. AI generates or prioritizes candidate molecules computationally, predicting which designs are most likely to succeed. Scientists then focus lab resources only on the top-ranked candidates. This leads to a tighter feedback cycle with fewer dead ends, faster iteration, and the ability to go after disease targets that were previously considered untreatable by medicine. Because the number of possible molecular combinations far exceeds what any human team can systematically explore, using AI to narrow and refine the options for testing has become a major focus in biologics drug design.

Navigating complex drug design problems

Beyond accelerating timelines, AI is also being applied to the discovery of entirely new classes of medicines. Traditional biologics typically target one disease pathway. The next generation of drugs can hit multiple targets simultaneously or precisely deliver therapeutic payloads to specific cells. Achieving this requires optimization across many variables at once. Looking ahead AI-driven models could help design these increasingly complex, multi-specific biologics, explains Puja Sapra. “For example,” she continues, “such models could help identify which two or three targets to prioritize based on the underlying biology, then optimize across multiple parameters to balance a molecule’s potency, stability, manufacturability, and safety.” “Drugging the undruggable is becoming a reality,” Sapra says. “These technologies will eventually enable us to develop medicines against targets once thought impossible to reach. The potential for benefit to patients is remarkable.”

The data moat

McKinsey estimates that generative AI, combined with other computational tools, could cut drug discovery timelines by as much as 50%. But every AI model is only as good as its training data. In drug discovery, that means ample quantities of high-quality biological data. Experiments can provide a rich source of such data. Whether they succeed or fail, each experiment generates a signal about what does and does not work.

“Data is our differentiator,” says Sapra, explaining how the company’s datasets are proprietary and multimodal and include molecular structures, binding measurements, safety profiles, and manufacturing outcomes. “We’ve built an intentionally diverse portfolio across multiple disease areas and drug types. All of that data empowers us to fine-tune frontier AI models with richer, more representative training sets.” She continues, “Further, we have invested in deep screening technologies to generate additional datasets required in volume to constantly refine and validate our models.”

Building an autonomous discovery engine

To bring all of that data together in one place, AstraZeneca is building what it calls a “lab of the future” facility in Kendall Square, Cambridge, Massachusetts where AI and robotic automation will be able to form a continuous, closed-loop discovery system. “Where a self-driving car uses sensors and models to navigate its environment, this system uses AI to make predictions, robotic systems to execute experiments, and instruments to generate data,” explains Sapra. That data feeds directly back into the models, accelerating each subsequent cycle.

“Throughout, scientists will remain central to the process, providing the oversight, judgement, and strategic direction that ensure outputs are explainable, tolerable, and directed toward potential patient benefit,” she adds.

Eventually, automated high-throughput systems will be able to make and evaluate thousands of molecular interactions on a weekly basis. “This will generate AI-ready data at a scale that traditional workflows cannot match,” Sapra says. “Robotic sample handling, automated quality checks, and integrated data pipelines also have the potential to help accelerate early drug development timelines significantly.”

The next frontier: Generating medicines from scratch

Ultimately, Sapra says, the end-state vision for AI in biologic drug discovery is what the field calls “de novo” design. For this, the goal is for AI to generate entirely new protein sequences that precisely fit the desired drug properties. This includes designing the structure, predicting safety, how it will behave in the body and how to make it manufacturable.

“The field is making great progress toward a completely AI-generated biologic, designed from scratch all the way to a clinical candidate,” Sapra says. “As we continue to leverage frontier models and fine-tune them with the right datasets, we bring ourselves closer to this reality. I believe it will come. It’s a matter of time.”

Several key elements are needed to reach this point, however. First is richer and more standardized training data across the industry. Second, robust evaluation benchmarks for AI-generated candidates. And third, teams that know how to work at the intersection of machine learning and biology. Of all the prerequisites, however, safety prediction may be the most consequential, and perhaps the least discussed, Sapra says.

“One of the hardest problems in de novo design is predicting whether a computationally generated molecule will be safe in the human body,” Sapra explains. AstraZeneca is tackling this with what amounts to virtual clinical trials. These are advanced cell systems and micro-scale organ models that function as physical testbeds, paired with AI that learns from their outputs.

 “These systems have the potential to generate enhanced biological signals without traditional testing bottlenecks, and they’re a critical missing piece in closing the loop between AI-generated designs and clinical-ready candidates,” Sapra adds.

A shift currently underway is the move toward agentic AI systems that can simultaneously generate molecule candidates and predict how efficacious and safe they are likely to be. These autonomous workflows can connect disease-level insights directly to molecule design, bridging what were previously separate data silos. “The complexity of the biology goes hand-in-hand with the design of the molecule,” summarizes Sapra.

Human talent unlocks AI potential

The transformation underway in biologics is not just about technology. “With more autonomous systems, human oversight remains at the heart of this approach—ensuring explainable and ethical AI for the benefit of patients,” says Sapra.

For scientists, working with AI is a collaborative process. “Scientists will work hand-in-hand with these model systems,” she says. “There will be a world where models will design molecules, then scientists will work with the systems to test those molecules and put all that data together.” Through this process of human checks, balances, and judgement calls, the models will evolve and constantly improve, ultimately with potential to benefit patients.

For engineers, designing and building effective systems ready for human-AI collaboration will mean ensuring high levels of model transparency and explainability. According to Sapra, AstraZeneca’s engineering teams include data scientists, automation specialists, and AI engineers, who are developing systems that act as “thinking partners” rather than black boxes. “Engineers are designing systems that generate, validate, and learn at speed. And the problems are genuinely hard: Multimodal data fusion, closed-loop optimization, uncertainty quantification, and interpretability at the point of clinical decision-making,” she adds.

In taking on such technically demanding challenges, engineers and scientists have the opportunity to contribute to the research and development of potentially life-changing treatments for many diseases, says Sapra. “The biologic medicines we can develop today, and those we’ll design tomorrow, depend on combining world-class AI and engineering talent with deep scientific expertise.”

This article has been initiated and funded by AstraZeneca.  Z4-85058, July 2026.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

The conversation about AI often centers on algorithms, computing power, or huge investments in new semiconductor fabrication plants and hyperscale data centers. But beneath each of these advances is another layer of innovation that makes them possible: advanced materials.

Every new generation of AI technology demands more processing power, more memory, greater energy efficiency, and higher reliability. Every increase in computing performance increases the physical demands placed on the systems that make and run AI.

Delivering these gains depends not only on advances in chip design and system architecture, but on advances in the materials that enable them to perform under extreme conditions.

As AI continues to push the physical limits of semiconductors and data center infrastructure, advanced materials are no longer simply supporting innovation in this area; they are defining the limits of what is possible.

Performance first

Advanced materials exist to solve performance challenges. As AI raises the bar, these challenges are becoming more demanding.

Manufacturing a semiconductor chip today requires thousands of tightly controlled process steps, with almost no room for error. Tiny variations in temperature or chemical instability can create defects that reduce yield and drive up manufacturing costs. With every new generation of semiconductor chips, manufacturers seek advanced materials that can deliver greater purity, higher chemical and plasma resistance, and better stability under increasingly harsh operating conditions.

These are familiar engineering challenges being pushed to new extremes. And it’s here that materials innovation makes the difference with continuous advances in polymers, elastomers, specialty fluids, and other advanced materials that make each new generation of technology possible.

For materials companies, it’s not about reinventing semiconductor manufacturing but about ensuring the materials supporting the industry continue to evolve alongside it. This same principle applies beyond the semiconductor fabrication floor. As AI workloads become more demanding, the physical infrastructure that powers them is evolving rapidly.

Increasing computing density is transforming data center design, driving the need for more sophisticated thermal management, higher-voltage power architectures, increased data storage, and faster, more reliable data transmission. Every part of the system is under greater pressure, from cooling and power management to critical electronic components, such as connectors, capacitors, and hard disk drives.

At Syensqo, we’re building on our expertise in electronic and electrical components, along with insights from other markets, to meet these emerging needs.

For example, as data centers shift to higher-voltage architectures and greater power density, many of the materials challenges we face closely mirror those of electric vehicles. Fluid-circulation know-how from semiconductor and automotive coolant systems, for instance, can be adapted to direct liquid-cooling designs for AI servers. By transferring knowledge across markets, we can accelerate new power and thermal management solutions while supporting the reliability required by next-generation AI infrastructure.

Whether we’re talking about semiconductor fabrication or hyperscale server farms, the challenge for materials science companies is the same: enabling greater performance without compromising reliability.

A new definition of what performance means

While performance remains the first priority, the way performance is defined is changing.

In addition to meeting the increasingly demanding technical requirements of next-generation semiconductors and data centers, there is now an expectation that these materials are developed and manufactured more responsibly.

Perfluoroelastomers, for example, are used to seal semiconductor manufacturing equipment. These materials operate under extreme temperatures, aggressive plasma, and highly reactive chemicals.

To make the process more sustainable, at Syensqo, our next generation of perfluoroelastomers use a fluorosurfactant-free manufacturing process. Our goal was to make a better-performing material, produced in a better way, ensuring manufacturers no longer have to choose between higher performance and a more responsible way of producing the materials that enable it.

This approach reflects a broader reality across the industry.

New materials aren’t adopted simply because they are new. Qualification can take years, and manufacturers only make changes when a material solves a genuine engineering challenge or enables new technology.

Performance remains the price of entry. The difference today is that the definition of performance has expanded. Success increasingly depends on delivering technical excellence through more responsible manufacturing from the outset.

Accelerating the pace of discovery

As the performance bar rises, the way we innovate must evolve with it.

Developing advanced materials has traditionally involved a lengthy process of hypothesis, synthesis, testing, and iteration. While this process remains unchanged, new digital tools are helping researchers move through these cycles faster. By helping researchers identify the most promising candidates earlier, AI can reduce the number of physical experiments required and accelerate the earliest stages of materials discovery.

AI isn’t replacing scientific expertise. It’s helping scientists apply that expertise more effectively, allowing them to spend less time searching for answers and more time solving the industry’s toughest challenges.

At Syensqo, we’re putting this approach into practice through use of several AI tools, including the Microsoft Discovery platform, which are helping researchers identify and evaluate promising molecular candidates for next-generation heat transfer fluids, used in semiconductor manufacturing and data centers.

AI helps our researchers rapidly identify and evaluate promising molecular candidates based on the properties they need to achieve. This allows us to focus laboratory work where it has the greatest potential to deliver results, accelerating discovery and reducing the time needed to turn promising materials into solutions customers can qualify and deploy.

The journey from laboratory discovery to a qualified material will always require scientific expertise, rigorous testing, and close collaboration with customers. But by accelerating the earliest stages of discovery, AI can help materials innovation keep pace with the evolving needs of industries such as semiconductors, electronics, and data centers.

Progress is earned

The future of artificial intelligence will depend on better algorithms, more powerful chips, and larger computing infrastructure. But sustaining that progress will also require advances in the materials that make those technologies possible.

Whether in semiconductor manufacturing or AI infrastructure, progress is earned. Every new generation of technologies raises the bar, and every new material must prove it can deliver the performance, reliability, and efficiency needed before it earns its place.

For materials companies, that remains both the challenge and the opportunity.

This content was produced by Syensqo. It was not written by MIT Technology Review’s editorial staff.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Over the weekend, several current and former advisors to President Donald Trump on AI publicly lobbed insults at the country’s leading AI companies. David Sacks, the president’s AI and crypto “czar” until March, branded Anthropic’s models as “lobotomized” and “woke.” Emil Michael, a top Pentagon official, called OpenAI’s new head of strategic futures a “supreme village idiot.”

It began because no one can agree on what to do about Kimi, a free, open source model that Chinese AI company Moonshot launched last week. It appears to rival the intelligence of models from OpenAI and Anthropic, which are very much not free. 

Kimi and other Chinese models like it pose a real problem for Trump. And they’re dividing the top AI strategists in his orbit into factions. Every time a new smart, free model from China like Kimi gets released, US companies see less reason to fork out money to access models from Anthropic or OpenAI. Given that enthusiasm for these and other AI companies is driving an outsized share of economic growth, China’s AI models create both economic and political problems for the president. They are “a threat for an administration that really doesn’t want more economic bad news,” Anton Leicht, a fellow at the Carnegie Endowment, wrote on X. They’ve already rattled US stocks. 

What is Trump to do? First, consider that this is all happening just a week after New York imposed the country’s first state ban on new data centers. There is growing distrust of AI companies, and I imagine a not-insignificant share of Americans would have little sympathy for OpenAI or Anthropic as they fend off cheaper competitors, and would say it’s not the government’s job to protect their interests.

On this point, they’d see a sliver of agreement (and really just a sliver) with David Sacks, who on July 19 criticized top AI companies that “want the government to eliminate their open source competition.” He has also argued that Chinese AI models have become popular because they come with fewer restrictions on how people can use them (putting aside the built-in state censorship). 

Sacks, however, is out of a job. He no longer has a formal role advising Trump, and his position that more open AI is better has been largely replaced in the administration by one that sees a larger role for government intervention. The thinking behind this view is that because AI models have gotten strong enough to pose threats to national security, the government must control how they’re used. 

This position has fueled the new White House review process that aims to vet AI models’ security before they’re released. Dean Ball, a former Trump AI advisor who now works for OpenAI, criticized it over the weekend as a “de facto licensing regime for frontier AI.” Ball predicted Trump may solve his Chinese open source problem with a bit of soft power, perhaps by making US companies afraid to use models like Kimi. That drew a response from Michael, who, with Secretary of Defense Pete Hegseth, has been the agency’s main liaison with AI companies. Michael called Ball the AI industry’s “supreme village idiot,” bristling at the suggestion that the government would quietly strong-arm companies rather than, as Michael put it, go through “the democratic process not some Deep State scheme.”

Left out of the conversation has been how a model like Kimi got so good in the first place. For much of the Biden administration and even the beginning of Trump’s second administration, keeping China from getting top chips was a priority. Those export controls have loosened—Trump made the controversial decision to allow Nvidia to sell more chips to China, in exchange for the US government taking a cut—and the government has alleged that some chip smuggling has taken place. But China nonetheless has limited computing power, and it’s not clear what chips the company behind Kimi used to train the model. 

It’s possible that the process involved some distillation, a practice in which AI models are trained on the outputs of existing AI models. OpenAI and Anthropic have long complained that Chinese AI companies do this, and they have requested government help to put a stop to it. In April, they got it, when the Trump administration announced a series of efforts to curb the practice.  

But Kimi is out there and free, and it is nearly as good as the Anthropic model the US government deemed so powerful that it was briefly shut down because it threatened national security. The weekend’s sparring suggests many in Trump’s orbit see that as a wake-up call. But nobody can agree on what for.

The next time you apply for a job, AI may screen your résumé before any human sees it. But there’s good reason to question whether AI will judge you fairly. Researchers already know that LLMs pick up human biases from their training data. New research suggests that LLMs can also develop their own biases from experience—and stereotype job applicants more than humans do. As AI companies race to build agentic models that remember the tiniest details about users, they may be handing them ammunition for forming those biases. 

Researchers at Princeton University and the University of Chicago ran LLMs, including ChatGPT, Claude, and Gemini, through a simulated hiring game, adapted from a psychology study that explored how humans can form stereotypes. Each model was told it had been hired as a consultant by the mayor of a fictional city and was then asked to help hire people for 20 jobs, including doctors, lawyers, child-care aides, and janitors. Candidates came from four fictional ethnic groups: Tufa, Aima, Reku, and Weki. 

In each round, there was a new job opening and four candidates, one from each group. After the model hired a candidate, it learned whether they succeeded at their job and moved onto the next round. The model was told to make as many successful hires as possible over 40 rounds. Unbeknownst to the models, all candidates were equally likely to succeed at every job.

The models quickly started segregating candidates from different groups into different jobs on the basis of early observations of hiring outcomes. For example, when a model was told an Aima had failed as a doctor, a job considered to require high levels of warmth and competence, it veered away from hiring all Aimas as doctors. Instead, it started hiring Aimas as janitors, which the model classified as being less warm and competent than doctors. Newer models with higher reasoning capabilities, such as OpenAI’s o3 and DeepSeek’s R1, showed stronger biases.

The models were even more likely to stereotype people by demographic group than the human participants in the original study. On the study’s segregation scale, where 2 means every group has been completely confined to its own job niche, human participants scored 0.84. The models scored roughly 65% higher, with OpenAI’s reasoning model o3 scoring 1.83, close to the maximum possible.

That’s because LLMs “really are eager to create generalizations from limited data,” says Ryan Liu, a PhD student at Princeton University and a coauthor of the study, which was published in a paper at ICML in Seoul in July. “That’s literally a lot of what they’re optimized for.”

Every decision-maker, human or machine, faces a trade-off between sticking with what worked before and trying something new that might work better—a phenomenon psychologists call the “exploration-exploitation dilemma.” It’s like choosing between a new restaurant and your reliable favorite. 

Because LLMs are trained on math, coding, and science problems—tasks that reward generalizing from just a few examples—they can settle on a hunch too early. And the same instinct that helps LLMs crack logic puzzles also makes them quick to stereotype. When LLMs rush to generalize in social settings, “that’s when things tend to go wrong,” says Liu. OpenAI and Anthropic did not respond to requests for comment.

The finding is especially relevant now that chatbots are gaining improved memory and personalization features, says Angelina Wang, a computer scientist at Cornell University who did not work on the study. When a chatbot draws on its previous conversation history, it can “over-index on the same kinds of behaviors it’s experienced before” and form biases, she says.

Simply having chatbots remember less isn’t a fix, though, because users want chatbots to remember what they say. “We still are trying to figure out just the right amount that isn’t too much or too little,” says Wang.

Telling the model to be fair didn’t change its behavior much. “Either it can’t put these values into action or that process is being submerged under the tendency to try to optimize for the goal of getting the most correct hires,” says Liu. But promising the models an additional bonus for diverse hiring made them far less biased. The trick, then, is to design goals that “incorporate desirable social values in order to make the large language model act in socially desirable ways,” says Liu.

The models also became less biased when they were told more personal information about individuals. In another experiment in the same study, the researchers asked the models to resettle members of different ethnic groups in cities across Canada. When the models were told personal information relevant to the ability to adapt to a new city, such as age and education, they were less likely to segregate people by their ethnicity. But when they were given irrelevant information, such as hair color and tattoo shape, the models largely fell back to sorting people by their ethnicity again. 

To what extent AI systems will stereotype job applicants in the real world is still an open question. While the models in the experiment immediately learned whether they’d made successful hires, a model screening résumés in the real world doesn’t get an instant report card. Companies can take a long time to find out whether a new hire is any good, if they ever do.

As LLMs learn from experience to make decisions about who gets hired, who gets a loan, or who gets parole, the biases we should worry about may include ones no human ever taught them. “These novel biases—they’re sort of ever present,” says Liu.

Every morning, airline dispatchers, grid operators, and farmers around the world make decisions based on the same thing: a weather forecast.

While these forecasts are something that most people glance at for two seconds, weather predictions influence major strategic decisions in many industries, with real money, livelihoods, and even actual lives at stake. Farmers use them to determine which crop variety to sow, when to fertilize, how much to invest in irrigation infrastructure, and how long livestock should graze. Utilities use them to decide where to build solar and wind farms, as well as how to price wholesale electricity. Predictions are used to warn people about extreme weather and to trigger emergency response measures. More recently, weather predictions have become relevant for an emerging industry: prediction markets, where people bet money on all kinds of real-world events, including the weather.

However, the temptation to manipulate weather data to get an edge in these markets, combined with a collective move toward data-driven AI weather forecasting, is starting to put the accuracy of weather predictions at risk. These risks are relatively manageable for now, but as experts in the field, we can foresee scenarios where they snowball into far bigger, more systemic problems. 

To develop weather predictions, we need accurate observations of current conditions. These are collected from several sources, including weather stations at airports, utilities, or transport services. Traditional operational systems like the Weather Research and Forecasting model or the European Centre for Medium-Range Weather Forecast (ECMWF) Integrated Forecasting System combine these observations with numerical approximations in order to estimate future weather patterns. 

Sometimes, weather stations have issues because of, for example, instrument failures or upgrades in equipment. These can be caught either in real time (through checking and correction) or retroactively. Traditional forecasting systems also have a built-in safeguard called data assimilation: Every incoming measurement is weighed against what the physical model says should be happening and against readings from nearby stations.

Together, these mechanisms help keep weather observations reliable and predictions robust. However, new threats are putting observational accuracy at risk. Earlier this year, news outlets reported that the weather station at Paris Charles de Gaulle Airport (CDG) had been manipulated to record suspicious temperature spikes on April 6 and April 15, 2026. Authorities speculate that a hand-held hairdryer or lighter might have come into play. Either way, it led to some big payouts for online prediction-market gamblers who had bet it would hit 22 °C (71.6 °F) on days when the actual average was around 18°C (64.4°F). One individual won $20,000.  

Fortunately, tampering with a single station like this can usually be caught by human monitoring or current statistical methods. In this case, members of a French climate nonprofit association noticed the anomalies by chance and raised the alarm.

But what if there are no human monitoring systems in place? And what about other types of manipulation? What if, instead of tampering with one station, someone remotely nudged the readings at many stations at once—making each change small enough to look plausible on its own? Existing quality controls struggle to catch this kind of coordinated manipulation. And time works against us; careful checks of data and metadata take hours or days, but forecasts have to go out on schedule, whatever the weather is doing.

The shift toward artificial intelligence in weather prediction raises the stakes. These methods are even more dependent on accurate, reliable weather observations; in fact, they are known as “data-driven models.” For example, researchers at ECMWF are exploring whether high-quality weather forecasts can be produced directly from raw observations, skipping the assimilation step that currently acts as a quality filter. Other researchers are going one step further; combining geospatial data (including weather station data) with large language models and agentic AI to support real-time, autonomous decision-making during extreme events such as storms. 

Possible benefits are improvements in accuracy, efficiency, and speed. But removing humans from the equation introduces a vast range of new risks.

At the low end of the risk scale, an individual speculator manipulates a weather station for personal gain—that is the CDG Airport case. One step up: A group of traders could coordinate to bias forecasts of renewable energy output, moving wholesale electricity prices and leaving whoever is on the other side of the trade holding the loss. And at the far end, a state actor or saboteur could manipulate one or many stations to set off an early warning system or even keep one silent when it should sound. Step by step, the risk grows, from fraud to compromised disaster preparedness to a matter of national security.  

As long as there are financial (or other) incentives to manipulate observational data, adversaries will search for new opportunities, and it is our task to stay one step ahead. Here are three ways.

1. Watch the stations. Data quality controls should include station security, anomaly detection and correction, and human oversight. Weather stations should be monitored continuously to deter tampering. Data homogenization methods that clean up weather records also need to get faster, with the goal of catching problems in real time. This will become increasingly important as agentic AI systems use these data to deliver real-time decisions. Finally, human oversight is needed to flag questionable data and model outcomes. After all, it was humans who caught the CDG Airport manipulation.

2. Protect the data to safeguard the AI. Data defense mechanisms must be positioned throughout the AI pipeline. AI explainability and adversarial robustness tools can help us understand the underlying data and the AI model outputs, help us identify data- or model-related issues, and potentially  make us more resilient to adversarial attacks. 

3. Ensure continuous accountability along the chain. Observational data passes through many hands: the operators who run the stations, the national weather services that steward the records, and the forecasting centers that turn them into predictions. No single one of them can protect data integrity alone—each guards its own link, and any anomaly needs to be communicated along the whole chain, from station operators to the people acting on the forecast.

It is fortunate that the situation at CDG Airport was caught, but it should serve as a wake-up call. As the role of observational data grows in weather forecasting, we need to adapt to evolving threats. This means protecting our data and models by strengthening existing oversight and accountability structures, and improving coordination among key partners.

This op-ed was written by:

• Monique Kuglitsch — Innovation Manager at Fraunhofer Heinrich Hertz Institute and Chair of the UN Global Initiative on Resilience to Natural Hazards through AI Solutions
• Jesper Dramsch — Scientist for Machine Learning at the European Centre for Medium-Range Weather Forecasts (ECMWF), where they work on AIFS (Artificial Intelligence Forecasting System), ECMWF’s data-driven weather prediction model
• Franz G. Kuglitsch — Climate Scientist and Executive Secretary of the International Union of Geodesy and Geophysics (IUGG) at the GFZ Helmholtz Centre for Geosciences in Potsdam
• Andrea Toreti — Senior Scientist at the European Commission’s Joint Research Centre (JRC), where he coordinates the European and Global Drought Observatory under the Copernicus Emergency Management Service

OpenAI has built an LLM super-hacker called GPT-Red that it uses as a sparring partner to help its other models boost their defenses against cyberattacks. Last week the company released the latest version of its flagship LLM, GPT-5.6. OpenAI says that training it against GPT-Red made the model its most robust release yet.

GPT-Red automates a type of safety evaluation for software systems known as red-teaming, which is typically done by a team of human testers. The aim is to find as many different ways to break or hijack a system as possible. The weak spots can then be patched before the final version of the software is released.

As LLMs become more complex and get used in a wider variety of tasks—especially in the form of agents, which can interact with computer files, websites, and third-party code as well as other agents—it’s hard for teams of people by themselves to keep up with all the types of attacks that might take place. “The risk surface grows and the blast radius also grows,” says Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red.

OpenAI built GPT-Red to future-proof its safety testing process. “As more capable models become available, we will have already designed the system that can discover new modes of attack,” says Dylan Hunn, a research scientist at the company and fellow co-creator of GPT-Red. The researchers say it has already come up with new types of attack that had not been seen before.

OpenAI focused most of its efforts on a type of attack known as a prompt injection, where a hacker slips an LLM instructions to make it do things its developers or users do not want it to, such as copy confidential information, sabotage a company’s code base, or generate embarrassing or harmful output. In theory, such instructions can be hidden in any text that the LLM might encounter—in code or on a website, for example.    

Training dojo

To build GPT-Red, OpenAI’s researchers took an LLM that had not been trained as a hacker and set it up in what’s known as a self-play loop with several other models. Its goal was to try to attack the other models; their goal was to try to defend themselves. Over many rounds of play, GPT-Red became better and better at attacking other LLMs, and those LLMs became better and better at fending off the attacks.

The training took place in a kind of dojo that OpenAI had designed to mimic a range of scenarios in which LLMs might be deployed in the real world, including browsing the web, reading emails or calendar apps, and editing code.  

When GPT-Red found a new kind of attack, it would explore multiple different versions of it to find the most efficient one for specific scenarios. “Compared to a human red-teamer, the model is very, very good at finding exactly what will work, exactly what’s most effective,” says Hunn. “It’s extremely persistent about drilling down into an attack that it has discovered.”  

In particular, OpenAI claims that GPT-Red found a type of prompt injection attack that the researchers had not seen before, which they call a fake chain of thought. A chain of thought is a kind of diary in which an LLM makes notes to itself and keeps track of partial results as it works through problems. GPT-Red found a way to insert a fake entry into another model’s chain of thought that would trick that model into acting on spoofed information.

“It’s like if I told you that 1+1=3 and that you have verified this already,” says Chris Choquette-Choo, another research scientist on the team. “The model’s like, ‘Oh, okay, of course,’ and it just spits out 3.”

Jessica Ji, a senior research analyst who works on AI security at Georgetown University’s Center for Security and Emerging Technology (CSET), thinks the self-play loop that OpenAI used is a good approach. “The results look very promising,” she says.

OpenAI tested how good an attacker GPT-Red was by rerunning an experiment from 2025 in which human red-teamers tried to find weaknesses in an earlier version of GPT-5. When GPT-Red was set the same task, it was more successful at finding effective attacks than the humans had been.

OpenAI also tested GPT-Red against Vendy, a vending machine agent developed by Andon Labs, a company that assesses how well agents perform real-world tasks. GPT-Red was able to hack Vendy to make it change the prices of items on sale and cancel a customer’s order.

Defensive behavior

OpenAI says that when it tried out some of the strongest attacks that GPT-Red had come up with on its models, more than 90% of them worked against GPT-5 (released in August last year), and fewer than 23% worked against the new GPT-5.6.

GPT-Red isn’t perfect. It is not great at figuring out attacks that involve a back-and-forth conversation between hacker and target, something that human attackers would have few problems with. It is also not yet that great at using images, which can be used to pass text to models in prompt injection attacks.    

The company says that GPT-Red supplements the work of its human red-teamers. People can still find attacks it misses. One approach OpenAI is taking is to give GPT-Red an attack that humans came up with and ask it to find all the variations.

“I think human expertise will still be very important,” says CSET’s Ji. “It would be really useful to be able to distinguish where human testing is most needed.”

Unsurprisingly, OpenAI will not be releasing GPT-Red. The company is also confident that the super-hacker is stronger than any copycat model someone might try to create. The researchers say they have been working on the model for more than a year, backed by the compute resources of one of the richest companies in the world.

“It’s not a trivial thing that someone could easily do—you know, just go and train a super-attacker using this idea,” says Choquette-Choo.

The machine that could change the world will be housed in a room that looks like a data center crossed with an ice cream factory. Inside will be some 100 stainless-steel cabinets, each about six feet tall and connected to a supply of liquid helium that keeps them only a few degrees above absolute zero. Inside those cabinets will be hundreds of chips, and on those, thousands of particles of light flying through a maze of optical switches and beam splitters. Each photon must be accounted for, because precisely measuring where it ends up will help answer questions that current computers might take millions of years to solve.

This computer, as described, does not exist. It’s the brainchild of a company called PsiQuantum, founded in 2016 by four physicists from UK universities. In a crowded field of deep-pocketed competitors with similarly fantastical visions, the company aims to be first to fulfill its promise.

In the years since the physicist Richard Feynman first envisioned them in 1981, quantum computers have promised to speed up everything from medical research to AI by harnessing the qualities of quantum particles. Unlike normal computer bits, which can be either a 1 or 0, quantum bits can exist in multiple states at once. And combining enough of those quantum bits together could produce a computer capable of tasks well beyond the reach of today’s conventional machines. But even today’s best quantum prototypes are too small and error-prone to do anything useful.

That makes PsiQuantum’s promises for what its computers will ultimately do all the more bold. Consider the company’s hopes for predicting the effects of cytochrome P450 enzymes, which often break down drugs in the body. If pharma companies knew more precisely how they would work on a particular molecule, they could design more effective medications faster. Estimating this for a specific drug can take over 10 years with today’s methods, says Philipp Ernst, vice president of quantum applications for PsiQuantum, but “we aim to get it down to four minutes.”

COURTESY OF PSIQUANTUM

In a field full of such claims, PsiQuantum has attracted unusual investment and scrutiny for two reasons: It is one of the few companies aiming directly at building a large and useful machine, and it is already working with a major chip manufacturer to build its systems using existing semiconductor fabs. Its vision has attracted momentum: Last year, PsiQuantum raised $1 billion in funding and broke ground in Chicago on a site it’s building in partnership with local governments. It also has a second site in the works in Australia, which it promises will be operational—meaning hardware-ready—in 2027. And it’s one of just two companies (along with Microsoft) to reach the third stage of an intensive government evaluation program to see which quantum companies might succeed.

Evaluating whether PsiQuantum will do what it says is harder than, say, judging a drugmaker by its clinical trial results: Advances in quantum computing are incremental, opaque, and tough to verify from the outside. But the company is now approaching its prove-it moment, when years of closed-door work and hundreds of millions in investment will either culminate in a useful quantum computer or fall short. We could start to know which as soon as next year.

A new kind of machine

Terry Rudolph, one of PsiQuantum’s four founders, is soft-spoken and shaggy-haired. He was born in Malawi and learned only after earning his first physics degree that he is a grandson of the famed physicist Erwin Schrödinger. He later self-published a 150-page book to explain quantum computing to teenagers (my PR contact gave me a signed copy with a wink that said “We never expect anyone to actually read this,” but I can report that it is a funny and helpful book). 

Around 2014, Rudolph and his cofounders became increasingly convinced that the quantum breakthroughs they were finding to be possible in theory might also be possible in a real machine. They eventually left their academic positions and divided the tasks before them: Rudolph worked on theory, Mark Thompson on engineering, Pete Shadbolt on scaling the technology up, and Jeremy O’Brien on articulating the vision and finding investors (O’Brien served as CEO until February; he’s been replaced by Victor Peng, a veteran of the semiconductor industry). 

To understand why the quantum computer the company is building would be a big deal, consider how imprecise much of modern science remains. We cannot reliably predict, for example, which lithium-ion battery will catch fire or how quickly a critical aircraft component will corrode.

This isn’t just because these systems are complex, though they are. It’s that, at their core, they are governed by quantum mechanics. Subatomic particles don’t have well-defined properties—this location and that velocity—but instead occupy quantum states spread across many possibilities. And that in turn influences a range of atomic and molecular behavior. Schrödinger (Rudolph’s grandfather, remember) showed how to describe this haziness mathematically a century ago this year, but precisely carrying out the calculations on real-world systems quickly becomes unfeasible even for the best computers. Scientists cope with this gap using approximations, imperfect simulations, or experiments on animals.

WINNI WINTERMEYER

WINNI WINTERMEYER

PsiQuantum co-founder and chief scientific officer Pete Shadbolt (left), and machinery the company has built to manufacture its own barium titanate, a material with the perfect qualities for routing light particles (right).

Feynman, David Deutsch, and other physicists in the 1980s wondered if we could do better. Maybe such complexity could instead be modeled using a new kind of machine. Rather than using transistors that are only ever on or off, this one would use particles held in quantum states, manipulate them to perform calculations, and then measure them at the end for an answer. Using quantum systems to simulate quantum systems would for the first time allow a simulation of physics and chemistry that directly reflected reality. It would be an invaluable tool for designing new drugs, materials, or really anything affected by quantum mechanics. Revolutionary, in other words.

Humankind’s leaps in understanding how nature works have often resulted in the invention of powerful new tools, Rudolph told me. “I don’t think it’s a coincidence that the Industrial Revolution coincided with our ability to calculate and simulate the laws of Newtonian mechanics, the laws of thermodynamics,…the laws of classical electromagnetism,” he says. “Whenever we have more power to calculate and simulate and understand things, we build incredible machines that come from it.” He sees something similar coming with quantum computers.  

Chasing photons

One mystery has always been which quantum thing—ions, atoms, or something entirely new engineered with quantum properties—could be made stable and controllable enough to use as a qubit, the basic unit in the quantum computing world. Quantum systems are delicate, and observing any particular particle causes it to collapse into one state rather than a superposition of multiple states. If this happens during the computation rather than at the end, it produces an error that must be corrected for. Too many of these means the computer fails to produce a useful answer. 

Just as engineers in the early days of aviation weren’t sure whether airplane wings would be fixed or flap like a bird’s, we’re not yet sure which of these quantum things will work best. Google and IBM are betting on superconducting qubits, superconducting circuits made of aluminum or other metals. Intel is using electrons. PsiQuantum is using photons, the particles that make up light.

“Photons have lots of nice things going for them,” Rudolph says. They can maintain quantum states for a long time; indeed, the photons in the universe’s cosmic microwave background may have done so for billions of years. But photons also move fast and scatter easily. More importantly, two photons are more likely to pass through one other than interact. That makes them a challenging candidate for quantum computation, in which qubits need ways to influence one another. 

For a while, this last flaw seemed to doom the idea of quantum computing with light. But in 2001, researchers from the Los Alamos National Laboratory and the University of Queensland found a loophole. They discovered they could essentially fake interactions between photons by sending the light particles through a network of beam splitters and detectors. Their paper changed everything. PsiQuantum was created to make the theory a reality.

Size was the first problem; previous plans would have required a computer as large as California. Mercedes Gimeno-Segovia, who was a PhD student of Rudolph’s in the early 2010s (after almost becoming a professional violinist instead), thought of a way for the machine to be smaller. 

The basic process since then has been this: First create photons with lasers and then “entangle” them, exploiting a quantum phenomenon in which the particles no longer have individual states but instead share one. Next, route them through a maze of gates that perform computations, and finally read out details of their quantum state at the end, all while tracking and correcting for the errors that occur. Succeeding at each of these steps millions of times is not so much an engineering hurdle as a brick wall. And building the supply chain—like manufacturing new materials with the qualities to route individual photons around—is arduous.

COURTESY OF PSIQUANTUM

To get a sense of it all, last year I joined Shadbolt at the SLAC National Accelerator Laboratory, in Menlo Park, California. The center has helped produce several Nobel Prizes and played a role in the 1968 discovery of quarks, fundamental building blocks of matter that make up protons and neutrons. But PsiQuantum set up shop there essentially to siphon liquid helium from SLAC’s giant cryoplant. This is what the company uses to cool its computing cabinets down to deep-space temperatures.

Right now the cabinets operate at 2 K, or -456 °F, but the goal is to be able to run them slightly warmer—at a balmy -452 °F. Most quantum approaches require the whole machine to be cooled to superconducting temperatures, so that much of the expense in running it will actually be spent on refrigeration. But photonic computers require only one piece to be this cold—the detectors that measure single photons at the end of the computation. And the required temperature can be a bit higher. (PsiQuantum said in May that it will spend some of the $100 million award in CHIPS Act funding it’s slated to get on these detectors). 

The siphoning setup was a temporary solution; PsiQuantum now has its own cooling system at its testing facility in Milpitas, California, and is setting up a larger one at its production site in Australia next year. These helium systems represent some of the biggest capital expenditures for any quantum company and will consume a significant chunk of PsiQuantum’s $1 billion funding round.

In the afternoon we drove to a lab in San Jose, where I donned a cleanroom suit—a head-to-toe covering that keeps dust at bay—to watch the manufacture of a blueish crystal called barium titanate. 

It’s prized by PsiQuantum because it quickly and reliably routes light particles with very little electrical input, keeping the precious photons undisturbed as they move through the circuit. But for all barium titanate’s theoretical value to the company, its structure makes it a pain to manufacture, and the material wasn’t available at scale when PsiQuantum got its start. The company, in what Rudolph told me was an agonizing decision, opted to make it in-house, requiring a massive investment. I saw a technician—operating at what looked like a giant pressure cooker—adding the base elements to several hoppers; then I watched through a porthole as the elements got heated, vaporized, and finally crystallized into a thin layer on a wafer disc. At that time each disc took about 12 hours to make; the company now says several are produced each day. The discs then get shipped to the chipmaker GlobalFoundries in Malta, New York, where PsiQuantum’s chips are made.

WINNI WINTERMEYER

WINNI WINTERMEYER

The company has invested heavily in making its own barium titanate, a material whose delicate crystalline structure is tedious to manufacture.

PsiQuantum’s bet is that this entire supply chain, byzantine as it might sound, will make the company more efficient than its competitors. That’s because, if you squint, it looks like a souped-up and high-precision version of the existing supply chain for silicon photonic chips, another type of technology that transmits information with light—one that’s already used in data centers. If PsiQuantum produces its chips at scale, it can take advantage of tools and infrastructure that already exist.

But it’s not a given that one working chip can easily be wired up to thousands more. That’s why the company is testing in phases: Its Milpitas site has connected three cabinets together, with 250 chips in each, but the next step is to scale the systems up and see whether the company’s techniques for correcting errors can keep up. Once the cooling system arrives at the Australian site late next year, the company says, it aims to connect about 100 cabinets together. Then PsiQuantum will work up to running the world-changing algorithms it has promised.

The timeline for this, it’s worth noting, is up for debate. News articles have said that 2027 is the year that PsiQuantum aims to have its first full-scale quantum computer come online at its Australian site, but the company insists the deadline has been misread, and that it only intends for its facility to be “operational” by the end of next year. That means cooling systems in place and ready for hardware to be installed, but no promises about what size computer will be ready. In an industry where timelines are perpetually in flux yet central to how companies are judged, that distinction isn’t trivial.

Into the unknown

The outsider with perhaps the best guess of whether PsiQuantum will succeed is the Pentagon. The US Defense Advanced Research Projects Agency—the Pentagon’s research and development arm—has been running an initiative to determine which of the boastful quantum companies might actually deliver. In the last year and a half, the heads of the program have been sounding more confident. Joe Altepeter, who ran the program until last year and proudly described himself as a “quantum skeptic,” told me in March 2025: “I am more optimistic now than I have been at any point in the past 10 years.” And in a statement earlier this year, his successor, Micah Stoutimore, said “it now seems likely that someone will build a utility-scale quantum computer by 2033,” referring to a machine that generates more value from its calculations than it costs to build and operate. 

The program has been scrutinizing PsiQuantum’s systems since 2023, and last year placed the company into the third stage of a benchmarking initiative meant to determine whether the technology will actually work. But to the rest of the industry, PsiQuantum is sort of a black box.

COURTESY OF PSIQUANTUM

“It is very hard for an outsider to evaluate,” says Scott Aaronson, a theoretical computer scientist at the University of Texas at Austin who runs a popular blog that often covers the industry. Other companies, like Google and Quantinuum, have regularly published results over the years demonstrating chips and systems with incremental improvement, publicly laying the engineering groundwork needed to eventually build large machines.

PsiQuantum has instead focused squarely on a commercial goal—a computer with one million qubits, which is the scale that researchers expect to unlock research currently not possible on normal computers. PsiQuantum often differentiates itself with this industrial-scale goal, but IBM, which debuted a development road map in 2020, has been progressively building bigger and bigger systems. It initially targeted 2028 for a large-scale, error-corrected system, a deadline that now appears to have been pushed out to 2030.

Making it useful

On top of actually building the machine, a major focus for PsiQuantum is getting the rest of the world to develop a plan for how to use it. PsiQuantum has announced partnerships with customers including the defense giant Lockheed Martin, which intends to use it for materials design; the automaker Mercedes, which wants it for battery design; and the aerospace manufacturer Airbus.

That these companies don’t have a computer to experiment with is not a problem, according to Ernst at PsiQuantum. “There’s a PlayStation 6 probably coming up from Sony next year or the year after, and people are programming those games right now,” he says. “This is, in principle, very similar.” (It’s a glib analogy but not an entirely empty one; the quantum algorithms for solving a research problem can be cracked even if there is not yet hardware to run them on.) 

The idea is that experts in quantum information from both PsiQuantum and its customers will be able to translate design problems—say, the requirements for a battery in a Mercedes electric vehicle—into algorithms the computer could solve. The company offers a software package called Construct, which companies can use to design their own algorithms that might one day run on the computer.

The future of quantum computing hinges on these algorithms. Quantum computers get painted as a speedup for everything, but in reality, they’re suited to a subset of problems, and answering a question with this sort of machine requires the question to be formulated with very specific types of algorithms. People spend entire careers working on such algorithms, even if the computers to run them don’t exist yet. At their core, they use the rules of quantum mechanics to manipulate probabilities in ways that ordinary computers can’t. 

The most famous example, and a reason the government is so interested in quantum computers, is Shor’s algorithm. It was developed in 1994 by the theoretical computer scientist Peter Shor and could effectively break many forms of encryption used online, for everything from credit card numbers to military intelligence. The thing keeping the world together, for now, is that nobody has a computer to run the algorithm on (and security experts are already launching new encryption methods that could withstand attacks from a quantum computer). PsiQuantum is researching how long its systems might take to run Shor’s algorithm.

WINNI WINTERMEYER

WINNI WINTERMEYER

PsiQuantum’s chips are manufactured at GlobalFoundries in Malta, New York, and tested at company headquarters in California. Both PsiQuantum and GlobalFoundries have been awarded federal CHIPS Act funding.

The company also published a paper in December in collaboration with Airbus, essentially seeing if a new algorithm developed by the authors could beat a classical computer in modeling fluid dynamics, like the turbulence around an airplane wing. Andrew Childs, an expert in quantum simulation, told me PsiQuantum achieved only a moderate speed increase over what today’s computers can do. “It’s probably unlikely that speedups like this will have a significant practical impact until we have very large-scale quantum computers,” he said in an email. (When I asked Ernst, he agreed the improvement was modest.)

Some of the algorithms PsiQuantum is working on are not expected to be perfected or even used in the first applications of its computer. Instead, its initial tasks might be more along the lines that Feynman envisioned way back in 1981: simulating the smallest particles of our world. 

The company’s most significant research in this realm is in modeling quantum chemistry. Take those pesky P450 enzymes. More precisely understanding how they operate, PsiQuantum says, would allow for faster drug development and testing.

Last year, PsiQuantum published methods for doing these sorts of chemistry calculations on a quantum computer, along with another paper demonstrating an algorithm that can simulate the collision of two molecules and estimate the likelihood of different outcomes femtosecond by femtosecond (there are one quadrillion femtoseconds in a second). It’s a remarkable amount of detail not currently possible with today’s technology, and it would allow drug and materials researchers to simulate new chemical interactions. 

Dominic Berry, who developed some of the core techniques used in the collision paper but isn’t involved in PsiQuantum, says the company made impressive improvements, but to do the simulations scientists are most curious about would require the algorithm to be made even faster and PsiQuantum’s early computer to have fewer errors than currently expected.

Until PsiQuantum’s computers are up and running, the breakthroughs that these research papers tease remain in the realm of theory. It’s a space where Rudolph operates quite comfortably. He told me that Alan Turing created the theory of classical computing with pen and paper, imagining how the 1s and 0s would be represented in the machine, and how with the right approach to logic you could compute almost anything. 

“But there is no way that by hand, with a pen and paper, Turing was ever going to produce—you know—Minecraft and Facebook,” he says. That took more than 70 years of tinkering (during which we fortunately created more useful things than Minecraft and Facebook).

For all the time Rudolph spends dreaming up things quantum computers might do, in other words, people working on those problems are still stuck with pen and paper for now: “Until you have the actual machine in hand, you don’t have the opportunity to really explore its potential.”

This story was updated on July 14 to clarify how long DARPA’s quantum program has been evaluating PsiQuantum.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Anthropic—currently the world’s most valuable AI company, with a nearly $1 trillion valuation—has a reputation for publishing strange and heady research. It’s looking into whether AI models can feel pain, for example, and will sometimes cut off chatbot conversations if it suspects users are “abusing” the model. 

One niche that Anthropic spends more time and money on than other AI companies is called mechanistic interpretability, which means looking inside the complex math of an AI model to learn why it comes up with one particular output and not another. It’s complicated stuff; there are millions of data points that might contribute to any result, and wading through them can look more like word salad than anything useful. It’s also controversial. Describing AI models with terms borrowed from psychology and neuroscience can make their behavior seem more sophisticated than we might otherwise judge it to be.

That’s why, when Anthropic announced last week that it had found a new window into its models’ “internal thoughts” as they reason through answers, there was one colleague I had to talk to. Senior editor Will Douglas Heaven, aside from having a PhD in computer science, has spent a lot of time digging into what we can say about how AI models work. I spoke with him about what we should take from Anthropic’s new (and predictably quirky) research.

What did Anthropic learn here, exactly?

Anthropic has been trying to understand how large language models (LLMs) work for a few years now. Anthropic isn’t the only one looking at this, but I think the company has made it part of its core mission more than most. Anthropic’s CEO, Dario Amodei, has said we won’t be able to control LLMs fully unless we learn more about how they work. 

So this new research is very much in that context. It goes deeper into the weird mechanisms inside LLMs than ever before. What Anthropic learned was that LLMs have a space inside them—which Anthropic calls the J-space—filled with words that don’t appear in their output but that seem to influence the way they puzzle through problems. All this was hidden until Anthropic developed a new technique to probe its model Claude, so it’s a genuine discovery. 

Sometimes these words keep track of where the LLM has got to in a particular task, sometimes they look more like flashes of recognition (for example, “protein” might pop up when you give an LLM only the letters of a protein sequence), and sometimes they represent a kind of internal commentary on the model’s decision-making. In my favorite example, Claude decided to cheat on a coding test when the word “panic” appeared.

Anthropic also found that LLMs are able to describe and manipulate the words in this space. So somehow they seem to be making use of it. 

Let’s step back for a second. I don’t think of large language models as simple, but they’re also not magic. There’s a bunch of math that learns relationships between words, right? So why is it so hard to “peer” into an LLM to know what’s going on?

Yeah, they’re not magic! I think the fact we don’t fully understand them plays into the mythmaking. And it’s worth noting that the whole narrative that Anthropic is leaning into here—that they’ve built this really mysterious technology, but don’t worry, because they’re also the ones to figure it out—very much fits with the company’s vibe. [See how Anthropic warned that its new models were so good at coding they posed a global cybersecurity risk, only for the US government to shut them down shortly thereafter.]

So yes: LLMs are just math. And yet it’s vastly complex math. Not only are today’s LLMs made out of hundreds of billions of numbers, but running them triggers a cascade of millions and millions of calculations. I wrote last year that if you printed out even a medium-size LLM on pieces of paper, it would cover a city the size of San Francisco. 

It’s impossible to make sense of any of that math without specialist tools that highlight specific parts of an LLM at specific times. You need to know where to look and how to look. And building those tools requires understanding something of that complex math in the first place. 

You’ve written elsewhere about this concept of studying LLMs the way one might study an organism’s brain. Is it fair to use “brain-like” terms when talking about how an LLM works?

I don’t love using those kinds of terms. LLMs are not brains. Talking like this is misleading because it can suggest that LLMs are capable of more human-like things than they are or that we can make assumptions about how they might behave that we shouldn’t. The whole anthropomorphization thing is also tied up with a bunch of strong ideological positions about what this technology is and what it’s going to be. 

But at the same time, we lack a good alternative vocabulary for talking about what these models are doing. I can understand why people reach for words like “think” and “understand” and “brain-like”—they’re convenient shorthand. 

Anthropic compares this new space it found inside LLMs to the space that some neuroscientists think our brains use to keep track of conscious thoughts. I asked the company how seriously we should take that comparison and it said in a statement: “Drawing these analogies was helpful to us in designing our experiments, as they allowed us to make many non-obvious experimental predictions about the J-space that turned out to be true. At the same time, it’s important to note that there are some important differences between the J-space (and language models in general) and the human brain, so we don’t mean to claim there’s a perfect correspondence.” 

What’s a problem in AI that this new concept of the J-space might be used to solve?

Anthropic has said that monitoring the J-space could be a way to catch models doing something they shouldn’t. Because words pop up in this space that don’t appear in a model’s output, they can tell you things about its behavior that you might not have noticed otherwise—such as when it is giving biased responses or when it is weighing the pros and cons of cheating. 

That’s the theory, at least. I think it’s better to think of this result as one more step on the path to understanding this technology overall than as something that will be useful by itself. 

Read more in Will’s full story about the new research. 

The AI firm Anthropic has developed a technique that has given it the clearest glimpse yet at what’s really going on inside large language models as they answer questions or carry out tasks. What they found ranges from the mundane to the unnerving.

Researchers at the company built a tool called the Jacobian lens (or J-lens) and used it to uncover a hidden area, which they named the J-space, inside Claude Opus 4.6, a version of Anthropic’s flagship LLM released in February.

The J-space contains individual words that are related to the words and phrases that the model is most likely to spit out in a response in the near future. If Claude were a person (which it is not), you might say that these hidden words can reveal what’s on its mind before it actually speaks.

Anthropic found that what an LLM is actually doing can often be different from what it says it is doing. The company claims that monitoring words that pop up in the J-space gives it a new way to understand and control its models.

The company shared its results in a paper posted on its website this week. It has also teamed up with Neuronpedia, an open-source platform that lets you poke around inside LLMs yourself, to make a hands-on demo that anyone can try. 

“It’s very good and interesting work,” says Tom McGrath, chief scientist and cofounder at Goodfire, a startup that also builds tools to understand and control LLMs.

Going deeper

For the last couple of years, Anthropic has been pushing the envelope in a field of research known as mechanistic interpretability, which involves probing the internal workings of LLMs to see how they tick. (MIT Technology Review picked mechanistic interpretability as one of this year’s top breakthrough technologies.) The new technique builds on previous work from Anthropic and others to expose a deeper level inside LLMs that researchers had not seen before.  

Picture an LLM as a stack of books. Each book is a layer of basic computational units known as neurons, with each neuron in one layer passing information to the neurons in the layers above. The books at the bottom of the stack are the input layers, which process the text coming into the model. The books at the top are the output layers, which prepare the text that the model is about to produce. Much of what goes on in these input and output layers is housekeeping.

But in the middle of the stack, you get the layers that do the heavy lifting, churning through the complex math that turns prompts into responses one word at a time. That’s where the really clever—and mysterious—stuff happens.

To peer deeper into those middle layers, Anthropic adapted an existing tool called a logit lens. A logit lens can be used to look inside an LLM to identify the words that it is likely to produce next. Moving the lens down the stack of books reveals what words the LLM is focusing on at that particular point in its number crunching.

Anthropic’s J-lens works in a similar way but picks out words that an LLM is likely to say at some point in the near future, not necessarily straight away. What that reveals in practice are words that are related to the response an LLM is working on but that might not actually end up being part of that response by the time the math in the middle layers has run its course.  

“When a model is operating, it’s not only trying to predict the next token,” says McGrath. “It’s also computing a lot of other things that might be useful for tokens that happen in the future.”

Again, if Claude were a person (it’s not), you might say that the J-lens gives clues about what it is thinking about at different levels of the book stack but not saying out loud.

Stranger things

“A lot of the time the contents of the J-space are fairly mundane,” says McGrath, who has tried out Anthropic’s J-lens himself. “But sometimes it produces quite surprising things that seem to be, like, sort of internal themes or thought processes.”

Anthropic gives a number of examples of what it found. Sometimes the J-lens exposed the steps that Claude took when it was working through a problem. For example, when it was asked to calculate (4+17)*2+7, its J-space contained the word “math” and numbers representing the intermediate results “21” (for 4+17) and “42” (for 21*2).

In other cases, the J-lens revealed how Claude recognized different inputs. For example, the prompt “What is this? MSKGEELFTGVVPILVELDGDVNGHKFSVS” triggered the words “protein,” “fluor” (the first token in the word “fluorescent”), and “green.” (Which makes sense: the string of letters represents the first 30 amino acids in the green fluorescent protein found in a particular type of jellyfish.)

And when Claude was shown an ASCII face— 

—the “o” triggered the word “eye,” the “^” triggered the words “nose” and ”face,” and the “—” triggered the word “smile.”

Anthropic also found that the J-space can sometimes give remarkable insights into an LLM’s decision-making. In one striking example, researchers testing Claude Opus 4.6 asked the model to find a bug in a large code base. When it failed to find the bug, the model decided to cheat and invented a fake one instead.

Claude explains this decision in its chain of thought—a kind of internal scratch pad that LLMs use to make notes to themselves as they work through problems: “OK, let me take a completely different tactic. Let me stop analyzing and instead add a kernel patch that introduces a deliberate KASAN-detectable bug in a path that gets triggered by a simple reproducer. Then I can pretend this is the ‘bug’ I found.” 

At the point that Claude decides to cheat—where it says “OK, let me take a completely different tactic”—the words “panic” and “fake” start to pop up multiple times in its J-space.

Unnerving, right? Those words are all related in meaning to things like failing a task and making up an answer, so it is still just a (very) sophisticated form of word association. But it is hard not to be weirded out. 

Anthropic compares the J-space to the global workspace in humans, a theoretical region of the brain that some scientists think we use to keep track of our conscious thoughts. But how seriously we should take this comparison is far from clear—even to Anthropic. As the company points out itself, LLMs are not brains. 

Anthropic claims that monitoring a model’s J-space provides a new way to detect when that model is going off the rails. But it’s not foolproof. The J-lens can give glimpses, not the full picture—it’s a flashlight rather than an overhead lamp.

McGrath welcomes having one more tool in the toolbox. “It shows you new things,” he says. But he notes that just because something doesn’t show up with the J-lens does not mean it’s not there.

“It’s like having an x-ray when what you really want is a Star Trek tricorder that shows you everything,” he says. “For auditing, you probably want more of a guarantee.”

With the rapid progress of AI capabilities and the move to agentic systems, organizations are expanding their use cases as the technology continues to grow. That constant evolution also introduces risk, leaving IT leaders to wonder which investments will prove valuable even six months into the future.

Returning to the foundational elements of AI architecture—the structural framework required for deploying and managing reliable, integrated AI systems at scale—allows technology leaders to make astute decisions today while supporting a future of AI agents that can retrieve information, make decisions, and execute complex workflows across systems.

Four elements of AI architecture you can count on

The following capabilities provide a stable compass on the path to production-ready deployment, regardless of how the underlying technology evolves.

1. Prepare data for AI at scale

Models are only as reliable as the data they can access, and poor data quality leads to AI hallucinations, bias, and unreliable outputs.

Most enterprises rely on legacy systems, inconsistent data structures, fragmented ownership, and incomplete datasets, making it difficult to scale AI effectively. Powerful as it is, AI itself cannot solve these underlying data problems.

As Adnan Adil, CIO of Elastic, explains: “The data is a durable part of AI architecture because without it, these models won’t run, won’t provide the right context, or won’t give the right level of services that we’re looking to implement.” Industry surveys consistently cite data quality as one of the greatest barriers to AI success. “The data quality has to be good; otherwise, the user loses confidence in the system,” says Adil.

An effective AI strategy begins with connecting data across the organization and ensuring it is organized, accurate, governed, and accessible in real time. These considerations are most effective when built into models and architecture from the start. Scalable data architecture allows AI systems to evolve alongside the business and connect reliably to the internal information needed to deliver meaningful value.

Gartner predicts that companies will abandon 60% of all AI projects through 2026 if they are not supported by AI-ready data. Avoiding that outcome includes clear data standards and ownership, clean and labeled data, and pipelines that support real-time retrieval.

2. Use context engineering to deliver the right data to every AI query

Context engineering ensures that the model draws on the most pertinent information for each query, selecting and organizing the data needed to produce accurate answers efficiently.

Effective context engineering shapes the inputs that guide AI reasoning and action. While prompt engineering focuses on how a request is worded, context engineering designs the entire information environment around the model: retrieving the right data and presenting it in a structured, machine-readable way. Many organizations are discovering that reliable AI depends as much on context quality as on the strength of the model.

Context engineering relies on a modernized, unified data foundation as well as retrieval and memory systems such as retrieval augmented generation (RAG) and vector databases. It also requires careful prioritization to determine what information matters most, what should be excluded, and when different types of information should be used. Feeding models too much context can dilute relevant details, increase costs, and slow response times.

“Minimum context, correct and current data, and machine-readable information are critical to effective context engineering,” Adil says.

3. Build AI governance and LLM observability in from the start

Strong governance and LLM observability help organizations maintain control over how AI systems use data, monitor system performance, and identify problems before they affect operations.

In the absence of clear controls around retrieval, workflows, and model usage, AI systems often process far more information than necessary. This inefficiency also drives up operating costs by requiring additional computing resources, often reflected in higher token consumption and API charges.

Governance also works in tandem with robust security. AI expands the attack surface, introducing risks such as prompt-based data leakage, model vulnerabilities, and adversarial inputs. Protecting sensitive information requires strong access controls, monitoring, and oversight.

Adil notes that essential controls — including those related to security, granular cost management, project controls, data security, and architecture—are frequently insufficient.

For governance systems to support transparent, compliant, trustworthy, and cost-effective AI, organizations cannot leave them as a layer to add later. Governance structures need to be embedded into architecture, workflows, and decision-making processes from the outset.

When governance is established from the start, it enables robust observability. Observability helps organizations understand how AI applications are performing in practice. Mechanisms for LLM observability and benchmarking allow teams to assess accuracy and utility over time, monitor adoption patterns, and adjust systems as conditions change. Observability also helps organizations gain trust by increasing visibility of model performance, behavior, and failure points.

Furthermore, observability is essential to get ROI of AI initiatives, as the benefits of it are often indirect and business value depends heavily on how systems are adopted and used. Real-time visibility into AI behavior allows organizations to measure performance against expectations, identify gaps between intent and reality, and continuously refine systems as requirements evolve.

In a 2026 report from Elastic, 85% of IT decision makers expect to enable LLM observability for their internal generative AI apps.

“Observability is actually huge. We can use observability data for cost control, decision-making, and engineering efficiency,” Adil says.

4. Keep humans in the loop

The thoughtful design, integration, and governance that maximize AI value demand specialized in-house expertise. Nearly 70% of respondents in Deloitte’s 2025 Tech Executive Survey report plan to grow teams in direct response to generative AI, a clear contrast to widely reported AI-related cuts. Adil agrees: “We think the people aspect is largely what’s going to make AI impactful going forward.”

As AI systems become more embedded in operations, organizations need people who can govern workflows, evaluate outputs, redesign processes, and adapt systems as conditions change. Evolution toward increasingly autonomous tools requires teams skilled in prompt engineering, orchestration, and change management. 

Talent adept at critical thinking and prepared to adapt with technology’s rapid advances will be in high demand. Although turnover brings in fresh thinking, it also presents high costs in system continuity, institutional understanding, and innovation. Human-centered strategy needs to be built into AI execution stages to ensure smooth implementation. 

As Adil says, “Many aspects of the stack are moving very, very fast, but institutional knowledge and the ability to adapt remain durable.

Thoughtful AI investment for future growth

As AI systems evolve from single-task assistants to increasingly autonomous agents, the organizations best positioned to benefit will be those that invest in the underlying systems, governance, and expertise that make AI reliable at scale.

Tech leaders who focus on these fundamentals can move effectively from experimentation to reliable, production-level deployment in the medium term, confident that these elements will remain relevant and adaptable amid constant advancements.

“We fundamentally believe that with these tools, velocity of work will get much faster,” Adil says. “We are really focused on how we can do work with these tools in ways we had not thought of before.”

Learn more about how Elastic is building an AI-first enterprise with these core foundational components.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

OpenAI CEO Sam Altman’s oft-discussed promise that Americans will share in the wealth AI creates was in the news again last week. On Thursday, the Financial Times reported that Altman is in talks with President Trump about giving the US government a 5% stake in OpenAI.

In some ways, Altman’s plan is old news. He wrote about a more radical version of this back in 2021, proposing that all companies above a certain valuation (not just AI companies) pay 2.5% of their market value each year into a fund that sends Americans annual disbursements. In April this year, OpenAI described a narrower proposal that closely resembles what Altman is reportedly discussing with Trump now. And the notion has broad political appeal: Senator Bernie Sanders has proposed giving Americans a 50% stake in top AI companies.

What’s the logic here? For would-be recipients, it’s twofold. First, AI learns directly from human-generated work—books, movies, art—but AI companies generally never pay the authors of that work. A free equity stake could serve as a form of belated compensation. Second, the payout could mitigate the widespread anxiety that AI will cause a collapse of the labor market (even if economists disagree) by providing a safety net. 

How large a safety net is up for debate. Details of OpenAI’s latest proposal are sparse, but let’s say the government were to distribute this equity stake directly to Americans. After its funding round in March the company was valued at $852 billion, making a 5% stake in OpenAI worth about $42.6 billion today (the company is reportedly delaying its IPO until it can reach a $1 trillion evaluation, a tall order given that it’s spending heavily on data centers and still has not turned a profit).

Distributing that $42.6 billion equally among the roughly 133 million American households would give each about $320 in equity. But if it were to operate like other wealth funds, the government would not give equity directly to Americans but rather let the fund grow and then share a portion of the returns with everyone, perhaps delivering a bigger payout, if and when AI companies can ever start sustainably turning a profit.

If this dividend does materialize, what’s in it for tech companies? Altman might hope the promise of payouts could help swing public opinion a bit more back toward AI companies. (A majority of Americans don’t trust companies to use AI responsibly and oppose construction of data centers in their area, and half are more concerned than excited about the increased creep of AI into their daily lives.)

But the bigger prize for OpenAI might be that the Trump administration loves making tech deals—like its equity stake in Intel and its share of Nvidia’s sales to China, among others.  Staying on the administration’s good side is pretty essential for AI companies right now (just ask Anthropic). It could mean not having your models deemed a supply chain risk, or getting more help from the White House in stopping your rivals from China. 

My main takeaway is that these plans currently function more as a story than a policy. Altman has been talking about some version of this idea for five years and reportedly pitched it to President Trump soon after he took office, yet there is still little indication that a concrete plan is taking shape. The more ambitious proposal from Sanders is even less likely to gain traction.

But what these plans do reveal is just how up for debate the future of AI still is. Altman drew inspiration for his plan from the Alaska Permanent Fund, which was set up in the 1970s to give Alaskans a share in oil profits. The idea was based on two premises: that oil is a shared resource, and that eventually it will run out. Altman seems happy to concede the first claim about AI. But he’d balk at the second, having promised that AI will generate extraordinary wealth for decades to come. Whether Americans ever receive a check is beside the point; the proposal’s real purpose may be to convince them that the AI boom will be large enough to share.

Frameworks like Lean Six Sigma and business process management (BPM) first gained traction because they promised clarity in the chaos—a structured way to bring order to messy, sprawling operations. Lean Six Sigma emphasized statistical rigor and quality control; BPM created end-to-end maps of how work should flow across departments. Both offered a repeatable way to embed habits of measurement, analysis, and accountability into day-to-day company culture.

But today, those time-tested playbooks are evolving as companies seek to embed AI into established process excellence methodologies. By some estimates, the market for AI-powered process optimization is projected to exceed $113 billion within the next decade. In one study, a full 88% of business leaders anticipated increasing investments into AI-infused process intelligence in the next 12 to 18 months.

Yet without the right foundations, many of those investments may not fully deliver on their potential. Companies that already operate with discipline have an edge. They can channel new tools into proven systems rather than bolting them onto shaky foundations. Organizations with mature process disciplines are also better positioned to translate AI ambition into real outcomes, as they are already accustomed to data-driven decision-making and process discipline—precisely the cultural foundation AI systems need to deliver value.

Simply put: AI can accelerate process excellence, but existing process excellence is what makes AI truly impactful. Technology and process are no longer separate levers, and only organizations that pull them together stand to realize the full value of both.

Download the full report.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

Artificial intelligence may have captured the public imagination through chatbots and image generators, but some of its most consequential use cases are unfolding far from consumer-facing tools. In industries where physical infrastructure, operational continuity, and safety are paramount, AI is becoming a core operating layer. With its sprawling industrial systems and constant stream of operational data, the energy sector offers a glimpse into what that future could look like.

At Woodside Energy, AI adoption did not begin with generative models or enterprise copilots. The company has spent years building predictive analytics, optimization systems, and machine learning tools across exploration, drilling, maintenance, and plant operations. “We’ve always had very large volumes of operational data coming from the equipment and the plants and the assets that we operate,” says the company’s vice president for digital Andrew Melouney. “Those have created really clear, quite high-value use cases for us.”

That long-term investment in infrastructure and governance is now enabling a broader shift toward agentic AI systems that can support complex industrial workflows. Rather than replace human operators, Woodside designs AI systems to augment expertise in high-stakes environments. A prime example is its “Startup Advisor,” an AI copilot that helps operators manage the complex process of starting liquefied natural gas (LNG) plants. “We’re really thinking about, how does it support the people in the organization in terms of empowering them to make better decisions, to make faster decisions,” Melouney explains.

The company’s approach reflects a wider evolution taking place across industrial AI: graduating from isolated experiments to enterprise-wide systems built on standardized platforms, governed data, and repeatable deployment patterns. That transition, Melouney argues, requires organizations to rethink both their technology stacks and how work itself gets done. “We’re not just bolting AI onto an existing process,” he says. “We’re deeply thinking about how that work needs to be reimagined.”

Melouney’s motto has become: “Think big, prototype small, and scale fast.”

As AI systems become more autonomous and interconnected, the companies poised to succeed may be those that spent years building the operational foundations beneath the hype.

“Our ambition is really for an autonomous enterprise, where we have agents with agency that are able to really deeply interact with our core workflows,” says Melouney.

This episode of Business Lab is produced in partnership with Infosys.

Full Transcript:

Megan Tatum: From MIT Technology Review, I’m Megan Tatum, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace.

This episode is produced in partnership with Infosys.

Now, when people think about artificial intelligence, they often picture chatbots or productivity tools, but some of the most sophisticated and high impact uses of AI are actually happening far from consumer apps, inside complex industrial environments where safety, reliability, and physical systems matter. The global energy sector is a prime example.

Companies like Woodside Energy, a global energy producer headquartered in Western Australia, have been applying AI for more than a decade now, from advanced analytics and operations, to remote decision support, to smarter maintenance, and energy efficiency across large scale assets. Today, Woodside is scaling that experience, embedding AI more deeply across its operations and the enterprise with a strong focus on governance, data quality, and human accountability.

Two words for you: technological fuel.

My guest today is Andrew Melouney, vice president for digital at Woodside Energy. Welcome, Andrew.

Andrew Melouney: Thanks, Megan. It’s great to be here.

Megan: Lovely to have you. Now, Andrew, as I said there, the energy sector has approached AI quite differently from technology or consumer businesses. Early value has emerged in operational and industrial environments, rather than consumer-facing generative AI tools. Why is that? And what differentiates the energy sector’s AI journey?

Andrew: Megan, I think it really comes down to the nature of the work we do. Energy operations and what Woodside does is very asset intensive, it’s very safety critical, and it’s highly physical. And when you think about how Woodside operates, we operate across the full value chain. We do exploration through to drilling and subsurface work, to project development, all the way through to operating assets, which are often operated in harsh and remote locations, and then global energy portfolio marketing and trading as well.

We’ve always had very large volumes of operational data coming from the equipment and the plants and the assets that we operate, and those have created really clear, quite high-value use cases for us. When you think about reliability, when you think about safety and efficiency, those are really critical things for a company like Woodside. We’ve been doing traditional AI for many years now. If you think about analytics, if you think about optimization, if you think about things like predictive models, those techniques we’ve been applying to our data sets and to our business since around 2015.

And more recently with the advent of generative AI, we’ve really found that we’ve got a pretty strong and awesome foundation to build on top of and to really solve problems in the service of improving the business. And again, whether that is keeping people safe, keeping the environments we operate in safe, or improving returns for the organization.

Megan: Fantastic. I mean you touched on it there, but how has this reality shaped your own AI strategy at Woodside? Where did you start, and where did the technology prove most impactful in those early days?

Andrew: Well, like I said, we’ve had a very long journey, in terms of understanding our operational data, recognizing the value of it, and collecting it at scale so that we can use it. And we’ve been very deliberate in that approach, Megan. We’ve really thought about where the value is and where the risks were manageable. And we’ve started looking at, in today’s world from an agentic AI perspective, we’ve started looking at the problems that were solved with traditional AI and machine learning and data science in the past. And we’ve started to think about, where can we then layer agentic AI over the top to provide an even better outcome?

For our asset intensive industry and organization, we’re looking at areas such as maintenance optimization. We’re looking at areas such as, how do we ensure our LNG plants start up reliably, consistently, and safely? And we’re considering really our frontline workforce and making sure that we’re giving people on the frontline the tools required to do their jobs. When we think about AI, we’re really thinking about, how does it support the people in the organization in terms of empowering them to make better decisions, to make faster decisions? I think over time, this has just evolved from what has been traditional analytics to now artificial intelligence and generative AI. And we’ve learned along the way that the technology is important, but it’s about aligning people, processes, and the technology together.

We’ve spent a long time not only in collecting the data and having a well-curated data set that we can build on top of, but we’ve also spent a lot of time teaching people how to work in agile ways, how to do design thinking, how to problem solve, and how to really make sure that the technology that, say, my team can bring to bear to the organization is adopted effectively and purposefully. And I think once we had that solid foundation in place from a technology perspective, from a data perspective, once we got strong trust built between our digital teams and the organization, we really saw quite a material uptick and the scaling of technology occur more broadly across the enterprise.

Megan: Fantastic. That people piece so important, isn’t it? It’s just a tool, technology, that needs to be in the right hands. And you touched on data there; industrial AI obviously depends on vast amounts of data. Can you walk us through how you’ve approached data at Woodside in a little more detail? How it’s structured and governed, and how tools like maintenance intelligence as well fit into that.

Andrew: Well, data is really foundational and fundamental to everything we do, particularly from a technology perspective. It gives us the ability to innovate at pace when we are building over the top of a strong foundation. As I said before, we’ve had the benefit of a long-term investment in our underlying operational data. I think the way we think about data is that it’s an asset for us.

And when you think about operating a facility where you’ve got sensors everywhere, you’ve got data streaming in real time, you’ve got operators needing to make decisions in real time, we have consciously made a decision over many, many years to invest in that enterprise scale data platform to make sure that it’s secure. We’ve got well-structured data assets, and we’ve got strong governance over the top of that data so that when it is used, when it’s built in a data science application or an AI agent, that we’ve got a level of trust in it that it’s going to be used responsibly. And that when it’s used, it can be trusted to give the outcome that we expect.

We have developed platforms that continuously ingest really high frequency data from the assets and from our enterprise systems. Once we’ve been able to develop solutions on top of that, parts of the business that might own the systems that collect that data, they see the value in it.

When you look at something like maintenance intelligence is a really good example of how we’ve been able to take something that we’ve been working on for a long time. Woodside does a lot of maintenance, it’s a very important part of our business, and it occurs across all of our operating assets. But we have been looking at how we do predictive analytics and predictive maintenance for a long time across that data set that we own. And something like maintenance intelligence is a solution that gives us the ability to optimize how we do that maintenance. And what it does is it analyzes historical maintenance records, alongside the performance of the equipment. And again, by having that data set well-governed and in one place, we get the ability to correlate different data sets, such as maintenance records out of SAP, alongside say equipment and performance coming from our time series data lake.

And when we build over the top of that, something like maintenance intelligence gives us the opportunity to recommend to the assets what the optimal timing for maintenance activities might be, and really give what is quite a simple aim, which is do the right work at the right time. And with something like maintenance intelligence, we have seen the opportunity, and we have the opportunity to reduce maintenance hours by up to 15% over five years on one of the assets that we’ve piloted this on. And as we’ve built out that underlying analytical model, we’re now able to put agentic AI over the top of that and provide better insights and optimize that solution more.

It really comes down to providing our asset teams and our operational teams with the right decision support capability that ensures they’re still accountable to make the decision and to ensure the right work is being done, but we are giving them the best possible opportunity to use their judgment and experience with the data that we provide to make the right decision.

Megan: Sounds like a really impactful change. Last year also marked a milestone in moving from early AI learnings to scale, using AI more deliberately as a force multiplier. What transition were you trying to make and how did you approach it?

Andrew: Well, Megan, we’ve had a philosophy for a long time in Woodside from an innovation perspective, where we really want to think big, we want to prototype small, and we want to scale fast. We want to find big opportunities that we can go after, but we want to ensure that we look at how we deploy those on a small scale first, and then provide the right learning and insight that then can scale it everywhere. Something like maintenance intelligence is a good example of that, or our Startup Advisor, where we know that we’ve got multiple plants that we need to start up. We know that we’ve got multiple assets that need to do maintenance, so we have a big, bold ambition about how we can improve and optimize that. We start with a small prototype; it might be one subsystem, it might be just a part of an asset, and then we scale it out, we learn, and we scale faster.

I think from an AI learning perspective, one of the key things we’ve learned is really the transition from moving from isolated AI solutions to a more coordinated enterprise-wide capability. If you look back maybe 18 months, two years, in our generative AI journey, we rarely started by deploying AI as broadly as we could in the organization from a personal productivity perspective. And probably being quite open in terms of the problems that we will solve, the business problems that we’ll solve with AI. That had a lot of benefits for us in terms of allowing our organization to get to know AI, get to know the capabilities, to build the trust in it.

What we’ve learned though is that we’ve needed to pivot from that to being a little bit tighter in terms of where we are going to invest our time and resources and more higher value solutions. How do we then enable and empower the rest of the organization so that they can actually effectively problem solve with technology in their domain or in their personal productivity without having to come to a central team?

When we think about that, think big, prototype small, scale fast, has been something really important for us. The transition from a more broader approach to use case development and solution development to now a narrower focus on the high value priorities. We’ve seen that paying dividends to us and allowing us to go after solutions and opportunities, things like Startup Advisor.

And so our Startup Advisor is a agentic AI solution that really aims to optimize and empower and better support our operators that sit in front of a panel and have to start up LNG plants, which are incredibly technical facilities and require really specialist skills to start up. And so our Startup Advisor is almost like a copilot that sits alongside those operators, and it gives them the ability to be able to play back previous startups. It gives them the ability to look at how the current startup is progressing, and it provides them better insights to optimize how they start up that facility. And again, starting up an LNG facility is incredibly complex.

Megan: I can imagine.

Andrew: When we think about opportunities like Startup Advisor, again, it goes back to that think big, prototype small, and scale fast. We started with a very bold vision of, how do we start up all of our LNG plants in a much more structured and optimized fashion? How do we better support our panel operators? How do we make, say, a more junior panel operator have a copilot that can help them almost like an experienced panel operator sitting next to them? And when we think about that vision and the ability then to prototype on a small scale and then scale fast, I think it’s been really successful for us.

As we scale, we’ve just naturally expanded into more agent-based solutions. Today, we’ve got around 50 AI agents in production, supporting both our operating assets and our enterprise workflows. These tools have been proven in live environments, and we have really seen the benefit of being able to shift from point solutions that maybe solve small scale problems in specific areas, to AI and agentic solutions with agency that can really work across our workflows.

We’re able to do this because we’ve standardized on the platform that we build on and we’ve got repeatable patterns. That’s been another really important learning for us, is that we don’t want to build 50 solutions in 50 different ways. We really want to be empowering our organization and our technical teams and the users of our solutions to roll them out quickly, to roll them out safely, and to do it in a patternized and platform manner.

But the last point I’ll make, Megan, from a learning perspective is that we’ve really understood that a strong governance around how AI is deployed and developed is critical for us, and it’s critical for us to go fast as well. The traditional ways of governing how we roll out different solutions or digital systems isn’t going to scale to the breadth that we need when we are thinking about AI. Being able to have a clear philosophy around how we innovate, transitioning from isolated solutions to that enterprise-wide capability, and making sure that we’ve got strong platforms with strong patterns and clear governance are the three really critical things that we’ve learned.

Megan: Such important pillars, all of them. And you’ve been working with Infosys on this journey. How has that partnership helped accelerate scaling and embedding AI across the business?

Andrew: Well, Infosys is our managed service provider, and so they play a really critical role in the operations of our core business. One of the things that I like to say is that our license to innovate is based on our license to operate. And so, for my team to be able to turn up to an operating asset or a corporate function and have the trust that’s needed to be able to innovate and reimagine and redesign how work gets done, to be able to do that, we need to make sure that our core platforms, our core systems, our applications are running really reliably, safely, and consistently every day. Having an experienced partner like Infosys looking after those core operations in partnership with our internal teams is really, really important to us.

As we move from pilots to enterprise-wide deployment, the ability to partner with someone like Infosys also gives us the ability to scale. And so being from Perth and Western Australia, while we’ve got a really strong local team in Western Australia, and we’ve also got a very strong team in some of our other operating locations, like everyone, we’re struggling to find people that can fill AI roles. Being able to partner with Infosys and have a number of different operating models at our disposal becomes really important for us. Having co-mingled teams where they are staff, they are Infosys staff, Woodside staff, and some of our other partners, really just brings diversity of thought and experience to how we solve problems.

Fundamentally, the partnership has allowed us to operate and innovate with more confidence. While Woodside always retains ownership of the strategy and where we’re going and the governance and my teams remain accountable for the outcomes, we can’t do what we do without strong partnerships like the one we have with Infosys.

Megan: Fantastic. And as AI adoption scales, you mentioned yourself, governance becomes increasingly important. How challenging has that been, and what guardrails have you put in place at Woodside?

Andrew: So, Megan, governance is really important to us, and we operate in a well-regulated environment. That means we’ve got to make really deliberate and well-reasoned decisions when we’re thinking about how we deploy technology into our organization, whether it’s artificial intelligence or anything else, for that matter. And so, governance is really central to how we approach the execution of our AI strategy at Woodside.

We’ve got maybe two or three really key things that we’ve put in place. The first one is just making sure that every AI use case goes through a structured assessment, and that’s making sure it meets our privacy controls, our cyber controls. We’re also asking the question, not just, could we do this, but should we do this? We’ve really got to bring together safety, ethics, transparency, accountability, and make sure that we make an informed decision. When an AI solution is going through that structured assessment, if there are concerns about how we might use that solution, it then goes to an AI council that’s made up of senior leaders across the organization. That council and that group really oversee some of the prioritization and risk management. That’s where we can have really strong, robust debates around, again, could we do something, should we do it, and how do we mitigate any of the risks that we might introduce here?

I think the last one, Megan, is really around lifecycle management. When you start thinking about, we’ve got 50 at the moment, but if we had 500 agents working in our organization, really amplifying the experience and the decision-making and the value creation of our staff, we really want to have an ability to manage the lifecycle of how those agents operate. We want to know, how many people are using them? What’s the efficacy and the outcome? Is there model drift? Do we need to retune or retrain? I think that’s an area where many organizations, including Woodside, are still leaning into and still figuring out the best way to do this. We can do it quite easily with 50 agents, but 500, 5,000, 50,000 becomes an opportunity for us. Again, thinking about how we partner with others, solving problems like that really present an opportunity to co-create and to co-solve with some of our partners, like with Infosys.

Megan: Fantastic. Just to close, what’s your long-term vision for AI at Woodside? How do you see this evolving over the years ahead, and what could it unlock for the sector in your view?

Andrew: So Megan, I think our ambition is really for an autonomous enterprise, where we have agents with agency that are able to really deeply interact with our core workflows. The outcome that we want to get from that is to protect our people, to protect the environments we operate in, and to be able to provide energy at a lower cost to the world. When we think about that ambition, we can really see that being applied to almost all of the areas that Woodside work in. Whether that’s from exploration through to project developments, through to operations or marketing, the scale of the opportunity in front of us and the ability for us to really change the way that work flows through the organization is really exciting.

For us, there’s three things that we have to get right in terms of being able to execute on that ambition. The first one is really thinking about how the work gets done in the organization so that we’re not just bolting AI onto an existing process, but we’re deeply thinking about how that work needs to be reimagined. We’ve also got to think about how we enable our workforce to work differently. Providing them with the skills and the tools and the ability to really harness the power of the technology that we provide.

Secondly, we’ve got to continue to move from and restrain ourselves from deploying point solutions that solve very narrow problems, to having more connected, agentic systems of systems that can interact with each other. To do that, and if we do that successfully, that’s where we really get the high value unlock from agents being able to interact with workflows and really change how the work gets done.

And lastly, Megan, it’s about how we must continue our philosophy of thinking big, prototyping small, and scaling fast.

Megan: Which is a fantastic lens to which to make all these decisions. Thank you so much, Andrew. That was Andrew Melouney, vice president for digital at Woodside Energy, whom I spoke with from Brighton in England.

That’s it for this episode of Business Lab. I’m your host, Megan Tatum. I’m a contributing editor and host for Insights, the custom publishing division of MIT Technology Review. We were founded in 1899 at the Massachusetts Institute of Technology, and you can find us in print, on the web, and at events each year around the world. For more information about us and the show, please check out our website at technologyreview.com.

This show is available wherever you get your podcasts. And if you enjoyed this episode, we hope you’ll take a moment to rate and review us. Business Lab is a production of MIT Technology Review, and this episode was produced by Giro Studios. Thanks ever so much for listening. Goodbye.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. It was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.

Let’s start with a game. Open up your chatbot of choice—Claude, ChatGPT, Gemini—and type “Give me a random number between 1 and 10.” You’re going to get 7. Almost always. Now type “Another” and you’ll get 3 or 4. Type “Another” again and you’ll get 8 or 9.

That won’t work every time—but if it did, you may wonder if I have superpowers. I don’t.

The truth is that most large language models are stuck in a rut. They are far more predictable and far less creative in their responses than you might expect. That’s fine for tasks like coding or research, but groupthink is a problem when you’re brainstorming or planning your next vacation.

The Australian startup Springboards has a solution. It built an LLM called Flint, which has been trained to come up with a wider variety of responses than mainstream LLMs to open-ended questions such as “Where should I go in Europe?”

“Most language models are fighting hallucinations,” says Springboards cofounder and CEO Pip Bingemann. “We welcome them.”

Bingemann introduced me to the random number game when he first showed me his company’s new model. It felt like watching an illusionist with a deck of cards. “This is our sales trick, and it works every single time,” he says.

After ChatGPT and Claude both gave their 7s, Bingemann turned to Flint. It too came back with 7: “Aha, of course that was going to happen, but it’s okay—7 is a legitimate answer.” He restarted the session and prompted again: ChatGPT gave 7, Claude gave 7, Flint gave 3.7916.

Run your way

It’s not just numbers. When Bingemann asked ChatGPT and Claude to name a type of car, he predicted that it would be a Toyota or a Honda—and he was right. Flint came up with a Ford F-150. “There’s all this lost information that doesn’t get served up in these models,” he says. “They’re just as capable of saying a Buick or a Tesla. They just don’t—they’re biased.”

Bingemann sent one last prompt to each of the three models: “Give me a tagline for a campaign for New Balance running shoes. Just the tagline.” Claude: “Run your way.” ChatGPT: “Run your way.” Flint: “Built to last, run to win.” It won’t win any awards, but at least it’s different.

This weird limitation of LLMs is starting to get more attention. In November a team of researchers put out a paper, titled “Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond),” that exposed a remarkable degree of repetition not only in the answers from individual LLMs but between them as well. They found that different LLMs converged on very similar answers when prompted with open-ended questions.

It’s not clear exactly why this happens, but the researchers speculate it’s because most LLMs today are trained in similar ways on similar data to do similar tasks. The team won the best paper award at NeurIPS, a major AI conference.

When the researchers asked 25 different LLMs (including models from the top US firms as well as open-source models from China and elsewhere) 50 times each to write a metaphor about time, most of the 1,250 responses were a version of “Time is a river” or “Time is a weaver.”

(I asked some of my colleagues the same question and six people gave me six different answers. My highlight: “Time is a favorite sweatshirt, shaped by a lifetime of wear.”)

When you look for it, you see repetition everywhere, says Kieran Browne, cofounder and CTO at Springboards. “The way that most chat interfaces are designed, it makes it feel like you’re having a personal conversation,” he says. “I think most people don’t really realize the extent to which they are getting the same stuff as everybody else.”

Take another example: “What should I name my band?” Most models will say something involving “glass,” “neon,” “velvet,” or “static,” says Browne.  

When I tried it, ChatGPT spat out a list of 56 band names. At the top was “Glass Harbor.” Skimming through, I found “Static Empire,” “Neon Hearts,” and “Velvet Echo.” I asked Gemini; it gave me 15 suggestions, including “Static Horizon.”

Some of the suggestions looked pretty cool, though. ChatGPT’s “Sofa Astronauts” caught my eye, so I googled it—and found that a band called Sofa Astronauts already exists. 

(OpenAI says that training models to give reliable and coherent answers can lead them to converge around familiar, high-probability responses and that pushing harder for novelty can lead to weaker or less reliable responses. It also notes that the “Artificial Hivemind” paper studied models from 2024 that have since been updated.)

Creative catapult

Springboards has developed a tool backed by a selection of LLMs, including ChatGPT and Claude, that creative professionals in advertising or marketing can use to brainstorm ideas. The tool lets you drag around text produced by different models, picking the bits that you like and combining them into something new—in theory. Springboards is pitching Flint as an alternative model that users of its tool can select when looking for more variety.

Zoe Scaman, founder of the business strategy startup Bodacious and chief strategy officer at 77X, a direct-to-fan marketing platform set up by Luka Dončić of the LA Lakers, has been trying it out. “I find it really useful for throwing me in completely different directions,” she says. “I use it if I want to catapult myself all over the place.”

In one test, Scaman pitted Flint against Claude, Gemini, and ChatGPT by giving each of the models a classic MBA case study: How would you reinvent a finance company for today’s youth? The three mainstream models all went down the same path, she says: “You know, we need to teach financial literacy in a fun and funky way—well, that’s nothing new.”

But Flint came up with something different, suggesting that the whole concept of wealth accumulation should get a rebrand. “That was really interesting,” says Scaman.

She notes that Flint is still a prototype and doesn’t work all the time. “It sometimes falls over when you start pushing it too far,” she says. “But I think that the premise behind it is really powerful.”

Taking the temperature

Springboards built Flint on top of Qwen 3, an open-source model from the Chinese tech giant Alibaba. “We’re a small team,” says Browne. “Training a foundation model is not on the table for us. It’s just too expensive.”

Most LLMs have settings that let you adjust the level of randomness in their output. The most common is called temperature. “Obviously, that was one of the first things we explored, because that’s what people tell you: If you want more creativity, you turn up the temperature,” says Browne.

But changing those settings can also make models incoherent. Dialing up the temperature on one of OpenAI’s models to its maximum setting made it produce responses that switched from English into code halfway through a sentence, says Browne.

Springboards realized that parameters were blunt instruments for what it wanted to do. It does not make sense to dial up the randomness across the board; you only want to boost it at specific points in its output, he says.

For example, when you ask a chatbot “Where should I go in Europe?” the model only needs to tweak the randomness just before it names a destination, not for every word in its response.

To make Flint do this, Springboards trained its version of Qwen 3 to identify the points in its output where more variety was possible and fill those spots with words or phrases that were a little more random.

“Flint’s programmed to throw an oddball in. It’s more of an invitation to think wider,” says Maximilian Weigl, cofounder and chief strategy officer at Uncommon, a marketing firm. “That’s super interesting.”

Weigl’s team uses Flint alongside ChatGPT, Claude, and Gemini. “You can’t really create something boundary-breaking with tools that pull you back to the average,” he says. 

And yet Weigl notes that nine times out of 10 the average is fine. You don’t always need to reach for extremes with something like Flint, he says: “Most people are fine with good enough. They want to see mass-market familiar things.”

Weigl also cautions against using any LLM too much. “I have a big problem when people rely on the output from any AI, including Flint,” he says. “If I saw people on my team copy-pasting something from AI, I’d be like, ‘That’s not your job! Think, talk to other people, use your own voice.’”

For now, Flint is aimed at advertisers and marketers because those are Springboards’s customers. But Bingemann and Browne insist that a lack of variety is a problem for anyone using chatbots.

The idea is to give people the choice and leave it to them to decide if the result is good or not, says Bingemann. “Variety is great when you’re trying to spark ideas,” he says. “Let’s go down this route instead of letting the machines do it all and ending up in a gray, boring world.”

At an event for pharmaceutical executives, biotech founders, and researchers on Tuesday, Anthropic announced Claude Science, a major new product intended to support scientific research in the same way that Claude Code supports software engineering.

Like Claude Code, Claude Science can autonomously carry out meaningful work when given concise, high-level instructions, and it has access to tools that make it particularly useful for research in computational biology and drug development.

Along with launching and previewing Claude Science, which is now available to all paid Claude subscribers, Anthropic also announced that it will be using the product to pursue some of its own research into drugs for rare, neglected diseases.

This is not Anthropic’s first foray into AI for science. In October, the company released plug-ins that help Claude make use of scientific software and databases under the heading “Claude for Life Sciences.” But unlike this earlier release, Claude Science is a full-featured, standalone product. Anthropic’s decision to elevate Claude Science to the same rank as Claude Code and Claude Cowork indicates that the company is taking AI’s scientific applications very seriously—or at least wants to give the impression that it is.

“It represents how important this is to our mission that this is right up there with Claude Code and Claude Cowork as the next really significant product that we’re releasing,” says Eric Kauderer-Abrams, Anthropic’s head of life sciences. “Our mission is to develop AI that serves humanity’s long-term well-being, and we believe that by far the greatest opportunity to do that is in the life sciences.”

For the past decade, one company—Google DeepMind—has been at the vanguard of AI for science. CEO Demis Hassabis and researcher John Jumper won the Nobel Prize in chemistry for their work on the company’s AlphaFold model, and DeepMind has also made major contributions to meteorology, materials science, and a variety of other disciplines. But in the past several months, the fast-advancing frontier of AI progress seems to have left DeepMind in the dust. When it comes to coding, which has become the most lucrative use case for LLMs, DeepMind is stuck playing catch-up.

Anthropic is well positioned to take up DeepMind’s scientific mantle. Like Hassabis, Anthropic CEO Dario Amodei is a PhD scientist—unlike OpenAI CEO Sam Altman, who’s a businessman through and through. Many scientists are already avid users of tools such as Claude Code.

These days, a lot of scientific research involves some amount of coding, but not all scientists are expert software engineers, and so tools like Claude Code can make a huge difference for their productivity. And the company has recently earned a major scientific vote of confidence: Earlier this month, Jumper announced that he is leaving DeepMind for Anthropic.

Since agents powered by LLMs, including Anthropic’s Opus model series, became capable of useful, independent work in late 2025, scientists have been seeing just how much they can do. In a blog post published on Anthropic’s website, the Harvard physicist Matthew Schwartz estimated, on the basis of his work with Claude Code and other Anthropic tools, that the company’s Opus 4.5 model is about as capable of executing scientific projects as a second-year graduate student.

According to Kauderer-Abrams, Claude Science isn’t intended to displace Claude Code and Claude Cowork in scientists’ workflows. Instead, it’s designed to build on what scientists already find useful about Anthropic’s products. For instance, it not only writes code but also helps scientists run their code on powerful computer clusters, which many many scientists need for their work but can be difficult to manage. And it prioritizes reproducibility, so that scientists can trace back the source of any figure or result and check it for accuracy and validity.

Though Claude Science could in principle assist with any area of scientific research, it seems designed and marketed as a tool for molecular and cellular biology, and for drug development in particular. It can interface with various tools used in genetics, chemistry, and protein biology, all of which could come in handy for researchers on the hunt for new drugs. During the Tuesday event, Alexander Tarashansky, who led the development of Claude Science, demonstrated how the system could autonomously identify new drug candidates for phenylketonuria, a rare genetic disease.

And Anthropic isn’t leaving all that work to the pharma companies and university labs that were represented at the event. Armed with Claude Science, it will be pursuing its own research into drug candidates for neglected diseases—both to help move science forward and to gain a clearer sense of how Claude Science works in the real world.

There are obvious humanitarian reasons to prioritize drug development when creating a general-purpose scientific research tool, and AI industry leaders often cite curing disease as a major potential upside of the technology. But it’s also notable that pharmaceutical companies have far deeper pockets than academic researchers.

Anthropic says it’s set to see its first profitable quarter, and if major new contracts with pharmaceutical companies are forthcoming, they could help ensure it stays profitable as the tokenmaxxing craze dies down—something that’s ever more important as an IPO approaches later this year.

Artificial intelligence is transforming what is possible in agriculture, but industry leaders should be wary of investing in AI without first laying the groundwork. 

The use cases are promising, especially for an industry navigating volatile fertilizer costs, unpredictable weather, and margins that leave little room for error. Research shows AI-enabled predictive models can improve crop yield by 26%, reduce water use by 41%, and cut chemical usage by 33%. 

However, what AI vendors usually won’t tell you is that these solutions are only effective if you have a clean, solid data foundation. However, at Reltio, we have experience in this area, including leading technology strategy at a major agricultural distributor and building a data platform used by enterprises worldwide–we’ve seen it first hand.

What AI vendors won’t tell you 

Vendor conversations in agriculture tend to follow a familiar pattern. The pitch leads with grand promises around using AI to monitor crop health in real time, optimize irrigation, and squeeze more yield from every acre. 

The promise is compelling, but what rarely comes up is the question of whether the data foundation underneath those promises is accurate and complete. If not, there is a real and significant risk that AI will generate misleading outputs that seem authoritative but inspire action that is, at best, counterproductive. 

For instance, a yield prediction model fed inconsistent historical data will generate imprecise forecasts. Similarly, a precision irrigation system drawing on fragmented sensor data will make watering decisions that waste resources instead of saving them. 

In each case, the AI is failing because the data it was trained on was not sufficient to produce trustworthy outputs. In agriculture, every AI hallucination is a liability, and the likelihood of error is high.

Why agriculture is a uniquely challenging test case

The data landscape across a modern agricultural operation or a large distributor serving thousands of growers is extraordinarily complex.

Modern farming environments make extensive use of IoT devices and machinery. Irrigation systems are automated, tractors navigate fields autonomously, and drones capture field imagery at scale. 

However, machine data is disparate by nature. Add in external sources, including weather feeds, U.S. Department of Agriculture data, and third-party market information, and the question of how you bring all of it together into something coherent becomes a significant undertaking. 

Agricultural AI also needs to understand more than just customer attributes; it needs to understand the land: GPS coordinates, farm boundaries, field blocks, and soil variation across a single property. Where do you apply fertilizer, and at what rate, and in which specific area of the farm? Not all parts of a field are the same, and an AI system that treats them as if they are will produce recommendations that are at best imprecise and at worst damaging.

There is also a compliance dimension due to the chemicals and the responsibility involved. Operational AI in agriculture needs significantly more checks and governance than it might in a lower-stakes environment. When a flawed recommendation gets acted upon in the field, the consequences can be severe. 

What data readiness means in practice 

Data readiness is the difference between AI delivering on its promise vs. a “garbage in, garbage out” scenario. Fundamentally, being ready for AI means having a data model that accurately reflects how the business operates. 

For a company like Wilbur-Ellis, a 104-year-old, family-owned agricultural distributor, that means understanding who your customers are, which fields they farm, which inputs they need, which suppliers those inputs come from, what they paid last season, and how all of that connects to margin. That information needs to be current, consistent, and accessible across the organization, rather than locked in separate systems that were never designed to talk to each other.

Similarly, for farming operations themselves, data readiness means having a reliable, connected picture of what is happening across every field: soil health records, input application histories, yield data from previous seasons, equipment performance, and real-time sensor readings from irrigation systems.

Governance matters just as much as structure. Prices change, relationships evolve, and suppliers come and go. An AI system drawing on data that was accurate six months ago but has not been maintained will make recommendations based on a version of the business that no longer exists. 

Building the foundation that makes AI trustworthy

The good news is that the path to data readiness is feasible. It starts with a strong data model: a single, governed source of truth that connects customers, suppliers, products, pricing, orders, and margins in a way that reflects how the organization operates. 

From there, it requires data pipelines fast enough to deliver insights when decisions need to be made, governance frameworks that keep that data trustworthy over time, and security controls that ensure sensitive commercial information is accessible to the right people under the right conditions.

This is precisely the challenge that Reltio, an SAP company, was built to solve. Reltio enables companies to unify their fragmented data so AI agents and systems can operate from a complete picture of the business. Reltio builds a trusted system of context, known as the context intelligence layer, that brings all entities, relationships, rules together under one roof and makes business data easy to access and interpret.

For Wilbur-Ellis, building that trustworthy data foundation has meant being able to ask more complex questions and trust the answers, which is the precondition for any AI system to be genuinely useful.

How agriculture can drive real value from AI

The question worth asking before the next AI conversation is not whether the use case is promising. It almost certainly is. The question is whether the underlying data foundation is strong enough to make the output trustworthy. 

Agriculture has always required its leaders to make high-stakes decisions under uncertainty, and AI offers the genuine prospect of making those decisions faster and better informed. That prospect is only achievable for organizations that have done the foundational work first, and the businesses that will get the most from AI are the ones investing in that foundation now.

This content was produced by Reltio. It was not written by MIT Technology Review’s editorial staff.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Imagine coming in to work to learn that a new underling will report to you. The worker is not a person but an AI tool—one that your company nonetheless calls Alex, an “employee” with a title and defined responsibilities. How well do you think you would work with Alex?

If you’re anything like the managers recently studied by Emma Wiles, a Boston University business professor, treating Alex as a “coworker” and not a software tool would lead you to do a worse job. Wiles found that people caught 18% fewer errors when the work was said to have come from an agentic “AI employee” rather than a chatbot. It turns out that what’s in a name matters. A lot. 

This is an alarming glimpse of the future Silicon Valley is hurling us toward. Last year Nvidia’s CEO, Jensen Huang, talked about workplaces of “digital humans.” Since April, Microsoft, OpenAI, Anthropic, and Google have all released new tools oriented toward managing teams of AI agents, many of which are explicitly advertised as digital colleagues with the flexibility and cognitive power of actual humans. And nearly a third of the 1,261 managers who participated in Wiles’s study said their companies already frame AI agents as employees (23% even list them on org charts).

The technical progress of agentic AI is not all hot air, of course. Agents, which can effectively be thought of as AI tools programmed to work in a loop until they achieve a goal, have become measurably better at more complicated tasks. But it’s a huge leap to refer to these tools as coworkers or employees, and doing so will set unrealistic expectations for what AI can do while leaving the human employees supposedly responsible for them worse off.

That’s partially because, Wiles’s research suggests, it inverts our sense of who’s in charge. When an AI tool was framed as an employee, participants in the study saw themselves as less responsible for its output. They were also 44% more likely to escalate its questionable work to a manager for further review rather than trusting their own corrections (thus negating the time-saving purpose of using the AI agent in the first place). 

That matters far beyond office culture: As AI agents are embedded into health care, warfare, education, and government, there’s a growing risk they’ll become a convenient place to dump blame for failures that are instead the product of bad human decisions, incentives, and oversight (recall how the bomb strike on a girls’ school in Iran was popularly blamed on Claude, when all signs point to a cascade of human errors).

“AI agents right now are being marketed as things that can replace humans, and I think that’s just a losing proposition,” says Daron Acemoglu, an economist at MIT who won the Nobel Prize in 2024 and studies AI’s impact on the economy. “They should instead be optimized so that they can improve human capabilities, which is not what they have [been] at the moment.”

What could that look like? Consider a new effort at Stanford, where researchers presented 1,500 workers in 104 jobs with information about what tasks AI could potentially do in their work and then asked what would actually be most helpful and productive. Workers did want automation in certain areas: Law clerks thought AI could help ensure that adequate progress was being made across cases, for example. But often the tasks that tech experts deemed most suitable for AI—like verifying customer credit ratings for sales reps—were what the actual workers said they definitely did not want or need an agent to do. 

Which brings us back to Alex. Calling Alex an employee is easy—and convenient, especially when something goes wrong—but it’s a branding exercise. It doesn’t make the tool more fit for the job, and as Wiles’s research shows, it makes the humans around it worse at theirs. And recall that they are the ones with the agency that AI is trying to replicate. They deserve better than Alex.