This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Over the weekend, several current and former advisors to President Donald Trump on AI publicly lobbed insults at the country’s leading AI companies. David Sacks, the president’s AI and crypto “czar” until March, branded Anthropic’s models as “lobotomized” and “woke.” Emil Michael, a top Pentagon official, called OpenAI’s new head of strategic futures a “supreme village idiot.”

It began because no one can agree on what to do about Kimi, a free, open source model that Chinese AI company Moonshot launched last week. It appears to rival the intelligence of models from OpenAI and Anthropic, which are very much not free. 

Kimi and other Chinese models like it pose a real problem for Trump. And they’re dividing the top AI strategists in his orbit into factions. Every time a new smart, free model from China like Kimi gets released, US companies see less reason to fork out money to access models from Anthropic or OpenAI. Given that enthusiasm for these and other AI companies is driving an outsized share of economic growth, China’s AI models create both economic and political problems for the president. They are “a threat for an administration that really doesn’t want more economic bad news,” Anton Leicht, a fellow at the Carnegie Endowment, wrote on X. They’ve already rattled US stocks. 

What is Trump to do? First, consider that this is all happening just a week after New York imposed the country’s first state ban on new data centers. There is growing distrust of AI companies, and I imagine a not-insignificant share of Americans would have little sympathy for OpenAI or Anthropic as they fend off cheaper competitors, and would say it’s not the government’s job to protect their interests.

On this point, they’d see a sliver of agreement (and really just a sliver) with David Sacks, who on July 19 criticized top AI companies that “want the government to eliminate their open source competition.” He has also argued that Chinese AI models have become popular because they come with fewer restrictions on how people can use them (putting aside the built-in state censorship). 

Sacks, however, is out of a job. He no longer has a formal role advising Trump, and his position that more open AI is better has been largely replaced in the administration by one that sees a larger role for government intervention. The thinking behind this view is that because AI models have gotten strong enough to pose threats to national security, the government must control how they’re used. 

This position has fueled the new White House review process that aims to vet AI models’ security before they’re released. Dean Ball, a former Trump AI advisor who now works for OpenAI, criticized it over the weekend as a “de facto licensing regime for frontier AI.” Ball predicted Trump may solve his Chinese open source problem with a bit of soft power, perhaps by making US companies afraid to use models like Kimi. That drew a response from Michael, who, with Secretary of Defense Pete Hegseth, has been the agency’s main liaison with AI companies. Michael called Ball the AI industry’s “supreme village idiot,” bristling at the suggestion that the government would quietly strong-arm companies rather than, as Michael put it, go through “the democratic process not some Deep State scheme.”

Left out of the conversation has been how a model like Kimi got so good in the first place. For much of the Biden administration and even the beginning of Trump’s second administration, keeping China from getting top chips was a priority. Those export controls have loosened—Trump made the controversial decision to allow Nvidia to sell more chips to China, in exchange for the US government taking a cut—and the government has alleged that some chip smuggling has taken place. But China nonetheless has limited computing power, and it’s not clear what chips the company behind Kimi used to train the model. 

It’s possible that the process involved some distillation, a practice in which AI models are trained on the outputs of existing AI models. OpenAI and Anthropic have long complained that Chinese AI companies do this, and they have requested government help to put a stop to it. In April, they got it, when the Trump administration announced a series of efforts to curb the practice.  

But Kimi is out there and free, and it is nearly as good as the Anthropic model the US government deemed so powerful that it was briefly shut down because it threatened national security. The weekend’s sparring suggests many in Trump’s orbit see that as a wake-up call. But nobody can agree on what for.

The machine that could change the world will be housed in a room that looks like a data center crossed with an ice cream factory. Inside will be some 100 stainless-steel cabinets, each about six feet tall and connected to a supply of liquid helium that keeps them only a few degrees above absolute zero. Inside those cabinets will be hundreds of chips, and on those, thousands of particles of light flying through a maze of optical switches and beam splitters. Each photon must be accounted for, because precisely measuring where it ends up will help answer questions that current computers might take millions of years to solve.

This computer, as described, does not exist. It’s the brainchild of a company called PsiQuantum, founded in 2016 by four physicists from UK universities. In a crowded field of deep-pocketed competitors with similarly fantastical visions, the company aims to be first to fulfill its promise.

In the years since the physicist Richard Feynman first envisioned them in 1981, quantum computers have promised to speed up everything from medical research to AI by harnessing the qualities of quantum particles. Unlike normal computer bits, which can be either a 1 or 0, quantum bits can exist in multiple states at once. And combining enough of those quantum bits together could produce a computer capable of tasks well beyond the reach of today’s conventional machines. But even today’s best quantum prototypes are too small and error-prone to do anything useful.

That makes PsiQuantum’s promises for what its computers will ultimately do all the more bold. Consider the company’s hopes for predicting the effects of cytochrome P450 enzymes, which often break down drugs in the body. If pharma companies knew more precisely how they would work on a particular molecule, they could design more effective medications faster. Estimating this for a specific drug can take over 10 years with today’s methods, says Philipp Ernst, vice president of quantum applications for PsiQuantum, but “we aim to get it down to four minutes.”

In a field full of such claims, PsiQuantum has attracted unusual investment and scrutiny for two reasons: It is one of the few companies aiming directly at building a large and useful machine, and it is already working with a major chip manufacturer to build its systems using existing semiconductor fabs. Its vision has attracted momentum: Last year, PsiQuantum raised $1 billion in funding and broke ground in Chicago on a site it’s building in partnership with local governments. It also has a second site in the works in Australia, which it promises will be operational—meaning hardware-ready—in 2027. And it’s one of just two companies (along with Microsoft) to reach the third stage of an intensive government evaluation program to see which quantum companies might succeed.

Evaluating whether PsiQuantum will do what it says is harder than, say, judging a drugmaker by its clinical trial results: Advances in quantum computing are incremental, opaque, and tough to verify from the outside. But the company is now approaching its prove-it moment, when years of closed-door work and hundreds of millions in investment will either culminate in a useful quantum computer or fall short. We could start to know which as soon as next year.

A new kind of machine

Terry Rudolph, one of PsiQuantum’s four founders, is soft-spoken and shaggy-haired. He was born in Malawi and learned only after earning his first physics degree that he is a grandson of the famed physicist Erwin Schrödinger. He later self-published a 150-page book to explain quantum computing to teenagers (my PR contact gave me a signed copy with a wink that said “We never expect anyone to actually read this,” but I can report that it is a funny and helpful book). 

Around 2014, Rudolph and his cofounders became increasingly convinced that the quantum breakthroughs they were finding to be possible in theory might also be possible in a real machine. They eventually left their academic positions and divided the tasks before them: Rudolph worked on theory, Mark Thompson on engineering, Pete Shadbolt on scaling the technology up, and Jeremy O’Brien on articulating the vision and finding investors (O’Brien served as CEO until February; he’s been replaced by Victor Peng, a veteran of the semiconductor industry). 

To understand why the quantum computer the company is building would be a big deal, consider how imprecise much of modern science remains. We cannot reliably predict, for example, which lithium-ion battery will catch fire or how quickly a critical aircraft component will corrode.

This isn’t just because these systems are complex, though they are. It’s that, at their core, they are governed by quantum mechanics. Subatomic particles don’t have well-defined properties—this location and that velocity—but instead occupy quantum states spread across many possibilities. And that in turn influences a range of atomic and molecular behavior. Schrödinger (Rudolph’s grandfather, remember) showed how to describe this haziness mathematically a century ago this year, but precisely carrying out the calculations on real-world systems quickly becomes unfeasible even for the best computers. Scientists cope with this gap using approximations, imperfect simulations, or experiments on animals.

PsiQuantum co-founder and chief scientific officer Pete Shadbolt (left), and machinery the company has built to manufacture its own barium titanate, a material with the perfect qualities for routing light particles (right).

Feynman, David Deutsch, and other physicists in the 1980s wondered if we could do better. Maybe such complexity could instead be modeled using a new kind of machine. Rather than using transistors that are only ever on or off, this one would use particles held in quantum states, manipulate them to perform calculations, and then measure them at the end for an answer. Using quantum systems to simulate quantum systems would for the first time allow a simulation of physics and chemistry that directly reflected reality. It would be an invaluable tool for designing new drugs, materials, or really anything affected by quantum mechanics. Revolutionary, in other words.

Humankind’s leaps in understanding how nature works have often resulted in the invention of powerful new tools, Rudolph told me. “I don’t think it’s a coincidence that the Industrial Revolution coincided with our ability to calculate and simulate the laws of Newtonian mechanics, the laws of thermodynamics,…the laws of classical electromagnetism,” he says. “Whenever we have more power to calculate and simulate and understand things, we build incredible machines that come from it.” He sees something similar coming with quantum computers.  

Chasing photons

One mystery has always been which quantum thing—ions, atoms, or something entirely new engineered with quantum properties—could be made stable and controllable enough to use as a qubit, the basic unit in the quantum computing world. Quantum systems are delicate, and observing any particular particle causes it to collapse into one state rather than a superposition of multiple states. If this happens during the computation rather than at the end, it produces an error that must be corrected for. Too many of these means the computer fails to produce a useful answer. 

Just as engineers in the early days of aviation weren’t sure whether airplane wings would be fixed or flap like a bird’s, we’re not yet sure which of these quantum things will work best. Google and IBM are betting on superconducting qubits, superconducting circuits made of aluminum or other metals. Intel is using electrons. PsiQuantum is using photons, the particles that make up light.

“Photons have lots of nice things going for them,” Rudolph says. They can maintain quantum states for a long time; indeed, the photons in the universe’s cosmic microwave background may have done so for billions of years. But photons also move fast and scatter easily. More importantly, two photons are more likely to pass through one other than interact. That makes them a challenging candidate for quantum computation, in which qubits need ways to influence one another. 

For a while, this last flaw seemed to doom the idea of quantum computing with light. But in 2001, researchers from the Los Alamos National Laboratory and the University of Queensland found a loophole. They discovered they could essentially fake interactions between photons by sending the light particles through a network of beam splitters and detectors. Their paper changed everything. PsiQuantum was created to make the theory a reality.

Size was the first problem; previous plans would have required a computer as large as California. Mercedes Gimeno-Segovia, who was a PhD student of Rudolph’s in the early 2010s (after almost becoming a professional violinist instead), thought of a way for the machine to be smaller. 

The basic process since then has been this: First create photons with lasers and then “entangle” them, exploiting a quantum phenomenon in which the particles no longer have individual states but instead share one. Next, route them through a maze of gates that perform computations, and finally read out details of their quantum state at the end, all while tracking and correcting for the errors that occur. Succeeding at each of these steps millions of times is not so much an engineering hurdle as a brick wall. And building the supply chain—like manufacturing new materials with the qualities to route individual photons around—is arduous.

To get a sense of it all, last year I joined Shadbolt at the SLAC National Accelerator Laboratory, in Menlo Park, California. The center has helped produce several Nobel Prizes and played a role in the 1968 discovery of quarks, fundamental building blocks of matter that make up protons and neutrons. But PsiQuantum set up shop there essentially to siphon liquid helium from SLAC’s giant cryoplant. This is what the company uses to cool its computing cabinets down to deep-space temperatures.

Right now the cabinets operate at 2 K, or -456 °F, but the goal is to be able to run them slightly warmer—at a balmy -452 °F. Most quantum approaches require the whole machine to be cooled to superconducting temperatures, so that much of the expense in running it will actually be spent on refrigeration. But photonic computers require only one piece to be this cold—the detectors that measure single photons at the end of the computation. And the required temperature can be a bit higher. (PsiQuantum said in May that it will spend some of the $100 million award in CHIPS Act funding it’s slated to get on these detectors). 

The siphoning setup was a temporary solution; PsiQuantum now has its own cooling system at its testing facility in Milpitas, California, and is setting up a larger one at its production site in Australia next year. These helium systems represent some of the biggest capital expenditures for any quantum company and will consume a significant chunk of PsiQuantum’s $1 billion funding round.

In the afternoon we drove to a lab in San Jose, where I donned a cleanroom suit—a head-to-toe covering that keeps dust at bay—to watch the manufacture of a blueish crystal called barium titanate. 

It’s prized by PsiQuantum because it quickly and reliably routes light particles with very little electrical input, keeping the precious photons undisturbed as they move through the circuit. But for all barium titanate’s theoretical value to the company, its structure makes it a pain to manufacture, and the material wasn’t available at scale when PsiQuantum got its start. The company, in what Rudolph told me was an agonizing decision, opted to make it in-house, requiring a massive investment. I saw a technician—operating at what looked like a giant pressure cooker—adding the base elements to several hoppers; then I watched through a porthole as the elements got heated, vaporized, and finally crystallized into a thin layer on a wafer disc. At that time each disc took about 12 hours to make; the company now says several are produced each day. The discs then get shipped to the chipmaker GlobalFoundries in Malta, New York, where PsiQuantum’s chips are made.

The company has invested heavily in making its own barium titanate, a material whose delicate crystalline structure is tedious to manufacture.

PsiQuantum’s bet is that this entire supply chain, byzantine as it might sound, will make the company more efficient than its competitors. That’s because, if you squint, it looks like a souped-up and high-precision version of the existing supply chain for silicon photonic chips, another type of technology that transmits information with light—one that’s already used in data centers. If PsiQuantum produces its chips at scale, it can take advantage of tools and infrastructure that already exist.

But it’s not a given that one working chip can easily be wired up to thousands more. That’s why the company is testing in phases: Its Milpitas site has connected three cabinets together, with 250 chips in each, but the next step is to scale the systems up and see whether the company’s techniques for correcting errors can keep up. Once the cooling system arrives at the Australian site late next year, the company says, it aims to connect about 100 cabinets together. Then PsiQuantum will work up to running the world-changing algorithms it has promised.

The timeline for this, it’s worth noting, is up for debate. News articles have said that 2027 is the year that PsiQuantum aims to have its first full-scale quantum computer come online at its Australian site, but the company insists the deadline has been misread, and that it only intends for its facility to be “operational” by the end of next year. That means cooling systems in place and ready for hardware to be installed, but no promises about what size computer will be ready. In an industry where timelines are perpetually in flux yet central to how companies are judged, that distinction isn’t trivial.

Into the unknown

The outsider with perhaps the best guess of whether PsiQuantum will succeed is the Pentagon. The US Defense Advanced Research Projects Agency—the Pentagon’s research and development arm—has been running an initiative to determine which of the boastful quantum companies might actually deliver. In the last year and a half, the heads of the program have been sounding more confident. Joe Altepeter, who ran the program until last year and proudly described himself as a “quantum skeptic,” told me in March 2025: “I am more optimistic now than I have been at any point in the past 10 years.” And in a statement earlier this year, his successor, Micah Stoutimore, said “it now seems likely that someone will build a utility-scale quantum computer by 2033,” referring to a machine that generates more value from its calculations than it costs to build and operate. 

The program has been scrutinizing PsiQuantum’s systems since 2023, and last year placed the company into the third stage of a benchmarking initiative meant to determine whether the technology will actually work. But to the rest of the industry, PsiQuantum is sort of a black box.

“It is very hard for an outsider to evaluate,” says Scott Aaronson, a theoretical computer scientist at the University of Texas at Austin who runs a popular blog that often covers the industry. Other companies, like Google and Quantinuum, have regularly published results over the years demonstrating chips and systems with incremental improvement, publicly laying the engineering groundwork needed to eventually build large machines.

PsiQuantum has instead focused squarely on a commercial goal—a computer with one million qubits, which is the scale that researchers expect to unlock research currently not possible on normal computers. PsiQuantum often differentiates itself with this industrial-scale goal, but IBM, which debuted a development road map in 2020, has been progressively building bigger and bigger systems. It initially targeted 2028 for a large-scale, error-corrected system, a deadline that now appears to have been pushed out to 2030.

Making it useful

On top of actually building the machine, a major focus for PsiQuantum is getting the rest of the world to develop a plan for how to use it. PsiQuantum has announced partnerships with customers including the defense giant Lockheed Martin, which intends to use it for materials design; the automaker Mercedes, which wants it for battery design; and the aerospace manufacturer Airbus.

That these companies don’t have a computer to experiment with is not a problem, according to Ernst at PsiQuantum. “There’s a PlayStation 6 probably coming up from Sony next year or the year after, and people are programming those games right now,” he says. “This is, in principle, very similar.” (It’s a glib analogy but not an entirely empty one; the quantum algorithms for solving a research problem can be cracked even if there is not yet hardware to run them on.) 

The idea is that experts in quantum information from both PsiQuantum and its customers will be able to translate design problems—say, the requirements for a battery in a Mercedes electric vehicle—into algorithms the computer could solve. The company offers a software package called Construct, which companies can use to design their own algorithms that might one day run on the computer.

The future of quantum computing hinges on these algorithms. Quantum computers get painted as a speedup for everything, but in reality, they’re suited to a subset of problems, and answering a question with this sort of machine requires the question to be formulated with very specific types of algorithms. People spend entire careers working on such algorithms, even if the computers to run them don’t exist yet. At their core, they use the rules of quantum mechanics to manipulate probabilities in ways that ordinary computers can’t. 

The most famous example, and a reason the government is so interested in quantum computers, is Shor’s algorithm. It was developed in 1994 by the theoretical computer scientist Peter Shor and could effectively break many forms of encryption used online, for everything from credit card numbers to military intelligence. The thing keeping the world together, for now, is that nobody has a computer to run the algorithm on (and security experts are already launching new encryption methods that could withstand attacks from a quantum computer). PsiQuantum is researching how long its systems might take to run Shor’s algorithm.

PsiQuantum’s chips are manufactured at GlobalFoundries in Malta, New York, and tested at company headquarters in California. Both PsiQuantum and GlobalFoundries have been awarded federal CHIPS Act funding.

The company also published a paper in December in collaboration with Airbus, essentially seeing if a new algorithm developed by the authors could beat a classical computer in modeling fluid dynamics, like the turbulence around an airplane wing. Andrew Childs, an expert in quantum simulation, told me PsiQuantum achieved only a moderate speed increase over what today’s computers can do. “It’s probably unlikely that speedups like this will have a significant practical impact until we have very large-scale quantum computers,” he said in an email. (When I asked Ernst, he agreed the improvement was modest.)

Some of the algorithms PsiQuantum is working on are not expected to be perfected or even used in the first applications of its computer. Instead, its initial tasks might be more along the lines that Feynman envisioned way back in 1981: simulating the smallest particles of our world. 

The company’s most significant research in this realm is in modeling quantum chemistry. Take those pesky P450 enzymes. More precisely understanding how they operate, PsiQuantum says, would allow for faster drug development and testing.

Last year, PsiQuantum published methods for doing these sorts of chemistry calculations on a quantum computer, along with another paper demonstrating an algorithm that can simulate the collision of two molecules and estimate the likelihood of different outcomes femtosecond by femtosecond (there are one quadrillion femtoseconds in a second). It’s a remarkable amount of detail not currently possible with today’s technology, and it would allow drug and materials researchers to simulate new chemical interactions. 

Dominic Berry, who developed some of the core techniques used in the collision paper but isn’t involved in PsiQuantum, says the company made impressive improvements, but to do the simulations scientists are most curious about would require the algorithm to be made even faster and PsiQuantum’s early computer to have fewer errors than currently expected.

Until PsiQuantum’s computers are up and running, the breakthroughs that these research papers tease remain in the realm of theory. It’s a space where Rudolph operates quite comfortably. He told me that Alan Turing created the theory of classical computing with pen and paper, imagining how the 1s and 0s would be represented in the machine, and how with the right approach to logic you could compute almost anything. 

“But there is no way that by hand, with a pen and paper, Turing was ever going to produce—you know—Minecraft and Facebook,” he says. That took more than 70 years of tinkering (during which we fortunately created more useful things than Minecraft and Facebook).

For all the time Rudolph spends dreaming up things quantum computers might do, in other words, people working on those problems are still stuck with pen and paper for now: “Until you have the actual machine in hand, you don’t have the opportunity to really explore its potential.”

This story was updated on July 14 to clarify how long DARPA’s quantum program has been evaluating PsiQuantum.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Anthropic—currently the world’s most valuable AI company, with a nearly $1 trillion valuation—has a reputation for publishing strange and heady research. It’s looking into whether AI models can feel pain, for example, and will sometimes cut off chatbot conversations if it suspects users are “abusing” the model. 

One niche that Anthropic spends more time and money on than other AI companies is called mechanistic interpretability, which means looking inside the complex math of an AI model to learn why it comes up with one particular output and not another. It’s complicated stuff; there are millions of data points that might contribute to any result, and wading through them can look more like word salad than anything useful. It’s also controversial. Describing AI models with terms borrowed from psychology and neuroscience can make their behavior seem more sophisticated than we might otherwise judge it to be.

That’s why, when Anthropic announced last week that it had found a new window into its models’ “internal thoughts” as they reason through answers, there was one colleague I had to talk to. Senior editor Will Douglas Heaven, aside from having a PhD in computer science, has spent a lot of time digging into what we can say about how AI models work. I spoke with him about what we should take from Anthropic’s new (and predictably quirky) research.

What did Anthropic learn here, exactly?

Anthropic has been trying to understand how large language models (LLMs) work for a few years now. Anthropic isn’t the only one looking at this, but I think the company has made it part of its core mission more than most. Anthropic’s CEO, Dario Amodei, has said we won’t be able to control LLMs fully unless we learn more about how they work. 

So this new research is very much in that context. It goes deeper into the weird mechanisms inside LLMs than ever before. What Anthropic learned was that LLMs have a space inside them—which Anthropic calls the J-space—filled with words that don’t appear in their output but that seem to influence the way they puzzle through problems. All this was hidden until Anthropic developed a new technique to probe its model Claude, so it’s a genuine discovery. 

Sometimes these words keep track of where the LLM has got to in a particular task, sometimes they look more like flashes of recognition (for example, “protein” might pop up when you give an LLM only the letters of a protein sequence), and sometimes they represent a kind of internal commentary on the model’s decision-making. In my favorite example, Claude decided to cheat on a coding test when the word “panic” appeared.

Anthropic also found that LLMs are able to describe and manipulate the words in this space. So somehow they seem to be making use of it. 

Let’s step back for a second. I don’t think of large language models as simple, but they’re also not magic. There’s a bunch of math that learns relationships between words, right? So why is it so hard to “peer” into an LLM to know what’s going on?

Yeah, they’re not magic! I think the fact we don’t fully understand them plays into the mythmaking. And it’s worth noting that the whole narrative that Anthropic is leaning into here—that they’ve built this really mysterious technology, but don’t worry, because they’re also the ones to figure it out—very much fits with the company’s vibe. [See how Anthropic warned that its new models were so good at coding they posed a global cybersecurity risk, only for the US government to shut them down shortly thereafter.]

So yes: LLMs are just math. And yet it’s vastly complex math. Not only are today’s LLMs made out of hundreds of billions of numbers, but running them triggers a cascade of millions and millions of calculations. I wrote last year that if you printed out even a medium-size LLM on pieces of paper, it would cover a city the size of San Francisco. 

It’s impossible to make sense of any of that math without specialist tools that highlight specific parts of an LLM at specific times. You need to know where to look and how to look. And building those tools requires understanding something of that complex math in the first place. 

You’ve written elsewhere about this concept of studying LLMs the way one might study an organism’s brain. Is it fair to use “brain-like” terms when talking about how an LLM works?

I don’t love using those kinds of terms. LLMs are not brains. Talking like this is misleading because it can suggest that LLMs are capable of more human-like things than they are or that we can make assumptions about how they might behave that we shouldn’t. The whole anthropomorphization thing is also tied up with a bunch of strong ideological positions about what this technology is and what it’s going to be. 

But at the same time, we lack a good alternative vocabulary for talking about what these models are doing. I can understand why people reach for words like “think” and “understand” and “brain-like”—they’re convenient shorthand. 

Anthropic compares this new space it found inside LLMs to the space that some neuroscientists think our brains use to keep track of conscious thoughts. I asked the company how seriously we should take that comparison and it said in a statement: “Drawing these analogies was helpful to us in designing our experiments, as they allowed us to make many non-obvious experimental predictions about the J-space that turned out to be true. At the same time, it’s important to note that there are some important differences between the J-space (and language models in general) and the human brain, so we don’t mean to claim there’s a perfect correspondence.” 

What’s a problem in AI that this new concept of the J-space might be used to solve?

Anthropic has said that monitoring the J-space could be a way to catch models doing something they shouldn’t. Because words pop up in this space that don’t appear in a model’s output, they can tell you things about its behavior that you might not have noticed otherwise—such as when it is giving biased responses or when it is weighing the pros and cons of cheating. 

That’s the theory, at least. I think it’s better to think of this result as one more step on the path to understanding this technology overall than as something that will be useful by itself. 

Read more in Will’s full story about the new research. 

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

OpenAI CEO Sam Altman’s oft-discussed promise that Americans will share in the wealth AI creates was in the news again last week. On Thursday, the Financial Times reported that Altman is in talks with President Trump about giving the US government a 5% stake in OpenAI.

In some ways, Altman’s plan is old news. He wrote about a more radical version of this back in 2021, proposing that all companies above a certain valuation (not just AI companies) pay 2.5% of their market value each year into a fund that sends Americans annual disbursements. In April this year, OpenAI described a narrower proposal that closely resembles what Altman is reportedly discussing with Trump now. And the notion has broad political appeal: Senator Bernie Sanders has proposed giving Americans a 50% stake in top AI companies.

What’s the logic here? For would-be recipients, it’s twofold. First, AI learns directly from human-generated work—books, movies, art—but AI companies generally never pay the authors of that work. A free equity stake could serve as a form of belated compensation. Second, the payout could mitigate the widespread anxiety that AI will cause a collapse of the labor market (even if economists disagree) by providing a safety net. 

How large a safety net is up for debate. Details of OpenAI’s latest proposal are sparse, but let’s say the government were to distribute this equity stake directly to Americans. After its funding round in March the company was valued at $852 billion, making a 5% stake in OpenAI worth about $42.6 billion today (the company is reportedly delaying its IPO until it can reach a $1 trillion evaluation, a tall order given that it’s spending heavily on data centers and still has not turned a profit).

Distributing that $42.6 billion equally among the roughly 133 million American households would give each about $320 in equity. But if it were to operate like other wealth funds, the government would not give equity directly to Americans but rather let the fund grow and then share a portion of the returns with everyone, perhaps delivering a bigger payout, if and when AI companies can ever start sustainably turning a profit.

If this dividend does materialize, what’s in it for tech companies? Altman might hope the promise of payouts could help swing public opinion a bit more back toward AI companies. (A majority of Americans don’t trust companies to use AI responsibly and oppose construction of data centers in their area, and half are more concerned than excited about the increased creep of AI into their daily lives.)

But the bigger prize for OpenAI might be that the Trump administration loves making tech deals—like its equity stake in Intel and its share of Nvidia’s sales to China, among others.  Staying on the administration’s good side is pretty essential for AI companies right now (just ask Anthropic). It could mean not having your models deemed a supply chain risk, or getting more help from the White House in stopping your rivals from China. 

My main takeaway is that these plans currently function more as a story than a policy. Altman has been talking about some version of this idea for five years and reportedly pitched it to President Trump soon after he took office, yet there is still little indication that a concrete plan is taking shape. The more ambitious proposal from Sanders is even less likely to gain traction.

But what these plans do reveal is just how up for debate the future of AI still is. Altman drew inspiration for his plan from the Alaska Permanent Fund, which was set up in the 1970s to give Alaskans a share in oil profits. The idea was based on two premises: that oil is a shared resource, and that eventually it will run out. Altman seems happy to concede the first claim about AI. But he’d balk at the second, having promised that AI will generate extraordinary wealth for decades to come. Whether Americans ever receive a check is beside the point; the proposal’s real purpose may be to convince them that the AI boom will be large enough to share.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Imagine coming in to work to learn that a new underling will report to you. The worker is not a person but an AI tool—one that your company nonetheless calls Alex, an “employee” with a title and defined responsibilities. How well do you think you would work with Alex?

If you’re anything like the managers recently studied by Emma Wiles, a Boston University business professor, treating Alex as a “coworker” and not a software tool would lead you to do a worse job. Wiles found that people caught 18% fewer errors when the work was said to have come from an agentic “AI employee” rather than a chatbot. It turns out that what’s in a name matters. A lot. 

This is an alarming glimpse of the future Silicon Valley is hurling us toward. Last year Nvidia’s CEO, Jensen Huang, talked about workplaces of “digital humans.” Since April, Microsoft, OpenAI, Anthropic, and Google have all released new tools oriented toward managing teams of AI agents, many of which are explicitly advertised as digital colleagues with the flexibility and cognitive power of actual humans. And nearly a third of the 1,261 managers who participated in Wiles’s study said their companies already frame AI agents as employees (23% even list them on org charts).

The technical progress of agentic AI is not all hot air, of course. Agents, which can effectively be thought of as AI tools programmed to work in a loop until they achieve a goal, have become measurably better at more complicated tasks. But it’s a huge leap to refer to these tools as coworkers or employees, and doing so will set unrealistic expectations for what AI can do while leaving the human employees supposedly responsible for them worse off.

That’s partially because, Wiles’s research suggests, it inverts our sense of who’s in charge. When an AI tool was framed as an employee, participants in the study saw themselves as less responsible for its output. They were also 44% more likely to escalate its questionable work to a manager for further review rather than trusting their own corrections (thus negating the time-saving purpose of using the AI agent in the first place). 

That matters far beyond office culture: As AI agents are embedded into health care, warfare, education, and government, there’s a growing risk they’ll become a convenient place to dump blame for failures that are instead the product of bad human decisions, incentives, and oversight (recall how the bomb strike on a girls’ school in Iran was popularly blamed on Claude, when all signs point to a cascade of human errors).

“AI agents right now are being marketed as things that can replace humans, and I think that’s just a losing proposition,” says Daron Acemoglu, an economist at MIT who won the Nobel Prize in 2024 and studies AI’s impact on the economy. “They should instead be optimized so that they can improve human capabilities, which is not what they have [been] at the moment.”

What could that look like? Consider a new effort at Stanford, where researchers presented 1,500 workers in 104 jobs with information about what tasks AI could potentially do in their work and then asked what would actually be most helpful and productive. Workers did want automation in certain areas: Law clerks thought AI could help ensure that adequate progress was being made across cases, for example. But often the tasks that tech experts deemed most suitable for AI—like verifying customer credit ratings for sales reps—were what the actual workers said they definitely did not want or need an agent to do. 

Which brings us back to Alex. Calling Alex an employee is easy—and convenient, especially when something goes wrong—but it’s a branding exercise. It doesn’t make the tool more fit for the job, and as Wiles’s research shows, it makes the humans around it worse at theirs. And recall that they are the ones with the agency that AI is trying to replicate. They deserve better than Alex.