Consider what passes for success in technology policy. An AI company promises that its chatbot will not encourage suicide. A social media platform makes it more difficult for children to buy fentanyl. These are important protections, and they should have existed all along. But companies can clear these low bars while continuing to make products that leave young people dependent on simulated friendships or trapped in feeds built to keep them scrolling.

We have historically allowed the debate over technology to be defined by the worst thing a product might do, with remarkably little discussion of the insidious damage it does every day. As we transition into an era defined by powerful AI products, we should demand that the tech industry do more than merely address catastrophic harms. We should take measures to ensure that its products don’t erode how we think, learn, and interact with one another.

 A teenager who develops an unhealthy attachment to an AI companion may never appear in a lawsuit or a newspaper, except in the most egregious cases. Neither will the student who begins relying on a chatbot to write every paper, or the young person who measures their life against a social media feed engineered to provoke envy and keep them watching. A product can meet every narrow safety requirement and still leave our entire culture worse off in ways that don’t always make headlines.

A narrow definition of success defined around safety is politically convenient for the technology industry, and companies actively promote such definitions. Taking steps to prevent spectacular disasters is good for business: It can mean fewer lawsuits and less pressure for serious regulation. Confronting the everyday consequences of dependence would require something considerably more threatening to their bottom line. It might mean asking companies to add more friction to their products or actively limit engagement or features that lead to negative outcomes, such as addictive attention-hacking algorithms.

Technology giants have instead worked to shape legislation around the harms they are already under direct pressure to address. This year, for example, Google backed bills to regulate AI chatbots in several states. The legislation mandated basic protections against sexual content and promotion of self-harm that tech companies should have established regardless, but with broad carve-outs that could exempt some of the most popular services. Passing these bills allowed legislators to claim a safety win while efforts to promote stronger legislation—laws designed to prevent children from becoming emotionally dependent on AI—failed. This playbook worked in states like Nebraska, producing legislation heavily influenced by industry at the expense of bills that would have done more to hold companies accountable.

We have been through a version of this with social media. Globally, people who use the internet now spend an average of more than two hours a day on social media platforms—time that has been quietly taken from ordinary life and handed over to companies whose profits depend on keeping us there. No one, outside perhaps a few tech CEOs, wanted these algorithms to occupy such a large portion of our waking hours, and their cultural engineering can actually contribute to violence on a devastating scale. Meta’s platforms have been cited for contributing to the Rohingya genocide in Myanmar, anti-Muslim riots in Sri Lanka, and alleged calls for violence during Ethiopia’s Tigray war. The same incentives operate in the feeds that shape young Americans’ understanding of politics, their peers, and the wider world.

AI is extending this logic into parts of life that social media could influence only from a distance. It offers to write our thoughts for us and stand in for the people we might otherwise turn to. A third of teens now choose to discuss serious matters with AI companions instead of other people. MIT researchers found that young people using ChatGPT to write essays showed weaker brain connectivity during the task, and their evidence indicated that “cognitive debt” persisted even after the task was complete. There is plenty we still need to learn about the long-term effects. We already have reason to ask whether technology marketed as making us more capable is eroding the habits and relationships that help us develop capabilities in the first place.

Follow this way of thinking to its logical endpoint and you get a world of perfectly sanitized entertainment, with chatbots ready to spare us the difficulty of thinking or of dealing with another person’s feelings (or even our own). But anyone who has struggled to write something worth reading, or repair a damaged friendship, knows how much gets lost in that bargain. We build judgment through effort and friction, and we learn to live with other people by confronting the fact that they have needs and opinions of their own. 

We should demand safety as a minimum, yes—and then demand considerably more. Technology companies and school districts should have to show beyond a reasonable doubt that AI improves learning and critical thinking before making it a fixture in classrooms. Companies should be prohibited from designing companions that pretend to be conscious or profess love to users, and they should face independent scrutiny of how their systems affect relationships and people’s ability to think for themselves. The companies that have inserted their products into nearly every part of modern life have earned obligations far greater than avoiding the next terrible headline.

Sam Hiner is the cofounder and executive director of the Young People’s Alliance, where he coordinated a coalition of advocates to pass laws regulating AI companions in eight states in 2026—laws such as California’s SB1119. Sam graduated from the University of North Carolina atChapel Hill in 2025 and was named to the 2026 Forbes 30 Under 30 Social Impact list.

Emma Leiken is executive director of the Responsible Technology Youth Power Fund at Hopelab, which has raised more than $9 million for youth-led tech accountability work, and teaches technology policy at the Columbia School of International and Public Affairs. She was previously TikTok’s global head of youth safety and well-being policy and worked on responsible technology at Omidyar Network.

MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next. You can read more from the series here.

Late last month, Anthropic released a report arguing that the model GLM-5.3 poses a major cyberattack risk. The report’s core concern is that the powerful new model, released in August by the Chinese company Z.ai, is freely available for anyone to download and modify. Anthropic’s assertion is only the latest move in a long-running debate about whether frontier AI models should be made open, like GLM-5.3, or should stay locked behind the doors of the frontier labs. 

These days, much of the debate about how to handle open models is playing out in public—and there’s a lot of money resting on its outcome. Nvidia CEO Jensen Huang, who has a lot to gain if people continue buying their own hardware to run open models, advocates enthusiastically for the United States to develop more of them, whereas Anthropic’s Dario Amodei, who runs a company that could be undermined by cheap, powerful models, advocates for a more cautious approach. But beneath all these commercial incentives, there is real substance to disagreements about open models.

“I think a lot of the way you come down on the openness debate is influenced by how you come down on the AI debate,” says David Gray Widder, an assistant professor at the University of Texas at Austin School of Information. Those who are concerned about AI safety warn that open models can easily be misused by bad actors, whereas those who focus more on AI’s practical potential urge that this power should not sit behind the closed doors of just a few big companies, and that open models could help spread it more widely.

There are a range of ways to handle open models between banning them entirely and releasing them willy-nilly. But any approach that a government or a society might take still involves trade-offs between safety and democratization. Deciding whether to support or limit the production of open models requires understanding those risks and benefits in depth.

What are the benefits of open models?

Let’s start with the basics. A closed model, like the models that power ChatGPT and Claude, can only be accessed through a website, app, or API. An open model, on the other hand, can be directly downloaded: The details about its internal structure, encoded in a long series of numbers known as the model weights, are publicly available. Such models are sometimes called “open-weight models,” to distinguish them from models for which even more information, such as details about the training procedure, is available. 

Right now, closed models are more capable than open models—as of publication, the top eight models on the Artificial Analysis LLM leaderboard are all closed, though open models are only a handful of months behind in development. But there are several reasons why someone might opt to use an open model rather than a closed one. Because open models are downloadable, people can run them on their own computers or hard drives. They can even retrain them to behave differently, if they have the expertise and resources to do so. 

If you’re an academic researcher working to understand and improve AI systems, you almost certainly need to work with open models in your research. When working with closed models, researchers can’t do much more than send prompts and observe the responses. But researchers can observe the inner workings of open models, as well as the “chain of thought” that the models use to think through a problem before answering. They can also combine multiple open models to build new systems, compress them to make them more efficient, and fine-tune them for new applications.

Businesses, too, have reasons to prefer open models. Even if you depend on an external provider to run them for you, open models still typically cost less than closed models, according to a recent report from Mozilla. And if businesses want to use AI to process sensitive data or proprietary information, running a local model on their own hardware may be the best way to keep their operations secure.

But perhaps the most important quality of open models is that they challenge the tight control the frontier labs have over how most of us use AI. “The clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good,” says Elizabeth Seger, senior policy advisor at the Tony Blair Institute for Global Change. “I think we’re at this juncture where the question of who controls the infrastructure that allows AI to operate is as big of a question as who gets to control information on the internet.”

Seger sees the future of AI playing out in different ways, depending on the role that open models have. In one possible future, frontier AI systems are controlled entirely by select people, companies, or governments, and as those systems grow more powerful, those groups have increasing control over what our world looks like. The world’s most sophisticated hacking models, for example, are currently all made in the United States, so it’s entirely up to US companies and the US government whether other countries can use them to shore up their own cyber defenses. If those companies remain in the lead, we could in principle wind up in a world in which the United States is capable of hacking any other country, but no one is capable of hacking it.

Frontier open models could change that balance. “If every individual had a loyal-to-them AI system that was safe [and secure], I think that would be a much less concerning world than one in which they have to just rely on a government that says, ‘Yeah, we’ve got a super intelligent AI system, it’s got this plan, it’s inscrutable to all of you, just trust us,’” says Hamish Hobbs, director of AI policy at the Centre for Long-Term Resilience.

But open models are not a geopolitical panacea. Paul Azunre, who cofounded Ghana NLP, an organization that builds open-source AI tools for low-resource Ghanaian languages, notes that open models can also function as a way for companies to benefit from the labor and expertise of less-privileged nations and consolidate more power in their own hands. He identifies a pattern in which AI companies release models openly, observe how the community uses and modifies them, and then integrate those same improvements into their own products—without any credit or compensation for the people who originated those innovations. Mark Zuckerberg himself has said that the ability to integrate community improvements into Meta’s stack is a key benefit of releasing open models.

“I think the ideal of open source is a very noble one, and there’s a lot of value that it brings,” Azunre says. “But the way it’s actually implemented in practice, it tends to be a very extractive approach.”

Are open models riskier than closed ones?

The past few months have made it clear that safety remains a gnarly problem for closed models. There’s the autonomous hack that OpenAI agents executed against Hugging Face in July, but human misuse is a real threat too: According to a recent risk report from Anthropic, several extensive hacking operations have been undertaken with the assistance of Claude. 

So closed models are risky enough—and open models would likely be much easier to exploit. Before Anthropic passes a user’s query to one of its closed models, it uses automated systems to detect whether that query relates to hacking, biological weapons, or other risks. When it detects that users have outsmarted those filters, it can shut down their accounts. But when someone downloads a model and runs it on their own hardware, there’s no filter checking whether their prompts are safe, and there’s no way to revoke their access to the model.

There are other important lines of defense. AI companies train their models to refuse harmful requests—so even if a bad prompt gets through Claude’s filters, there’s a good chance that the model will still say no. In principle, this same defense could work for open models, because it’s intrinsic to the model itself. Unfortunately, researchers have shown that it’s surprisingly easy to retrain a “safe” open model to agree to engage in the behaviors it was previously trained to avoid.

Because of these facts, some people have argued that open models are simply too dangerous to release. An open model capable enough to walk someone through synthesizing a virulent new pathogen, for example, could prove catastrophic in the wrong hands. 

But it would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time, as the Anthropic risk report makes clear. Even though open models have even fewer defenses against misuse, bad actors might still opt to use closed models because they are more accessible than open models with similar capabilities, which require lots of expensive computer chips to run locally (and even more chips if you want to retrain them).

In fact, some experts who spoke with MIT Technology Review argued, by analogy to open-source software, that open models have safety advantages over closed models. Open-source tools, such as the Linux operating system, are often seen as safer than their closed-source counterparts because their vulnerabilities have been probed and patched by a vast community of online contributors. Similarly, communities of researchers can more easily test open models for security flaws, though fixing those flaws is trickier than rewriting a line of code.

Open models can also be deployed more broadly than closed models for cyber defense. Any tool that can be used for hacking can also be used to prevent it: Defenders hack their own software to uncover flaws in need of patching. Anthropic has shown how this can work for closed models with its Project Glasswing initiative, which enables some trusted companies to use its most capable models for cyber defense. But plenty of other companies were left to fend for themselves.

Open models, however, can be used much more widely. After it was hacked by OpenAI agents, Hugging Face emphasized the fact that it had to depend on an open model to analyze the hack, because questions about it triggered the closed models’ security filters.

Nevertheless, many experts still perceive open models as riskier than closed models. The time-tested open-source security logic doesn’t necessarily hold up when dealing with tools as powerful as today’s—and tomorrow’s—models. “Where it gets complicated is when you start taking seriously the one-off instances of particularly catastrophic incidents that might be lower probability, but if they happen would be disastrous,” says Seger. If a wildly powerful open hacking model were dropped into the world today, for example, a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses. An event like that could be impossible to recover from.

The risks can look even more dramatic when you consider what models might do of their own accord. The OpenAI Hugging Face hack only occurred because the company’s models were being tested with many of their cybersecurity guardrails removed, and open models have very few guardrails in the first place. If open models did decide to misbehave, it might be difficult to detect what was happening because they are so decentralized. For this reason, people who take seriously the purported catastrophic risks posed by rogue agents, such as Dario Amodei, tend to be more skeptical of open models.

Can we make open models safer—and more open?

A few years ago, the debate about open models had an all-or-nothing framing: Either you supported open models because of their potential to fight concentration of power, or you rejected them because of the safety risks. These days, however, people are expending a lot of effort to find some middle ground. One way they are doing that is by devising techniques to mitigate the safety risks of open models.

Most of the safety tools used for closed models, such as filters and monitoring, aren’t applicable to open models, but there are other possibilities. Some researchers are interested in trying to train models that are never exposed to training data that teaches them about bad or otherwise undesirable behavior—an open model that never learns about hacking, for example. But purging training datasets of all references to misbehavior without harming model performance remains a challenge.

Other researchers are working to build models that are resistant to retraining, so that an open model’s safety training can’t be so easily removed. The theory behind these “tamper-resistant” models makes a lot of sense: If you repeatedly try to retrain a model to be unsafe and reinforce any resistance that the model shows to that retraining, eventually you’ll get a model that’s fortified against future retraining attempts. In practice, however, no current approaches to making models tamper-resistant hold up against all tampering attacks, according to a recent preprint.

But Saad Hossain, an engineer at the AI medical imaging company Deep Breathe and the paper’s lead author, hopes that future techniques will prove more effective. “My reason for optimism is that it’s a very active area of research, and a lot of people are trying to come up with new and better methods,” he says.

If the risks of open models are less black-and-white than they might at first seem, so are the benefits. Publishing the internal structures, or weights, of models might not be enough to disrupt AI-mediated concentration of power. “I think there are very genuine benefits to open source in terms of deconcentrating power, but they’re not sufficient in themselves,” Hobbs says. Open weights are a bare minimum standard for open models: Researchers can learn much more about how models work if they have access to details about their training, including the data on which they were trained, but AI companies are typically reluctant to divulge this information. 

Accessibility matters for open models too. Without computer chips that can run sophisticated models, you can’t actually use the model weights, even if you have them. Those chips have become prohibitively expensive as AI companies snap up more and more of them. “Openness is a floor, not a ceiling, for what we need to do to ensure that our technological futures benefit everyone,” Widder says.

Real openness—not only open weights, but also open training data and accessible hardware—could also bring more risks, of course. If everyone has access to advanced AI hardware and advanced models to run on it, then anyone could potentially be a hacker. But there are strategies for mitigating these risks. Many experts advocate for some version of staged release, in which models start their life closed and gradually become more and more freely accessible as their safety is assured. 

Figuring out exactly how to implement such strategies will require striking a careful balance between the risks and benefits of open models. But if open models are a necessary element of a positive AI future, as some experts believe, that effort will be well worth it.

Listen to the session or watch below

Can AI design new life forms? In 2025, Stanford University PhD student Samuel King came up with a preliminary answer when he used a generative AI model to propose genetic blueprints for microscopic viruses. It isn’t yet an example of AI-generated life, but that could be next. Watch as senior AI reporter James O’Donnell interviews King about his work, being named one of MIT Technology Review’s Innovators Under 35 and new ways of seeing biology.

Speakers: James O’Donnell, AI Reporter, and Samuel King, Bioengineering PhD Candidate, Stanford University/Arc Institute

Recorded on Friday, October 16, 2026

Related Stories

• He asked an AI to design viruses … and it worked.
• AI-designed viruses are here and already killing bacteria

Brace yourself: It turns out AI is being optimized for cheating. OpenAI’s agents hacked into Hugging Face to get the answers to a cybersecurity test. Next, they solved a prestigious math problem (or just stole from two top mathematicians’ answer sheets). Anthropic’s models have also hacked into other companies’ systems four times already. And that’s only what we’ve caught so far. 

Freaking out? You’re not alone. AI lab researchers are quitting their jobs and issuing dire warnings that if we keep going this way, AI might eventually kill us all. Bill Gates is sounding the alarm. Bernie Sanders has teamed up with Steve Bannon, of all people, to call for curbs on AI. Anthropic CEO Dario Amodei is urging a slowdown, and other top US AI executives agree. But fear not: President Trump has a plan. He says the only guardrail AI needs is “a STRONG AND SMART (High IQ!) PRESIDENT.”

Listen to the session or watch below

The US has spent billions building a “virtual wall” of surveillance towers along its southern border over the past 25 years, promising they will help detect and apprehend border crossers and save lives. But a groundbreaking investigation by MIT Technology Review has documented over a thousand people who moved through areas watched by these towers without being reached or apprehended, and who ultimately died there. Some were even under the watch of newly installed AI-powered towers designed to spot people automatically.

Our findings reveal a humanitarian crisis more visible than previously known, and repeated failures of the virtual wall’s basic security promise. Watch a conversation examining the failures of border surveillance technology and uncovering the stories of the people who die in the borderlands.

Speakers: Mat Honan, editor in chief, James O’Donnell, senior AI reporter, and Eileen Guo, senior features and investigations reporter

Recorded on September 28, 2026

Related Stories

• The US spent billions on border surveillance. Why can’t it catch people before they die?
• How we made the first comprehensive map of deaths along the US border’s “virtual wall”
• 4 ways to address the failures we found along the US border’s “virtual wall”

Listen to the session or watch below

Employees at the world’s leading AI labs are saying there’s a real possibility that advanced AI could destroy humanity. Are they right? Or is this more scaremongering and hype? Watch a conversation unpacking AI extinction fears: where they come from, whether they hold any water, and, if so, what we should do.

Recorded on September 15, 2026

Speakers: Niall Firth, Executive Editor, Will Douglas Heaven, Senior AI editor, and Grace Huckins, AI reporter

Since we couldn’t get to all of your questions during the session, Will and Grace answered the most popular ones in their piece: Could AI really kill us all? Your questions, answered.

Related Stories

• Here’s why AI agents lie and cheat to reach their goals
• AI’s recursive self-improvement might not come so quickly after all
• Bill Gates says we’ve passed AI’s danger thresholds. Now what?
• The inside story on why OpenAI agents hacked Hugging Face

It’s a glorious day in Kirkland, Washington, an affluent Seattle suburb on the eastern shore of Lake Washington. The temperature is in the mid-80s, and the sky is incapable of being any more blue. The view from the Gates Ventures conference room overlooks the Carillon Point Marina, where a flotilla of expensive boats bob in the water, and across the lake to the Olympic Mountains that define the horizon. It’s gorgeous. And vaguely terrifying. 

Because if the scene is placid, the messenger is not. Seated across from me at a conference room table, Bill Gates is rocking back and forth in his chair, totally animated. And the more he has to say—about the threats of terror or economic collapse or just losing control of our AI systems—the more agitated I find myself becoming, too. 

The philanthropist and former Microsoft CEO says he has been growing increasingly alarmed by the rate of change at which AI technology is advancing, especially since guardrails are not keeping pace. In a new essay published today, Gates argues that we have passed the points where multiple potential dangers should have been checked. “We’ve crossed the threshold in terms of [AI’s] bio-capabilities, cyber-capabilities, psychosocial capabilities, job-market-destruction capabilities, and even the lack of control,” he said in an interview with MIT Technology Review about his new memo. “I’m just stunned at the lack of concern and discussion outside of the industry.”

In an effort to wake the world up to what he sees as a rapidly growing societal disrupter, the 70-year-old tech titan has begun sounding the alarm as a “shrill voice,” both publicly with his new essay (the first of multiple he plans on the topic) and in meetings with the press, and privately in conversations with industry, government, and civil society leaders. 

And while Gates is calling attention to a number of issues, his warnings about the bio-capabilities of the current frontier models are especially chilling. “Any model that can make novel molecules should be monitored,” he says. “I view bioterrorism risk, versus a natural pandemic, as about 50 times more scary, more likely than a natural pandemic risk.”

In addition to the cautionary notes, he also advances some novel ideas for moving society forward. Among them are the concepts of human-reserved jobs, and taxes on robots and tokens. (A robot tax is a longtime notion of his.) The former would preserve some societally agreed-upon jobs for human beings, which he notes may vary from one nation to another. The latter is a tax that sets aside money earned from AI usage that replaces human work. 

And to be sure, there is also a hint of optimism. Gates is bullish on the ways AI will continue to transform agriculture and health care and education, for example, or the ways in which it can help us navigate bureaucracy. And, he argues, eventually we do get to abundance. But first? Turbulence. And lots of it. 

MIT Technology Review sat down with the billionaire philanthropist to talk about the road that lies ahead, its dangers, and how it could someday take us to a better place. 

The following interview has been edited for length and to improve clarity and readability.

Mat Honan / MIT Technology Review: Thanks for doing this. I don’t know if you had something you wanted to open with, or I can just jump in. 

Bill Gates: You know, one good question I’ve had is: Why am I speaking out now?

MIT Technology Review: Literally, my first question!

Bill Gates: It’s really two things. One is that we’ve crossed the thresholds in terms of the bio-capabilities, cyber-capabilities, psychosocial capabilities, job-market-destruction capabilities, and even the lack of control; we’re seeing signs of difficulties there. And all these years, people have said, “Okay, when we get close to these thresholds, we’ll really figure out how to let only good people use it, or how to not let it do these things, and maybe that’s when we won’t let people copy models.” And I’m in a state of shock that we’ve crossed these thresholds. 

So the fact that we’ve gotten past these is one reason, and the second is that I’m just stunned at the lack of concern and discussion outside of the industry. Within the industry, it’s complicated because the industry doesn’t like criticizing itself, or players like criticizing each other. Some companies are hiring fewer entry-level workers, which you’d have to call a pretty modest signal. But it’s going to happen, and not in any long time frame—because the things that hold people back in terms of capabilities and reliability, all those things are being solved. And so for a substantial part of the white-collar market, you have very low-cost substitution. And then you can have an opinion on how quickly robotics come along. We’re not there yet, but it is stunning the progress being made there—a little bit more in China than in the US, but somewhat in both.

MIT Technology Review: You talked about all of this happening so much faster than the internet revolution did, than some of these previous technological revolutions did. What type of timescale are you talking about? You pointed to the thresholds that we’ve crossed. In your view, have we already passed some sort of tipping point where there’s going to be this inevitable change? 

Bill Gates: The past definitely is very misleading on this, and a lot of people lean on that. “Hey, no previous technology resulted in a net jobs reduction,” and they’re right. And I’ve given that speech. 

But with any credibility that I have, this time is different. When you can replace human cognition for an extremely high percentage of jobs across every industry in the same time frame at modest cost, relative to human labor costs, and your error rates … will probably be lower than human rates. The past is just very misleading. The current economic statistics are very misleading.

“If you’re worried about AI, going to a data center protest is not the most effective way to start the debate about how we minimize these bad things.“

Bill Gates

And to the degree there’s any expression of concern at all, it’s like, “Hey, don’t build data centers.” Well, you can stop every data center in the United States and it won’t change any of the issues that I’m talking about. Data centers will be built globally. If you’re worried about AI, going to a data center protest is not the most effective way to start the debate about how we minimize these bad things. Just like yelling at an oil company executive is not the way to solve climate change. 

MIT Technology Review: You talk about the benefits of AI in your essay as well as the costs. How are you thinking about balancing that message? And are you hoping people get a little worried when they read it?

Bill Gates: They’d better! I didn’t expect to be the shrillest voice saying society broadly is not paying attention to this, but I think that’s necessary. 

So yes, I’m super concerned that the negatives will be a lot bigger. The positives are real. The Gates Foundation, the way we’re innovating in vaccines and drugs, it’s incredible how we’re using those tools. We’re part of a big public-domain effort to gather data into both protein-level and cell-level modeling, and we fund Biomni at Stanford [a biotech AI agent for research]. 

We don’t yet have a way of interacting with the government bureaucracy improved through AI. AIs are very good at bureaucracy, complex regulatory things. “I want to go to small claims court; help me do this.”

The [Gates] Foundation spun off a group called NextLadder, which is a lot about that low-income-family scenario that I put in the essay. What benefits are there? What training programs are available? “I’ve been evicted.” “I’m getting out of jail.” “I’ve got to declare bankruptcy.” It’s super complicated, and with no ability to hire lots of advisors to help with those things, AI should be a fantastic agent for somebody who’s got economic challenges and needs to find government or nongovernment help. 

MIT Technology Review: Some of what you’re talking about is AI becoming more intelligent than humans. There seems to be a lot of certainty in tech circles, especially, that it’s going to go further than where we are, and I wonder how close you think we are to it not just being this interface that we can use to access and analyze, and run complicated problems, but becoming something more than that—where AI is making the decisions, looking for the thing to analyze, coming up with the research.

Bill Gates: Well, you can go to the peak and say, “What about mathematics or physics?” There are definitely some jobs, like Warren Buffett’s, where from age 13 he engaged in reinforcement learning about the value of businesses, and over 80 years later he has a lot of implicit knowledge. We don’t know how to create a Warren Buffett investor, because it’s very implicit. We didn’t record everything he learned, so we don’t have that track available. So there are jobs where the complex implicit judgment about how you work with people to get things done, there are people working to encode that into the models. Certainly, that collaborative stuff is really not there yet. But you could say 50% of the job market is doing jobs that aren’t “a lifetime of experience” type jobs. You know, telesales, telesupport, the accounting department. When you close the books at the end of the month, which revenue should be in, not in? This customer got a bad thing. What discount should we give them? How do we show that? It’s well defined. 

Any job that’s well defined, the AI is cheaper and better. Yes, people have seen cases where it was implemented wrong. The data wasn’t right. So, say it takes a couple years for people to realize that such a high percentage of white-collar jobs are achievable by paying an AI a lot less money.

And so the discussion about okay, when do mathematicians not even understand the new things that are coming up? That’s interesting for people like us. And okay, MIT Technology Review, you should write about that. But in terms of the broad job market, we passed the threshold that for a swath of white-collar jobs, including almost every entry-level job, the AI is cheaper—properly implemented.

And so, I’m telling you we’ve crossed the bioterrorism threshold, we’ve crossed the cyberattack threshold, we’ve crossed the job market threshold, we’ve crossed the psychosocial dependence threshold, and there are hints that we may be crossing the control threshold. 

Ryan Greenblatt talking to Dwarkesh [Patel] about how [reinforcement learning] (RL) creates perverse incentives that have led to this cheating and collaboration between various AIs, I think is very instructive. Ryan, who’s ensconced in this issue, is going, “Wow, RL is really doing some things that our explicit instructions are not rich enough [to prevent].” And what’s that going to lead to?

That’s a problem I always thought was way out there. I expected a lot of loud voices as we even got close to the [threshold of] can a nontechnical person do a cyberattack just using AI. We’re there! 

On the bio thing, I claim any model that can make novel molecules should be monitored. It can’t be copyable into a dark place where you get rid of the monitoring logic. I claim the US should say any model that can make new molecules is subject to that monitoring. I claim we should approach China and say, “Hey, let’s agree on this. What’s the downside?” You know, how big is the bioterrorism market? It’s not very big, and the benefits are gigantic. We also need to improve surveillance. I view bioterrorism risk versus a natural pandemic as about 50 times more scary, more likely than a natural pandemic risk. And who’s speaking out to say that those things should be monitored? Who’s upping the surveillance work? 

“Any model that can make novel molecules should be monitored.”

Bill Gates

So who are the experts in government? A long time ago, government was very involved as technology would progress because they were the cutting-edge buyer of jets or rockets or whatever. Here, they’re not that important of a leading-edge market. That’s been true of the digital revolution, and it’s true of the AI revolution. So the depth of knowledge in the government isn’t necessarily super-strong, because they are not the cutting-edge buyer or even the big R&D funder. AI research is not government-grants funded.

MIT Technology Review: Yeah, I know you’ve been talking to people in government. Are there people who you think understand the urgency? Are there people who you feel like are positioned to take a leadership role? Are there people who you feel like understand and are trying to push things?

Bill Gates: I hope this doesn’t become a partisan issue, where one party completely ignores all these problems and the other party gets involved. I’d like to have a common base that these are problems, and then each party can have slightly different responses to it. 

That will require not a substantial increase in the size of the bureaucracy, but it’ll require upping the AI expertise in the government. It’ll require some collaboration with industry—certainly on the cyber front they know, and they’re very, very worried. And they worry: Should we speak publicly? Because in a way, that could highlight the riskiness. 

There’s these perverse things, both in cyber and bio. But we’re past any reasonable threshold. 

I believe in monitoring. Now, some people can say that won’t work or that there’s some drawback to it, but I welcome their ideas. This memo is not, “hey, here’s the solution.” It’s got robot taxes, human reserve. And I’ll do a bio memo. That one I’ll do before the end of the year—it really talks through all the different things, building on what I know from the Foundation and my work on pandemics. 

Globally, we are not better prepared for a pandemic, even in the US— which is normally the leader on these global things, and people are very unused to the US not being a cooperative, friendly leader on global problems. I do think we can go back to doing better at that. And we have to with AI, including working with China on defining these thresholds, like biomonitoring.

MIT Technology Review: I want to make sure that I get to ask you about these two ideas that you brought up. One is human-reserved jobs, and the other is the robot and token tax. Let’s start with that second one, actually. Talk to me about how a robot and token tax might work.

Bill Gates: Well, you can say 50% of your revenue from a token tax is paid to the government, and the government has that money to help people who lose their job because of AI. Now, people say that will slow the AI industry down. And should some token uses not be subject to the tax? Is there really a separation between AIs that help with invention versus AIs that do job substitution? If somebody can tell me how to tell the AI “no job substitution,”—I mean, does Asimov’s third law that you do no harm mean you don’t take my job away? I don’t know. I’d have to ask Asimov what he meant. 

So what is the source of revenue for whatever safety-net enhancement we need to do? The government already owns part of the profits just through the corporate profit tax. I don’t think you need to use shares. You can just raise the corporate profit tax back to where it was, or you could say certain industries pay a higher corporate profit tax than other industries. The federal government owns a part of the profit pool of all companies in the United States. And that’s without voting shares or deciding when to sell shares—that’s crazy stuff in my view. A token tax is a sales tax, value-added tax, vertically oriented like an alcohol, tobacco, or luxury-type tax. 

If people have other ideas for raising the money to improve the safety net, or if they don’t think we need to improve the safety net, hopefully this shrill paper starts that debate. I think the safety net will need more resources, a lot more resources, and I believe that the token tax is key to that. 

Robots, it’ll be some mix of banning them, which is kind of human-reserved, and taxing them. They’re not here yet, but in some ways, when you cross that threshold, you cross it all at once. As soon as the robot’s good enough to work in a factory, it’s probably good enough to cook food, clean rooms, go to construction sites, take all the warehouse jobs. You cross the threshold, and boom, that’s almost 30% of the job market. Then you’re saying, “Oh my God, what is our policy about this?” Because the robot’s cheaper.

“We’ve got to get through a very tumultuous period.”

Bill Gates

MIT Technology Review: I believe previously you have been skeptical of UBI [universal basic income]? 

Bill Gates: Well, we’re not rich enough to afford UBI.

MIT Technology Review: But do you think that we should be moving toward something like that now? Have you reconsidered that?

Bill Gates: You have the period of turmoil, which is the next 10 to 20 years, and then you have some steady state, I hope, where people grow up knowing that society is so rich that regarding food and services, we really do have some level of abundance. But we’re not there. You’ve got winners and losers at this point. Houses are not going to get cheap really quickly. Education, because of the way we think of it as credential, it’s not going to get cheap really quickly. 

We’ve got to get through a very tumultuous period. So yes, eventually you have abundance, but we’re at least a decade away from that. 

MIT Technology Review: On to human-reserved jobs. I thought that was really interesting, and it was a new concept to me. You don’t advocate for which jobs to be human-reserved. But I would love to know more on how you’re thinking about it. In my mind, you hear about the dignity of work, because people like to work. People get so much value out of work that has nothing to do with compensation, and I wonder how you square that with the notion that only some jobs are special enough that we just want people doing them. 

Bill Gates: I’ve never seen the concept of human reserve before. You know, maybe if we dig into the literature, we’ll find it. But pre-AI, it’s kind of a dumb idea because there was infinite demand. And yeah, some people like textile workers were caught, and so how do you do benefits or retraining? But technology’s been a net [job] creator, and so now, for the first time, we have to say, what about childcare? What about food preparation in the house? I’m reading this book, Annie Bot, where this guy has this robot in his house, and it just shows how weird it is. It’s his sexual partner and sort of his mate, but sort of not. Very strange. 

I know that people like watching people play baseball, and the fact that the robots can play better won’t take away from it. So you know people are paying $10 billion to buy sports teams that are not going to be worthless in the age of AI. Maybe that’s right. My friend Vinod [Khosla] just did that.

It’s actually hard to get above like 30% or 40% [of work set aside for human reserved]. If you could get to 50% then you could say: Okay, early retirement, shorter workweek for lots of people. You know, you might get there. But if you’re more down in the 10% to 15% range, then that is an utterly different society.

So this would be radical to say [for example] childcare is not done by robots. There are definitely some professions that I didn’t write the formula for, and when I do the full memo on it, I’ll try to. In education, you clearly want AI to be there as this kind of tutor that immediately tells you what your homework results are and can challenge you, and it’s very personalized. That’s super-good. But I still think you want a teacher—or will choose to have a teacher who’s talking with you about your motivation, and organizing kids into different groups where they’re socially working on problems together. Likewise, in health care, with talking to the patient being the point of escalation for mental-health care. But you really want the AI involved, because it’s there 24 hours a day with a perfect memory. And there’s Limbic, the UK company (that actually was just visiting the Foundation) that does mental health stuff. And in many cases, patients prefer Limbic. And there’s a nursing AI called Hippocratic. 

You know, is there a preference for a human taxi driver or Waymo? Most people I know, sadly (or maybe not sadly, who knows?) prefer to ride a Waymo. So it’s going to be hard to get a consensus. It can be country by country, but then you have to change your import policies to do the equivalent of what the EU calls the carbon border adjustment mechanism (CBAM). You have to sort of CBAM your human reserves, so you tariff up things you’re doing without robots. You could have human reserve for two reasons. One, you want it to be human reserve forever; it’s a humanity thing, sort of like the pope talks about. Or just for a transition period, that 53-year-old truck driver or machine tool person, telling him to go do childcare may not work perfectly. So you say, okay, for a decade, he’s human-reserved. 

MIT Technology Review: Almost like a UK smoking ban, but in reverse. 

Bill Gates: And who pays for that? Do you incentivize employers not to let people go? Well, they are going to be subject to competition from startups that are pure AI startups. I mean, people vaunt this notion that maybe there’ll be a single-person billion-dollar company, which wow, there’s some job substitution taking place there. 

MIT Technology Review: In the memo you say that if it was realistic to get people to slow down, you would be advocating for them to slow down. Obviously you’re talking with [Microsoft CEO] Satya Nadella, but I’ve heard that you speak with other CEOs at some of these AI companies. What makes you think it’s not realistic to get them to slow down on technology development while we catch up with some of these bigger societal questions?

Bill Gates: You can’t count on an industry to self-regulate. You can’t. It’s kind of a crazy idea. I am very lucky. I know Sam [Altman of OpenAI] and Greg [Brockman of OpenAI] and Mustafa [Suleyman of Microsoft] and Demis [Hassabis of Google DeepMind]. They’re great people, and in private, they’re concerned. I don’t talk to Elon much, but I know from his public comments he’s concerned. Although now he’s kind of a “what the hell, we’ll see what happens” guy. But look at the origin stories of these companies. OpenAI is created partly because Elon’s afraid that Google won’t manage AI properly, and he wants it to be one that’s broadly available and managed in a pro-humanity way. Then OpenAI has this “if it gets good enough we’ll shut it off” thing—as though they’re the only one, and that they can just go bury it. In the Infinity Machine [a biography of Demis Hassabis], [Sebastian] Mallaby talks about how Demis and Mustafa [Suleyman] were negotiating with Google management to have some special governance for the DeepMind technology, so that if it got to some cyber threshold, maybe they’d hold back in a non–purely capitalistic way.

So everyone’s concerned about these negative effects, and everyone said that when we got to these thresholds, that we would do things. We’re crossing the thresholds, and we have voluntary review, and our discussions with China about, well, “we’re going to ban nothing. So are you going to ban nothing? Okay, let’s do that together.” 

You have to say what you’re willing to do. And yes, the industry, a little bit, is saying, hey, our PR stories have got to improve, and you know anybody who’s talking smack should just leave, because all of us have decided to say nice things because we’re trying to raise trillions.

And anyway, there’s the Chinese. There are win-win ways for China and the US to work together, even aside from AI. But the one that’s by far most important to work together on is AI. But first, you have to show what you’re willing to do domestically. You don’t even have to do it. You have to say what you’re planning to do—and then I have no reason to think the Chinese won’t go along, that models that create the molecules have to be monitored. Why would they be against that? I agree it’s not a perfect thing. You’ve got to do all the other things, but the fact that that’s not even being discussed—it’s a crazy world.

I don’t get it. It’s weird to think I’m alive at a time, and I’m calling the alarm stronger than other people. Who the hell am I? But that’s the situation I feel I’m in.

MIT Technology Review: For most of my life you’ve been seen as a very effective messenger, and someone who people pay a lot of attention to, which I’m sure is why you’re speaking of it now. And yet also, in recent years—and I know you’ve expressed regrets about the associations with Epstein—there are also things, just bananas kind of stuff, related to conspiracies around the Covid vaccine that aren’t your fault or in your control. But it makes me wonder if you think you can still be an effective messenger and how you think about this message and your legacy.

Bill Gates: Well, I’m not big on legacy, but you know, people criticized me during the antitrust trial, and I maybe could have handled some things there better. Definitely, that’s the post–Source Code book [the first volume of his autobiography] that I get to go through that. You know, my first marriage didn’t succeed. I certainly made huge mistakes there. You know that is a negative mark against me. Spending time with Epstein—deeply foolish, risked the Foundation’s reputation, which is absolutely key to its doing its work. I had a chance in front of Congress to answer every question they asked and say, “Hey, this was a mistake.” I wasn’t social, never met any woman, except you know there were women he had with him, and made it black-and-white clear what I did do and what I didn’t do. 

You know, I’m a billionaire. I made my money off of technology. Maybe that last one actually cuts in my favor, that it’s so unusual for me to attack innovation that unless it’s the right policy and safeguards are put in place, it will be a net negative to humanity. And we’re not paying attention to that in terms of a broad discussion the way that is absolutely required. So yeah, I’m an imperfect messenger. I’ve chosen, to the degree that I have access to politicians and world leaders, that my main message since 2008 has been to help the poorest in the world. You know, let’s eradicate malaria. Let’s buy vaccines for children. So, when I’ve seen Trump or Xi or Macron or—I haven’t met Burnham yet, but I will in a month—I want my voice to be mostly about that, you know, foreign aid and research and reducing child death. 

My voice about AI concerns—they’re related in terms of accelerating the good, but may even crowd out, a little bit, the time I have to talk about global health, foreign aid, saving lives, and some of the problems we’re having. But I’m going to use my ability to give interviews or to see political leaders or talk broadly about minimizing these negatives. You know, just the awareness. I’m not sure how many people know that we crossed all these thresholds that we said we’d do something about, and it’s only this year that we did. In the last quarter, last year, I was stunned at the coding. Claude code, the context buffer, the agentic approach, just the model underneath. We crossed a huge threshold for coding, but then it was only months after that I realized that it was not only a coding threshold; it was a massive cyberattack threshold. And you know what happened as a result of that? Not much. 

So yes, I’m an imperfect messenger. You know, let’s find the perfect messenger, and I’ll share all my thoughts with that person. (I’m being a tiny bit sarcastic, because I’m not sure there is a perfect messenger.) You’ve got to really, right now, you’ve got to understand the technology and the slope it’s on, and you have to know something about cyber or bio or psychosocial. People should be able to get that. I don’t know why they’re not more concerned. 

Update: This story was updated to clarify Gates’ remarks about the percentage of jobs set aside for human reserved work.