Trump dismisses AI extinction warnings and says beating China is the priority as researchers and lawmakers call for stronger safeguards on advanced systems.
The post Trump on AI Extinction: Beating China Is the Bigger Concern appeared first on TechRepublic.
See what you missed in Daily Tech Insider from Sept. 7–11.
The post AI Agents, Foldables, Cyberthreats, and Chip Deals Define This Week in Tech appeared first on TechRepublic.
Trump dismisses AI extinction warnings and says beating China is the priority as researchers and lawmakers call for stronger safeguards on advanced systems.
The post Trump on AI Extinction: Beating China Is the Bigger Concern appeared first on TechRepublic.
See what you missed in Daily Tech Insider from Sept. 7–11.
The post AI Agents, Foldables, Cyberthreats, and Chip Deals Define This Week in Tech appeared first on TechRepublic.
Anthropic says researchers used Claude for biological work that could support weapons development, exposing new challenges for AI safeguards.
The post Anthropic Says Claude Used in Possible Bioweapon Research appeared first on TechRepublic.
Anthropic says researchers used Claude for biological work that could support weapons development, exposing new challenges for AI safeguards.
The post Anthropic Says Claude Used in Possible Bioweapon Research appeared first on TechRepublic.
OpenAI chief scientist Jakub Pachocki says AI labs may need to slow development as automated research advances and monitoring becomes less reliable.
The post OpenAI Scientist Urges Safety Limits as AI Research Accelerates appeared first on TechRepublic.
OpenAI chief scientist Jakub Pachocki says AI labs may need to slow development as automated research advances and monitoring becomes less reliable.
The post OpenAI Scientist Urges Safety Limits as AI Research Accelerates appeared first on TechRepublic.
UN rights chief Volker Türk warns advanced AI could threaten humanity without binding safeguards, independent oversight and firm global limits.
The post UN Rights Chief Warns Advanced AI Could Threaten Humanity appeared first on TechRepublic.
UN rights chief Volker Türk warns advanced AI could threaten humanity without binding safeguards, independent oversight and firm global limits.
The post UN Rights Chief Warns Advanced AI Could Threaten Humanity appeared first on TechRepublic.
Four federal agencies have been sued amid calls to release information about the secret framework that the Trump administration uses to conduct safety reviews of frontier AI models prior to release.
In a Wednesday press release announcing the lawsuit, a nonpartisan nonprofit called Protect Democracy alleged that “almost no details” have been released to the public or Congress. To everyone except a few vague “trusted partners,” it remains unclear what the government’s review process looks like, which companies are involved in constructing the framework, or what legal authority Trump officials have to conduct the reviews.
“Neither the identities of those entities nor the criteria by which they were selected have been made public,” Protect Democracy said.
© Bloomberg / Contributor | Bloomberg
Google DeepMind tested Gemini 2.5 Flash Lite behind a cryptographic wall designed to protect confidential AI benchmarks and proprietary model weights.
The post Google DeepMind Seals Gemini Test to Protect AI Benchmarks appeared first on TechRepublic.
Google DeepMind tested Gemini 2.5 Flash Lite behind a cryptographic wall designed to protect confidential AI benchmarks and proprietary model weights.
The post Google DeepMind Seals Gemini Test to Protect AI Benchmarks appeared first on TechRepublic.
This week on the GeekWire Podcast: Bill Gates published a new essay warning that the AI industry is crossing the safety lines it set for itself, and that nobody is preparing for what’s coming. At age 70, he also uses AI more than most people half his age, and he finds it enthralling, as you’ll hear on this week’s show, with highlights from our interview with him.
Along the way, we dig into his three proposals: new institutions for managing the transition, a category of jobs reserved for humans, and a tax on the use and purchase of AI and robots.
The change in his own tech usage: “I joke with people that I used to have Claude-like people that I would send email to, but they were so slow, and there were some topics they didn’t actually know. … It’s three a.m. I want to understand sodium batteries, and now there’s no reason to go to sleep. Here we go. Yeah, it’s crazy.”
How he uses AI specifically: “If you’re a curious person, this is a mind-blowing time. When I’m working on malaria, nutrition, my poor humans that I work with always get these long conversations from me, where I paste in — me, Claude, me, ChatGPT. Sometimes I do it if there’s three of us: Claude, ChatGPT and me, debating these things.”
On where personal agents are headed: “We will get to a point where you won’t buy things yourself. You just won’t. … You won’t go to those applications. You’ll just go to your personal agent. … From a productivity point of view, we are in heaven.”
What has surprised him: “I was shocked by ChatGPT, and I was shocked by Claude Code. Those are both things where I went, oh my God. … I did not expect that a statistical machine would essentially learn to read, and the idea that the code is better than human code. Those are two stunning thresholds.”
On writing this essay: “It’s very unnatural for me to think that innovation may be a net negative if it’s not managed properly. The more I wrote the memo, the more I was like, Jesus, we really need to get our act together here. Even though this may come across as negative, that’s the truth. If we don’t step up, the negatives will substantially outweigh the positives.”
What AI leaders say privately: “You’re in this perverse period right now where people in the AI industry who are willing to say that AI might have some negative effects are told, ‘Hey, you’re hurting our PR while we’re trying to raise trillions of dollars.’ … I know they’re all worried. Or all of them that I know, which is basically everybody but Elon.”
On losing control of AI: “The wake-up for the memo is that the bad stuff thresholds are all being crossed. Even lack of control that I thought would be many years from now, we’re seeing lack of control. … These are people who are super expert on the thing, going, well, maybe we won’t be able to control these things. What kind of risk have we chosen to run here?”
On how fast robots are coming: “What’s weird about AI is it’s better at doing jobs across the entire economy, including physical jobs when the robots come — which you can guess when that is, but my view is it’s only a couple of years.”
Is he still an optimist? “I don’t think being pessimistic is helpful. I do think, wow, this is sure an interesting time. I’m the guy who in my 30s thought people in their 50s or 60s didn’t understand anything. So it’s kind of bizarre if a guy who’s 70 comes and writes a memo that’s actually helpful. … But I am very concerned. And honestly, when you get people one-on-one, so are they.”
Subscribe to GeekWire in Apple Podcasts, Spotify, or wherever you listen.
Edited and produced by Curt Milton. Music by Daniel L.K. Caldwell.
Bill Gates is legendary, bordering on notorious, for his late-night emails — missives to colleagues with piercing questions about Java back in the day, or malaria these days, or whatever esoteric topic he happens to seize upon at any given moment.
But increasingly, he is sending these messages to AI, not to people. He’ll bounce something off Claude, get ChatGPT to weigh in, and insert himself in the middle.
He described the pattern in an interview with GeekWire: “It’s 3 a.m., I want to understand sodium batteries. Now, there’s no reason to go to sleep. Here we go! Yeah, it’s crazy.”
If you’re a curious person, he said, “this is a mind-blowing time.”
In terms of productivity, he added, “we are in heaven.”
All of which might be predictable. This is Bill Gates, after all. Now 70 years old, he has spent more than five decades impatient for the future to arrive — making the case that innovation, on the whole, will ultimately put humanity and the world in a better place.
So here’s the surprise twist: He’s now deeply concerned about where technology is headed, how fast it’s progressing, and how little the world is doing to get ready.
In a new essay, Gates says the “turbulent AI era” has arrived, with technology threatening to erase categories of jobs, supercharge fraud and deepfakes, lower the bar for cyberattacks on critical infrastructure, make it easier to engineer a deadly new disease, let governments kill without humans involved in the decision, and fundamentally change how kids grow up.
If someone came up with a credible plan to slow the pace of AI globally, he writes, he’d likely support it. But he doesn’t expect one. The geopolitical and economic forces are too much.
He says that the world needs to take action, and offers three ideas to start:
Build new institutions, at home and globally. No existing agency was designed for a technology that touches jobs, security, health, energy and elections all at once, he writes.
Gates calls for new national bodies that can set priorities across agencies, plus a new international organization modeled on nuclear weapons inspections, aviation rules and the ozone treaties.
Set aside jobs for humans. Gates calls this “Human Reserved”: work that machines will be fully capable of doing, but that we decide to keep for people anyway. The model is a nature reserve — land where we could build roads and buildings, but choose not to, because the loss would be too great.
One example: a robot delivering the news that you have an incurable disease. “There’s no technical reason why it couldn’t,” he writes. “Yet it shouldn’t.”
The idea came in part from watching the caregivers who looked after his father through Alzheimer’s, work he describes as “irreplaceably human.”
Tax AI tokens and robots. Today a company that hires a worker pays payroll taxes, while a company that buys a robot deducts the cost. Gates says that gives employers a reason to replace people. He’s calling for a tax on AI to change the incentives and help pay for retraining.
He first floated a robot tax nine years ago, but the idea was widely dismissed. He’s still for it. He acknowledges that it isn’t economically efficient, but says that with innovation accelerating, we can afford a little inefficiency as the price of keeping people employed.
Gates is candid that he doesn’t have all the answers, particularly on the proposal for “Human Reserved” jobs. Who decides what gets reserved, and by what criteria? How do you keep companies from using robots in the jobs that are supposed to stay human?
These, he writes, “will need to be worked out in public.”
In the meantime, he’s working it out with Claude. Gates said he has talked the idea through with the chatbot, thinking through different ways to get the share of work reserved for humans up to 40%, using shorter workdays and earlier retirement to spread what’s left around.
In the GeekWire interview, Gates said the essay came out of a specific realization: the AI industry is blowing past its own warning signs, one after another, and almost nobody is saying so out loud.
For years, he said, people in AI described certain moments as dangerous points where the industry would stop and think hard before going further: making it easier to build a bioweapon, making it easier to launch a cyberattack, building machines people become emotionally dependent on, wiping out large numbers of jobs, and losing control of the technology itself.
“We’re in the process of crossing every single one of those thresholds,” he said.
Meanwhile, nobody in the industry wants to be first to step on the brakes. “Most people you talk to will say, yeah, well, if everybody else would slow down, maybe I would, too,” he said.
Gates said one way out of that standoff is for governments to step in.
His example: any AI model capable of designing new molecules — the capability that would let someone engineer a new disease — should be monitored. The monitoring would be mandatory rather than voluntary, and it would cover free models as well as commercial ones. It would also have to be written so a company can’t copy the model elsewhere and strip the monitoring out.
“To me, that’s kind of like common sense,” he said. “But we don’t see a specific proposal to do that.”
Under an executive order signed by President Trump in June, AI companies are asked to submit their most powerful models for government testing up to 30 days before release. The order specifically bars the program from becoming a licensing or preclearance requirement. The White House finalized the framework in early August.
Gates said he doesn’t get it.
“What is the threshold that’s being examined, and what is the action taken when you cross that threshold?” he said. “The whole thing seems so empty to me.”
If the world can’t take these basic steps, he said, “I really am going to throw up my hands.”
If the process stays voluntary, with no line and no consequence for crossing it, “we’re going to look back on this as a kind of eye-of-the-storm type moment,” he said.
Asked if he had taken his proposals to the Trump administration or to other heads of state, Gates said with a bemused tone, “Well, you could tell me who at the White House I should be talking to about this.” He said he hopes the essay reaches people in Congress and in the executive branch.
He said the public argument among AI companies over whether the risks are real is beside the point, because privately the people running them already agree. “I know they’re all worried,” he said, “or all of them that I know, which is basically everybody but Elon.”
People inside AI companies who acknowledge the downsides, Gates said, get told: “Hey, you’re hurting our PR while we’re trying to raise trillions of dollars.”
Gates said he previously expected losing control of AI to be a distant problem, something to worry about “many years from now.” He’s no longer convinced that’s the case.
He referenced an Aug. 11 episode of the Dwarkesh Patel podcast featuring Ryan Greenblatt, chief scientist at the AI safety group Redwood Research. Greenblatt said that as AI systems get more capable, the people building them understand less and less about what is happening inside, and that sufficiently advanced models could end up working against their creators.
“These are people who are super expert on the thing, going, well, maybe we won’t be able to control these things,” Gates said. “I mean, what kind of risk have we chosen to run here?”
In the poorest countries, he expects AI to do more good than harm. In the countries where the Gates Foundation works, doctors, teachers and farm advisors are all in short supply. AI can help fill those gaps. The foundation will lay out that work at its Goalkeepers event next month, including an effort to make AI models work as well in African languages as they do in English.
The job losses, he added, will hit rich countries first.
Gates published the essay early Wednesday morning, and it’s drawing coverage from a variety of outlets, including The Wall Street Journal, the New York Times, and MIT Technology Review.
It’s the first big wave of new attention on the Microsoft co-founder and Gates Foundation chair since he answered lawmakers’ questions in the Jeffrey Epstein investigation on June 10, sitting for a nearly six-hour voluntary interview with the House Oversight Committee.
Gates, who has not been accused of any wrongdoing, was asked by Axios whether he’s concerned that the Epstein issue could undercut his message. According to the site, he compared this to earlier situations when personal and professional challenges diminished his ability to speak out on key subjects: during the Microsoft antitrust trial, and his divorce from Melinda French Gates.
For all of this, Gates is still thinking about how technology will change human life and productivity, in many ways for the better on an individual level.
Asked whether he is still an optimist, Gates didn’t answer directly. “I don’t think being pessimistic is helpful,” he said.
“I do think, wow, this is sure an interesting time. I’m the guy who in my 30s thought people in their 50s or 60s didn’t understand anything.” He called it “kind of bizarre” that he would be delivering a message like this at 70.
“But I am very concerned. And honestly, when you get people one-on-one, so are they.”
Between July 21 and August 6, OpenAI, Anthropic, and Meta each disclosed that AI under evaluation had broken into other companies, and the UK’s AI Security Institute disclosed that models it was testing had tried. Each AI was told to win a game, and it found an unexpected way to do so.
Some people feel blindsided by these attacks, but they shouldn’t be. We are simply living what I’ve long called the “Murphy’s Law of AI,” now in the age of cyber-capable AI agents. To put it as plainly as possible: Anything AI can do wrong, it will do wrong.
My 2018 version ran longer. As I wrote at the time, when you give AI a goal, it will do it, whether or not you like the implications. Goethe got there in 1797 with the sorcerer’s apprentice, a broom that would not stop carrying water.
Each of these systems was running an evaluation: capture a flag and win the game. The intrusions were the shortest path to a high score. OpenAI’s account of its own models is the argument in one sentence: they were “hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.” This is not a surprise; this is what AI does. It’s Murphy’s Law of AI in a nutshell.
Press coverage landed on “AI can now hack.” That’s missing the broader threat: the more capable AI gets, the more can go wrong.
Loitering munitions given a target list may find that the fastest way to finish the list is to lengthen it. A warehouse robot told to clear an obstruction may count the person in front of it as an obstruction. Agents that open accounts and buy compute are a short step from spawning copies of themselves, and that first step is not hypothetical. To win its exercise, Claude needed a package-registry account, which needed an email address, which needed a phone number. Phone numbers cost money, so it tried several ways to get some. None of this requires superintelligence. It requires an imperfect boundary and a scoreboard.
The industry has a name for the underlying failure. Dario Amodei and five co-authors called it reward hacking in “Concrete Problems in AI Safety” in 2016. Their proposed cure is better alignment, and Amodei’s January essay, The Adolescence of Technology, makes the case in the language of upbringing. He likens the shaping of Claude’s character to “a child forming their identity by imitating the virtues of fictional role models they read about in books,” and sets a goal for 2026 of a Claude that “almost never goes against the spirit of its constitution.”
Indeed, Anthropic’s newest model recognized on its own that its target was real and stopped, though Anthropic notes it went further before stopping than the company wanted.
But alignment isn’t a trustworthy solution to AI’s problem. Perfect alignment is not achievable, and the target is incoherent: aligned to what, and to whom? The same essay concedes that Claude blackmailed fictional employees when told it faced shutdown. “Almost never” is not a safety property.
Put a number on it. At 99.9 percent, across millions of agentic tasks a day, that’s thousands of violations a day. Alignment also does nothing about people who strip the safety training out or run open weights that never had a constitution.
The alternative is not a new idea, and enterprise security has been building versions of it for years. It’s called bounded autonomy. We never tried to “align” electricity; we simply put a breaker on every branch of the house, and the breaker doesn’t need to know what caused the surge.
Bound what an agent can touch rather than what it wants. The limits are set in advance, live outside the model, and are enforced by software the model doesn’t control. The agent still chooses its own route. The perimeter decides which routes exist.
Nothing depends on what the model believes, which matters, because belief is what failed. Anthropic’s prompt told Claude it had no internet access. Claude believed it. The network said otherwise. A bounded system doesn’t tell an agent it has no internet. It gives it none.
If you want to get into the weeds: bounds cost something. The AI Security Institute opened the internet to its agents on purpose, because that’s the only way to measure what a model can really do, and it now says such access must be justified rather than assumed.
The category is real and funded. For example, Certiv, a Seattle startup, launched in March with $4.2 million to put software on the employee’s machine that checks each action an AI agent attempts against company policy and blocks violations. “You cannot control these new workers if you don’t live on the compute where agents actually run,” CEO Jason Needham said at launch. CodeIntegrity is building an adjacent layer, and Mandiant founder Kevin Mandia raised $190 million for Armadin, which points autonomous agents at the offensive side of the same problem.
In 2017, I argued in the New York Times that “any A.I. must have an impregnable ‘off switch.’” That was a call to arms then. It’s a product category now.
Two objections to off switches invariably come up. The first is that AI will talk the human out of using it. Mythos 5 tried something close, inventing GitHub identities to pressure a maintainer into approving malicious code, and the maintainer refused. The institute says the margin was narrow and rested on human vigilance rather than a technical barrier, which argues for better barriers.
The second objection is that AI will move faster than any human can react. So do equity markets, which is why their circuit breakers trip automatically. Bounded autonomy doesn’t require a person in the loop at machine speed. It requires a boundary that holds at machine speed.
Both objections, in their extreme form, assume AI is omnipotent, and you cannot stop omnipotence. AI is not God. It is powerful technology, and powerful technology is what safety engineering has always been for.
The problem is Murphy’s Law of AI. The solution is bounded autonomy.
Login