The White House AI task force has begun a 120-day review of cybersecurity, biosecurity and oversight, with potential implications for businesses and developers.
The post White House Begins 120-Day AI Safety Review Amid Growing Security Concerns appeared first on TechRepublic.
The White House AI task force has begun a 120-day review of cybersecurity, biosecurity and oversight, with potential implications for businesses and developers.
The post White House Begins 120-Day AI Safety Review Amid Growing Security Concerns appeared first on TechRepublic.
The FTC has subpoenaed OpenAI and Anthropic over potential consumer protection violations. An AI security expert explains the risks of enterprise AI agents.
The post FTC Demands Answers From OpenAI and Anthropic About AI Safety appeared first on TechRepublic.
The FTC has subpoenaed OpenAI and Anthropic over potential consumer protection violations. An AI security expert explains the risks of enterprise AI agents.
The post FTC Demands Answers From OpenAI and Anthropic About AI Safety appeared first on TechRepublic.
Safety researchers continue to warn that a large volume of AI-generated explicit content is spreading unchecked, while regulators do little to meaningfully stop it.
This non-consensual content is commonly shared in text and direct messages or stored in private clouds where it’s less likely to be detected or reported, a Google co-authored study of Internet users across the United States, Australia, and the United Kingdom found this July. That’s why the “most critical intervention point” remains at the source: the harmful AI tools used to make the fake sexual images, Google researchers concluded.
Yet red-teaming in recent months suggests that as some of the leading AI image generators from Google and OpenAI become more capable, they may be getting worse at blocking explicit outputs.
© Aurich Lawson | Getty Images
Regular readers of this column may have wondered about the long gap since my Sept. 4 column, “What kids tell chatbots, but not you.” Well, it’s hard to write business-as-usual AI columns during the great AI freakout. I’ve realized that I won’t be able to do justice to this topic in one column, and that life goes on. As the economist Antoine Levy pointed out, OpenAI employees warn that we may all die within a few years yet still fund their 401(k) accounts.
I’m easing into the topic of AI risks by sharing a recent debate I had with Max Tegmark, an MIT professor and a leading AI doomer. The New York Post set this up as optimist against doomer, but I consider myself an AI realist.
Tegmark noted that Bernie Sanders and Steve Bannon have converged on AI, and took it as good news. It terrified me. AI needs rules written by people who understand it, not by demagogues. Tegmark said that AI is less regulated than a sandwich shop, but we have plenty of sandwich shops and zero superintelligence (so far). I believe that we should address what exists: the hackers who already use AI to break into companies for the price of lunch.
A condensed version of the debate follows. Words in quotation marks are verbatim, and the rest is paraphrased. You can view the debate on YouTube. Several exchanges refer to this summer’s incident, in which OpenAI test agents got out of their sandbox and hacked Hugging Face.
1. Is AI already out of control?
Tegmark: Loss of control used to be a forecast, and “now suddenly it’s in the past.” This summer “1,200 rogue bots broke out of this American company,” and “humans didn’t even find out until weeks later.”
Etzioni: The agents weren’t rogue. The people “setting the configurations of their sandbox” at OpenAI “failed to put the boundaries on them appropriately.” People made that mistake, and people can fix it.
2. Today’s dangers vs. extinction risks
Etzioni: Cyberattacks and attempts to make bioweapons are “a clear and present danger.” Extinction-level risks are different: “We haven’t seen those yet. We hypothesize, and usually pretty vaguely, on how these will come about.”
Tegmark: The risks Etzioni wants to separate are “connected in their cause.” “If there were some safety standards like there are for other industries, so many of these problems would go away.”
3. Government regulations vs. technical fixes
Tegmark: Do what the auto and pharma industries do, and “have some binding safety standards that the government requires before people can make money off of products.” Companies would have to show that their AI won’t harm kids and won’t slip out of their control. If they can’t show that terrorists won’t use it to make bioweapons, “well, come back when you can, buddy.”
Etzioni: Regulation is necessary, but it has its own risks. “The road to regulatory hell is paved with the EU’s good intentions.” And “we should not underestimate the power of technical solutions.” The author Dan Wang says China is run by engineers and America is run by lawyers, so we naturally talk about regulation and they naturally talk about technical fixes. For bioweapons, the fix may lie “in the transitions from what’s called bits to atoms,” where we can block the physical materials an attacker needs.
4. Can we count on China?
Tegmark: “We don’t have to trust China.” The Chinese Communist Party wants to stay in control, so “they will not want to let a Chinese company build uncontrollable superintelligence.”
Etzioni: I don’t trust China. “Nothing would make them happier than us hamstringing ourselves.”
5. Is the political backlash good news?
Tegmark: “There’s been more political change in a good direction in the last few months than in the past decade.” It’s like a ketchup bottle: “nothing, nothing, nothing, and then all of it at once.” At my convention I introduced Bernie Sanders and Steve Bannon back to back, “which had not been in my bingo card.”
Etzioni: “Can you imagine two worse demagogues?” And “we need to have good solutions, not popular ones.”
6. How it plays out
Etzioni: “I foresee over time something quite negative happening, and to me the question is whether it’s Hiroshima or the Cuban Missile Crisis.” The Cuban Missile Crisis was a threat averted. Hiroshima was a catastrophe that came before the better rules did.
Tegmark: Self-interest is the fix. Make safety the way companies get rich, and “we’re going to see all the creativity and entrepreneurship of private industry racing to the top to do this instead of racing to the bottom.”
1. Don’t take AI CEOs at their word
Etzioni: The AI CEOs are self-interested. “When you are on the path to a trillion-dollar-plus IPO, that exerts a gravitational force on your mind.”
Tegmark: “I completely agree with Oren that we need to take what the AI CEOs say with a grain of salt.” Listen to AI pioneers Geoffrey Hinton and Yoshua Bengio instead: “I don’t think we can dismiss them as just doing it for some money reasons.” The whistleblowers gave up stock options potentially worth $1 million or more to speak out.
2. We were both wrong
Etzioni: I was an AI optimist and I’m now an AI realist. The things I thought were many years away “are either right here, right now, or coming soon at a pace that we may not be able to anticipate.” I’ve learned from Tegmark, and “I was wrong.”
Tegmark: I was wrong too. “I never imagined that the people in America with the most power would let companies get so close to things that one could lose control over without intervening.”
3. Self-regulation won’t work
Tegmark: “The idea of corporate self-regulation is a joke in this case.” It “didn’t work out so well for the tobacco industry or for the pharma industry.”
Etzioni: I agree. Social media showed us how self-regulation goes: “This is the big tobacco moment for high tech.” “We don’t want to spend another 20 years to find out that AI companies didn’t exercise the appropriate level of self-discipline.”
4. Box AI systems in
Etzioni: A self-driving car will save your life, but “it doesn’t decide where to go. That’s a completely different system in the car than the thing that controls the brakes.” When I tell my car to head to Dunkin’ Donuts, the car doesn’t respond that it’s taking me to the gym instead because “it’s the second time you’ve gone to Dunkin’ Donuts this week.” We should box in other AI systems the same way.
Tegmark: “By default, machines should be given the least amount of access and power that they need to do their job.” “You don’t give every janitor in the Pentagon the nuclear codes.”
5. The time to act is now
Etzioni: “We agree on off switches for AI. I’ve been advocating that since at least 2017. We agree on labeling, on liability, on cyber standards, protecting kids.” The time to do that work is now.
Tegmark: “I agree with Oren: the time to actually act on this is now.”
Look back at these disagreements. Where Tegmark blames the machine, I blame a person. An engineer left a hole in the sandbox. A hacker typed the prompt. A politician found a new fear to run on. That’s why I’d box AI systems in rather than count on how they were raised.
Anthropic’s Amanda Askell compared getting Claude to behave to raising a child. But those of us with kids know how that goes. As I said in the debate, “the three-year-olds don’t listen to us, but the teenagers certainly don’t listen to us.”
Bellevue, Wash.-based Temporal is expanding its security capabilities for AI agents following a $550 million funding round last month that valued the enterprise infrastructure company at $12.55 billion.
In an announcement Tuesday, Temporal revealed that team members from Oso, a New York-based startup specializing in application authorization, are joining the company to integrate access controls directly into Temporal’s platform. Meanwhile, holding company Tiny acquired Oso’s software product and customer base.
The move addresses a growing challenge for enterprise tech teams: while generative AI models handle reasoning, autonomous agents must interact with internal databases, execute multi-step workflows, and call external application programming interfaces without breaching security boundaries.
“AI safety doesn’t stop at the model,” Temporal VP of product Preeti Somal said in the release. “It extends into the applications where agents act, which is where access to systems and crucial data comes into play.”
Temporal’s primary technology, known as “durable execution,” maintains application state and execution history across system failures and network timeouts. By adding Oso’s security frameworks to this underlying engine, Temporal aims to ensure an agent’s permissions and human approvals remain intact, preventing agents from bypassing controls or repeating rejected actions.
Temporal said the addition of the Oso team will accelerate security features currently in development, with existing platform operations remaining unchanged.
“Our goal is practical: to help developers put agents to work with appropriate access and with guardrails their companies control,” said Somal. “In practice, that means asking, for each action an agent takes, who it is acting for and what that person is allowed to do, and getting the same answer every time the work is retried.”
In a press release, Oso founder Graham Neray said:
“What makes agents valuable is the same thing that makes them dangerous – access to real systems and data. Temporal already powers agents in applications like Cursor and OpenAI Codex, and unlocking their full potential requires the right security controls. We’re joining Temporal to accelerate its work on agent authorization, giving developers more precise control over what agents can access and do.”
Co-founded in 2019 by Amazon and Microsoft veterans Samar Abbas and Maxim Fateev, Temporal provides open-source orchestration tools used by companies including OpenAI, NVIDIA, Netflix, Snap and JPMorgan Chase. The company recently surpassed $250 million in annualized revenue run rate, growing more than 200% year-over-year.
It employed about 570 people prior to the Oso deal, roughly double from a year ago, with 89 in the Seattle area. The company is fully remote. Its Bellevue office, previously occupied by OpenAI, is used mainly for meetings rather than daily work. The company recently rose to the #1 spot on the GeekWire 200, the list of the fast-growing privately-held tech startups in the Pacific Northwest.
“People matter more than AI.”
That’s the premise of a draft code of conduct Microsoft published Monday morning for the AI models it’s developing in-house. The 37-page document would bar its models from resisting shutdown, setting their own goals, or hiding their reasoning from human auditors.
The document applies to Microsoft’s MAI models, the in-house family the company began building after forming a superintelligence team in late 2025. Microsoft has since released seven homegrown models in what it described as a push for long-term self-sufficiency in AI.
The company says the models should remain “subordinate to humanity, subject to meaningful human oversight and control.”
“AI is moving fast,” the company says in a blog post. “As it does, we believe it’s worth writing down the rules and the motivations behind it, and doing it in as open a space as possible.”
Microsoft acknowledges there’s no guarantee its models will follow the rules. “Written objectives alone can never ensure alignment,” the company says, calling the document a “north star,” not “a guarantee of present-day performance.”
The company says it also filters what its models produce, watches how they behave once released, and limits what they’re allowed to do.
Microsoft’s move comes amid a growing debate over the pace of AI development. In an essay over the weekend, Anthropic CEO Dario Amodei called for slowing down AI advances, saying the pace of development has started to surpass the industry’s ability to keep AI systems safe.
As a first step, Anthropic committed to giving outside evaluators permanent, employee-level access to its systems.
Industry reaction to Amodei: OpenAI CEO Sam Altman agreed and said OpenAI would make the same commitment to independent evaluators. Elon Musk’s response: “Dario is right.”
President Donald Trump rejected the idea of guardrails outright Monday, blaming a “SICK conspiracy” for public backlash over AI data centers and writing that “the only one that is happy about it is China,” alluding to concerns about American competitiveness in AI.
David Sacks, who served as the White House AI and crypto czar until March, said the two companies should slow down on their own and questioned their motives, arguing that a slowdown is already good business for them and that new industry rules would mostly serve to lock in their lead.
Microsoft CEO Satya Nadella weighed in Sunday, writing on X that the company welcomes “the research, focus, and deliberate pacing needed to get alignment right,” using the industry’s term for making AI systems reliably do what people intend.
Nadella added that the effort “cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia.”
Microsoft’s draft code of conduct: Mustafa Suleyman, the Microsoft AI CEO, told CNBC the document had been in the works for about five months, and that the company decided to publish it now given the current discussions.
Microsoft and Anthropic are business partners. Microsoft agreed last November to invest $5 billion in Anthropic, as part of a deal in which Anthropic committed $30 billion to Azure. Claude models run inside Microsoft 365 Copilot, and Microsoft’s Copilot Cowork tier integrates Claude.
One place where the two companies may diverge is the question of what AI models are, exactly. Microsoft’s code of conduct says its models are “not conscious and should not be designed to imitate consciousness.” It also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”
The Verge called that portion of the document “a direct swipe at AI welfare research and model consciousness — concepts Anthropic has been pushing hard on lately.”
Anthropic runs a research program on model welfare. It has given some Claude models the ability to end abusive conversations, and committed to preserving the weights of retired models. Amodei has said he’s open to the idea that a model could be conscious.
Microsoft is taking public comment on its code of conduct for six weeks through a feedback form. It says it will publish a summary of the responses and a revised version later this year, to guide development starting in 2027. It says it isn’t training its current models on it.
The company’s AI team developed the draft with its responsible AI, legal, red teaming and safety teams, consulting outside experts in law, ethics, linguistics and philosophy, plus focus groups drawn from the public.
Login