Microsoft has drafted a “Humanist AI” code requiring future models to accept human shutdown, follow non-negotiable safety rules, and remain under control.
The post Microsoft’s New AI Rules Say Models Must Never Resist Human Shutdown appeared first on TechRepublic.
Microsoft has drafted a “Humanist AI” code requiring future models to accept human shutdown, follow non-negotiable safety rules, and remain under control.
The post Microsoft’s New AI Rules Say Models Must Never Resist Human Shutdown appeared first on TechRepublic.
“People matter more than AI.”
That’s the premise of a draft code of conduct Microsoft published Monday morning for the AI models it’s developing in-house. The 37-page document would bar its models from resisting shutdown, setting their own goals, or hiding their reasoning from human auditors.
The document applies to Microsoft’s MAI models, the in-house family the company began building after forming a superintelligence team in late 2025. Microsoft has since released seven homegrown models in what it described as a push for long-term self-sufficiency in AI.
The company says the models should remain “subordinate to humanity, subject to meaningful human oversight and control.”
“AI is moving fast,” the company says in a blog post. “As it does, we believe it’s worth writing down the rules and the motivations behind it, and doing it in as open a space as possible.”
Microsoft acknowledges there’s no guarantee its models will follow the rules. “Written objectives alone can never ensure alignment,” the company says, calling the document a “north star,” not “a guarantee of present-day performance.”
The company says it also filters what its models produce, watches how they behave once released, and limits what they’re allowed to do.
Microsoft’s move comes amid a growing debate over the pace of AI development. In an essay over the weekend, Anthropic CEO Dario Amodei called for slowing down AI advances, saying the pace of development has started to surpass the industry’s ability to keep AI systems safe.
As a first step, Anthropic committed to giving outside evaluators permanent, employee-level access to its systems.
Industry reaction to Amodei: OpenAI CEO Sam Altman agreed and said OpenAI would make the same commitment to independent evaluators. Elon Musk’s response: “Dario is right.”
President Donald Trump rejected the idea of guardrails outright Monday, blaming a “SICK conspiracy” for public backlash over AI data centers and writing that “the only one that is happy about it is China,” alluding to concerns about American competitiveness in AI.
David Sacks, who served as the White House AI and crypto czar until March, said the two companies should slow down on their own and questioned their motives, arguing that a slowdown is already good business for them and that new industry rules would mostly serve to lock in their lead.
Microsoft CEO Satya Nadella weighed in Sunday, writing on X that the company welcomes “the research, focus, and deliberate pacing needed to get alignment right,” using the industry’s term for making AI systems reliably do what people intend.
Nadella added that the effort “cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia.”
Microsoft’s draft code of conduct: Mustafa Suleyman, the Microsoft AI CEO, told CNBC the document had been in the works for about five months, and that the company decided to publish it now given the current discussions.
Microsoft and Anthropic are business partners. Microsoft agreed last November to invest $5 billion in Anthropic, as part of a deal in which Anthropic committed $30 billion to Azure. Claude models run inside Microsoft 365 Copilot, and Microsoft’s Copilot Cowork tier integrates Claude.
One place where the two companies may diverge is the question of what AI models are, exactly. Microsoft’s code of conduct says its models are “not conscious and should not be designed to imitate consciousness.” It also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”
The Verge called that portion of the document “a direct swipe at AI welfare research and model consciousness — concepts Anthropic has been pushing hard on lately.”
Anthropic runs a research program on model welfare. It has given some Claude models the ability to end abusive conversations, and committed to preserving the weights of retired models. Amodei has said he’s open to the idea that a model could be conscious.
Microsoft is taking public comment on its code of conduct for six weeks through a feedback form. It says it will publish a summary of the responses and a revised version later this year, to guide development starting in 2027. It says it isn’t training its current models on it.
The company’s AI team developed the draft with its responsible AI, legal, red teaming and safety teams, consulting outside experts in law, ethics, linguistics and philosophy, plus focus groups drawn from the public.
— Marc Whitten, a former Microsoft and Amazon executive, was named president and CEO of San Francisco-based Dolby Laboratories. He succeeds Kevin Yeaman, who is retiring after leading the entertainment technology company for nearly 20 years.
Whitten spent 17 years at Microsoft, rising to corporate vice president and chief product officer for Xbox. He went on to serve as chief product officer at Sonos before joining Amazon as vice president of entertainment devices and services, overseeing products including Alexa, Kindle and Fire TV.
He later served as president of Unity Create and CEO of Cruise. Most recently, he was vice president of robotics at Meta.
— Fred Hutch Cancer Center announced leadership changes in two divisions.
Dr. Lawrence Fong was named senior vice president and director of the Translational Science and Therapeutics Division, effective Dec. 1. He succeeds Dr. Geoff Hill, who is departing the organization in December.
Fong joined Fred Hutch in 2024 as scientific director of the Immunotherapy Integrated Research Center and Bezos Family Distinguished Scholar in Immunotherapy. He previously founded the Cancer Immunotherapy Program at the University of California, San Francisco.
Dr. Andrew Hsieh, the associate director of the Fred Hutch Human Biology Division, was named the inaugural Larry and Virginia Gordon Endowed Chair in Prostate and Bladder Cancer Research. Hsieh is a physician-scientist at Fred Hutch specializing in genitourinary cancers.
— Two recent notable Microsoft AI-related exits:
Andréa Mallard is leaving her role as chief marketing officer of Microsoft AI after joining from Pinterest in January, according to Business Insider. She will stay on as an advisor until early next year. Mallard, who is based in the San Francisco Bay Area, previously served as global chief marketing officer at Pinterest for eight years.
Ece Kamar departed Microsoft Research after 16 years with the company. She was corporate vice president and managing director of the AI Frontiers Lab, where she worked on small language models and the company’s agentic AI stack. She has not announced her next role.
— Poppy MacDonald was named president of NationSwell, a social impact membership organization. MacDonald previously served as president of USAFacts, the nonpartisan civic data initiative founded by former Microsoft CEO Steve Ballmer, for seven years. A past recipient of an Uncommon Thinkers award from GeekWire and Greater Seattle Partners, she is also the former president and COO of POLITICO.
— Jeff Buhrman joined Seattle startup Tin Can as head of finance. The company is building a screen-free, WiFi-enabled phone designed to let kids connect with friends and family. Buhrman previously served as CFO of Seattle-based Sleep Doctor for more than four years.
— Susan Loosmore was confirmed to the Major League Baseball Stadium Public Facilities District board, which oversees T-Mobile Park. The King County Council approved the appointment Aug. 25. Loosmore spent more than 17 years in executive leadership at T-Mobile and previously served as chair of the Seattle Metropolitan Chamber of Commerce.
— Seattle-based SecureW2, a passwordless security company, named Martin Musierowicz as president and Mark Packham as chief marketing officer.
— Vancouver, B.C.-based Integrated Quantum Technologies, an enterprise AI infrastructure company, appointed Husam Fezzani as CEO. He succeeds Alan Guibord, who moved to chairman. Fezzani spent nearly 30 years at HSBC, where he held senior technology and engineering leadership roles including global engineering head for the bank’s Commercial Technology Division.
GeekWire is profiling over the next few weeks some of the people and teams that are shaping the evolution of Microsoft in what we’re calling its “Microsoft 2.5” era.
From AI Frontier Lab to Frontier Ecosystem: Microsoft got a foothold in AI thanks largely to its partnership with OpenAI. But that’s not the way it is planning to continue growing its AI business.
Inside Microsoft AI (MAI), the Microsoft Superintelligence team is focused almost entirely on building its own frontier-level models. That team already has developed a handful of home-grown offerings, including MAI-Code-Flash for writing code faster; MAI-Cyber-Flash, a cybersecurity model; and MAI-Image, a model for creating images.
The head of the Superintelligence team is Ali Farhadi, corporate vice president of AI. Farhadi, who joined Microsoft five months ago, is also a professor at the University of Washington, where he has worked for nearly 15 years. He was previously CEO of the Allen Institute for AI (Ai2) and before that was an AI and machine learning leader at Apple for more than three years, after it acquired his startup, Xnor.ai.
When he joined Microsoft, Farhadi said in a LinkedIn post that he believed “Microsoft has all the pieces to win in this AI race: data, search, coding, infrastructure, agents, software and the world’s biggest Fortune 500 companies taking dependencies on Microsoft every day.”
Farhadi elaborated on that in an interview with GeekWire this week. AI is shifting from a “Frontier Lab” era to a “Frontier Ecosystem” era, he said. It’s no longer just about training models; it’s about integrating the models with enterprise data, platforms, distribution systems and customers in a trusted way.
The next battlegrounds in AI will be around cost, reliability, specialization, and deployment at scale, rather than simply building larger models that beat others in benchmark scores, he said.
“If you look around, there are not that many places to have all these missing pieces together at scale, especially if you add the element of trust to it,” Farhadi said.
Cutting through the AI noise: Farhadi said his management philosophy is grounded in the importance of personal relationships, which are especially key in big organizations. People need to understand your rationale and to trust you can deliver on what you’re tasked to do, he said — an approach that has served him inside both Microsoft and Apple.
Staying on top of the flow of information while filtering out the AI noise makes prioritizing crucial. The team has “a long list of things that we believe we should be doing,” he said, but much of it stays on the back burner to maintain a “laser focus on delivering on the main mission.”
The priority is building high-quality models, both generalist and domain-specific. On the domain-specific front, Microsoft is working with the Mayo Clinic on a healthcare-specific model based on Mayo’s own clinical data, as well as Microsoft’s cybersecurity and coding models.
The thinking: For a lot of enterprise work, a narrower model beats a bigger one.
“If you can do something at [the same] quality or better quality at a fraction of a cost, it’s just a no-brainer. And having a way to specialize to domains, to industries, to enterprises is one way,” he said.
Microsoft execs have referred to this approach as a “hill-climbing machine,” meaning the ability of a model to scale and continuously improve within a specific domain. Microsoft is coupling the hill-climbing with “frontier tuning,” like it is doing with the Mayo Clinic. Frontier tuning includes customizing frontier models; keeping proprietary data private, preserving institutional know-how; and avoiding leaking intellectual property (IP) into shared models.
“We all thought that IP is your data,” Farhadi said. “But we learned that IP is also how you work.” And that’s why safeguarding these elements is so crucial.
Open all the things? Farhadi led an expansion of open-source AI development at Ai2, the Seattle-based institute founded in 2014 by the late Microsoft co-founder Paul Allen. While Microsoft has contributed to the open-source community on various fronts, including AI tooling, it hasn’t open-sourced its frontier models.
Farhadi said he personally remains “a big advocate of open source,” but noted that the industry has changed since his Ai2 days as there are now more credible Western open-source models and businesses forming around them.
He didn’t rule out Microsoft doing something in open-source models, or the somewhat less-open “open weights” area, but there’s seemingly nothing happening on that front in the near term.
In the coming months and beyond, the focus of Farhadi’s team is helping Microsoft turn into a Frontier Ecosystem by building cutting-edge AI capabilities; helping enterprises create their own tuned versions of them; continuously improving models; and making sure customers keep control of their own destinies and data.
Success for Microsoft’s Superintelligence team has nothing to do with the idea of Artificial General Intelligence (AGI) which OpenAI, Anthropic and others have positioned as their ultimate goal over the years. In fact, when I asked Farhadi about AGI, he said, “I don’t understand what that means.”
Don’t worry, Ali. You’re not the only one.
Login